Capstone — Replicate ERP CORE

Brief

Pick one ERP CORE paradigm. Starting from the raw data and the pipeline you wrote for C2 — unchanged — reproduce the published component effect and report it properly: amplitude, latency, effect size, standardized measurement error, a cluster-based permutation test, a comparison with the published values, and one paragraph on any divergence.

This is a replication, so the interesting part is the constraint. You may not tune the pipeline to the result, you may not move the measurement window after seeing the difference wave, and you may not describe the cluster test as saying more than it says. Those three prohibitions are the rubric, and they are exactly the three failure modes Level 3 spent seven lessons on. If the effect comes out smaller than published, that is a result; if it comes out larger, that is also a result; if it comes out however you needed it to, the analysis had degrees of freedom you did not declare.

Write the analysis plan first — paradigm, component, electrodes, measurement window, measure, test, exclusion rule — and keep the file. Everything after that is execution.

Inputs

• Data. ds-erpcore (Kappenman et al., 2021): CC BY 4.0, open access, per-subject and per-file downloadable, roughly 58 MB per subject per paradigm. Biosemi ActiveTwo, 30 EEG + 3 EOG electrodes in a 10-20 placement scheme, 1024 Hz, CMS reference, 60 Hz mains, no software filters applied, 40 participants per paradigm, one run per paradigm per subject. Six paradigms yielding seven components — N170, MMN, N2pc, N400, P3, ERN and LRP. TODO(confirm): the author mirrors the ERP CORE entry into data/registry.yaml (§10.11 item 8) and signs off the dataset page; the shipped asset sidecars record a licence conflict in the source — the OSF node record says CC BY 4.0, the per-paradigm component’s own LICENSE file says CC BY-SA 4.0 and its dataset_description.json says CC0 — which the author reconciles (§13 item 22). The published values you compare against come from the dataset’s own publication, which you must cite and which this site does not reproduce.
• Your C2 pipeline. The configuration file, run_subject(config, subject) and the QC report, exactly as you left them. The only change permitted is the configuration — and every value you change from the C2 run has to be named in the report with its reason.
• Lessons. L3.1 (epoch window, baseline and overlap), L3.2 (averaging, SNR and the component–peak distinction), L3.3 (measurement method, latency and non-circular windows), L3.4 (topographies), L3.5 (SME, trial counts and design confounds), L3.6 (single-trial views, if the paradigm makes one useful), L3.7 (the test and what it licenses).
• The paradigm’s own hazards. A lateralized paradigm (N2pc, LRP) obliges you to handle eye movements explicitly (pf-eye-movements-lateralized); a paradigm with rare targets (P3, MMN) obliges you to report per-condition trial counts and SME (pf-peak-amplitude-noise-bias, pf-condition-biased-rejection); a paradigm with a slow component obliges you to justify the high-pass cutoff (pf-hp-cutoff-erp).

Subset rule and runtime

Capstone notebooks run on a documented subset within the §11 limit of ten minutes on Colab’s free tier, and scale to the full cohort locally:

• The documented subset for C3 is 10–20 subjects, listed by ID in the notebook’s first cells, together with the paradigm and the reason for the selection. Select the subset by a rule that cannot depend on the effect — the first n subject IDs, or every subject that passes the pre-specified exclusion rule, in ID order.
• A FULL_COHORT switch at the top of the notebook (default False) extends the run to all 40 participants of the paradigm for local execution. The per-subject function is identical; only the number of rows in the group table changes. Report which state the numbers you quote came from, because the effect size, the cluster p-value and the comparison with the published values all depend on it.
• Each subject is roughly 58 MB, so the subset run downloads on the order of 0.6–1.2 GB. Download only the paradigm and subjects you need, cache locally, and delete raw downloads when the derived per-subject products exist.
• Per-subject products that are expensive to recompute — cleaned epochs, per-subject evoked arrays, per-subject measurements and SMEs — may be precomputed and shipped as assets so that the group-level figures reproduce without the full download.
• The notebook uses the shared helpers, contains no absolute paths, states every analysis parameter in one place with the seed, and prints in its final cell every number the report quotes.

Deliverables

1. The analysis plan, written before any group result was computed and included verbatim: paradigm, component, electrode set, measurement window and where it came from, measure (amplitude and latency), the statistical test, the subject-exclusion rule, and the trial-count requirement derived from a pilot SME curve. A note of anything that was changed afterwards, with the reason.
2. The pipeline run. The C2 configuration file with the C3 diff highlighted, and the per-subject QC reports it produced — bad channels, ICA components removed with reasons, per-condition rejection percentages, filter settings, run log with versions and seed.
3. Amplitude. The component’s amplitude per subject and at the group level, measured as mean amplitude over the a-priori window at the pre-specified electrodes, on the difference wave and on each condition separately. Peak amplitude only as a secondary measure, if the published comparison requires it, and then with the trial counts beside it.
4. Latency. A latency measure chosen for noisy data — fractional-area latency, or a jackknifed peak latency with the corresponding correction to the test statistic — with the method stated and the correction shown.
5. Effect size. A standardized effect size for the condition difference with a confidence interval, and the raw difference in µV with its interval. Not a p-value alone.
6. SME. The standardized measurement error per subject and per condition for the reported measurement, analytic where the measure is linear and bootstrap otherwise, aggregated across subjects, and plotted against trial count. The per-condition comparison is what shows whether the two conditions were measured with equal precision.
7. A cluster-based permutation test over the pre-specified channel set and time window, reported in full: the space tested, the point-wise statistic, the cluster-forming threshold and how it was chosen, the neighbour definition, the permutation scheme and count, the seed, α, the tails, and every cluster with its statistic and p-value — including the ones that did not survive.
8. A topography or map sequence for the component, with the electrode positions drawn, the reference stated, a shared colour scale in µV across panels, and the latencies chosen in advance.
9. The comparison with published values, as a table: the published amplitude, latency and effect size for this paradigm against yours, with the number of subjects behind each, and the measurement definitions of both stated so that a reader can see whether the two numbers are the same quantity.
10. One paragraph on divergence. Where your numbers differ from the published ones, the candidate explanations ranked by how much of the gap they could account for — subset size, reference, filter settings, measurement window and measure, rejection and trial counts, and ordinary sampling variability — and what would distinguish between them.

Rubric

• Pipeline reused unchanged from C2 — the same code, driven by a configuration file; every changed parameter named with a reason, and no parameter changed after the group effect was visible.
• Measurement windows fixed a priori — the window, electrodes and measure were written down before the condition difference was plotted, and the written version is in the report. A collapsed localizer is acceptable where declared; choosing from the condition difference is not (pf-post-hoc-windows).
• Interpretation of the cluster test is correct — the result is stated as a difference somewhere in the tested space with family-wise error controlled, the cluster extent is reported as a description and not as an estimate, and no onset, offset, duration or localization is claimed from it (pf-cluster-inference-misread).
• Amplitude, latency, effect size and SME are all reported, per condition where that is meaningful, with intervals rather than p-values alone.
• Trial counts are reported per subject and per condition, and the measurement chosen is robust to the imbalance the paradigm creates (pf-peak-amplitude-noise-bias).
• The baseline interval is stated, the un-baselined averages are shown, and the pre-stimulus interval carries no unreported condition difference (pf-baseline-contamination).
• The subject-exclusion rule was fixed in advance, applied blind to the effect, and every exclusion is listed with the criterion that removed it.
• The comparison table states both measurement definitions, so the published and replicated numbers are comparable quantities.
• The notebook runs top to bottom on the documented subset inside the §11 runtime limit, and the FULL_COHORT state of every quoted number is stated.
• A reader who has the dataset could reproduce every number from the report alone.

Example report structure

1. Analysis plan. Reproduced verbatim, with the date it was fixed, and a list of any deviations with reasons.
2. Data and subset. Paradigm, subject list and how it was selected, FULL_COHORT state, download size, and the dataset’s licence and citation.
3. Pipeline. The C2 steps in order; the configuration diff for C3 with one sentence per changed parameter; the group QC summary — bads interpolated, components removed, percentage rejected per condition, subjects excluded and why.
4. Waveforms. Grand-average condition waveforms and the difference wave at the pre-specified electrodes, un-baselined and baselined, with the measurement window shaded and the trial counts in the caption.
5. Topography. The component’s map or map sequence, electrodes drawn, reference and scale stated.
6. Measurements. Amplitude and latency per subject and at the group level; the effect size with its interval; the SME table per subject and per condition with the SME-versus-trial-count curve.
7. Statistics. The pre-specified test on the measurement, reported in full; then the cluster-based permutation test, reported in full, with the map and the cluster outlined; then one sentence saying exactly what has been established.
8. Comparison with published values. The table, with both measurement definitions.
9. Divergence. The one-paragraph discussion, with the candidate explanations ranked.
10. Reproducibility. Package versions, seed, configuration hash, runtime, and what a reader would need in order to re-run it.

Estimated time: TODO(confirm) — the minutes value in the frontmatter is a site-design estimate, not from the spec.