Pipeline order and reproducibility

Objectives

• State and justify a canonical order: load, montage, bad-channel detection, filter, interpolate, re-reference, ICA, epoch, reject
• Script the pipeline with a config file
• Log every rejection and emit a per-subject QC report
• Pin versions and seeds

Why this matters

By now you have every step of a preprocessing pipeline and a reason for each one. What you do not yet have is an order, a way to run it on sixty subjects without touching anything, and a report that lets a stranger check what happened. Those three things are what turn a sequence of defensible choices into a result somebody else can reproduce — and the evidence is that preprocessing choices move published effects, so the record of those choices is part of the finding.

Concepts

The canonical order, and the reason for each placement

load → montage → bad-channel detection → filter → interpolate
     → re-reference → ICA → epoch → reject

Each arrow has an argument behind it:

1. Load first, obviously — but “load” includes setting channel types and reading the metadata (L2.1). Everything after this assumes the file has been understood, not merely opened.
2. Montage before anything spatial. Bad-channel criteria, interpolation, re-referencing and topographies all need positions, and a montage attached late silently invalidates whatever ran before it (L2.2).
3. Bad-channel detection before filtering and before re-referencing. A bad channel that joins an average reference contaminates every channel; a flat channel drags the average toward zero. Detection is also easier on data that has not yet been smoothed by a low-pass, because high-frequency noise is one of the criteria.
4. Filter on continuous data, before epoching, so that edge artifacts sit at the ends of the recording rather than inside every epoch, and with boundary annotations respected (L2.4). This is also where the two-pass split happens: a 1 Hz copy for the ICA fit alongside the 0.1 Hz analysis data.
5. Interpolate after detection and after filtering. Interpolating before the filter means the filter operates on reconstructed values; interpolating before detection is circular. Every interpolated channel costs one rank (L2.2) — start carrying that number here.
6. Re-reference after interpolation, so that the average is taken over a complete, good channel set, and after reconstructing any absent online-reference channel (L2.3). Costs one more rank.
7. ICA after re-referencing, fitted on the 1 Hz copy with the rank you have been carrying, with a recorded seed; the unmixing is applied to the 0.1 Hz analysis data (L2.4, L2.6).
8. Epoch after all continuous-domain operations. Epoching is where events become sample indices, so it must come after any resampling.
9. Reject last, on the cleaned epochs, with criteria set condition-blind and the per-condition table recorded (L2.5).

Two honest qualifications. The order is not unique: linear operations commute arithmetically, so filtering and re-referencing could swap without changing the numbers, and reasonable published pipelines differ in where they put ICA relative to re-referencing and interpolation. What does not vary is the set of reasons: detection before anything that mixes channels, continuous-domain filtering before epoching, rank tracked through every linear step, rejection after cleaning and blind to condition. And the order interacts with the data: a recording with one catastrophic segment may need that segment annotated BAD_ before the bad-channel step, or the detection statistics will be driven by it.

Reference designs worth reading

You do not have to invent a pipeline. Four published designs, each with a different philosophy:

• PREP (Bigdely-Shamlo et al., 2015) — a standardized early pipeline: line-noise removal, robust average referencing, and an iterative bad-channel detection that re-estimates the reference as channels are excluded. It is the source of the bad-channel criteria in L2.2, and its robust-reference idea directly addresses the ordering problem between detection and referencing.
• mne-bids-pipeline — a configuration-driven pipeline that consumes a BIDS dataset and produces cleaned data, epochs, evoked responses and HTML reports, with every choice in one Python config file. It is the closest working model for what you will build.
• HAPPE and Automagic — automated pipelines aimed at developmental and large-cohort data respectively, both of which fold quality metrics back into the output as per-subject numbers rather than leaving them in a log.

Read them for their decisions, not to copy them: each encodes a position on the same questions you have been answering, and seeing three different defensible answers is the fastest way to understand which parts of your own pipeline are choices.

Configuration, not code edits

The rule is that two runs differ only by their configuration file. One YAML (or Python) file holds every parameter — subject list, montage, filter cutoffs, detection thresholds, reference, ICA method and rank policy, rejection criteria, the random seed, the output directory — and the code reads it. Nothing is typed at a prompt, nothing is commented out to switch behaviour, and no parameter appears in two places.

seed: 20260917
montage: standard_1005
filter: { l_freq: 0.1, h_freq: 30.0, method: fir, phase: zero }
ica:    { fit_l_freq: 1.0, method: picard, n_components: rank, seed: 20260917 }
reference: average
reject: { eeg_ptp_uv: 100, flat_uv: 1 }

The practical test: can you reproduce last month’s result by checking out the config file? If the answer requires remembering anything, the pipeline is not configuration-driven yet. The pipelines/ package built by this lesson’s notebook implements exactly this — run_subject(config, subject) — and the same structure is the capstone C2 deliverable.

Logging: every rejection, every time

A run log is not a debug convenience; it is the evidence. Each step records, at minimum: its name, its parameters as actually used (after defaults are resolved), its duration, and what it did — which channels were flagged and by which criterion, which components were removed and why, how many epochs were rejected per condition and for what reason. Written per subject as machine-readable JSON, it is also what the QC report renders and what the group-level sanity checks read.

Two rules make logs trustworthy. Log the resolved values, not the requested ones — “high-pass 0.1 Hz, FIR, length 5281 samples, transition band 0.1 Hz” rather than “filter: default”. And log failures as data, not as exceptions that stop the loop: a subject whose ICA did not converge should appear in the report as a subject whose ICA did not converge.

The QC report

One HTML page per subject, readable by someone who did not write the pipeline. The minimum contents, which are also the C2 rubric:

• Bad-channel table: which channels, which criterion fired, in what fraction of windows, and which were interpolated. The resulting rank.
• ICA components removed, each with its topography, its class, the evidence, and the ICLabel probability where available — plus the total removed and the rank after cleaning.
• Percent data rejected per condition, with counts, so condition-biased rejection is visible without asking (L2.5).
• Filter settings and justification: the cutoffs, the type, the length or order, zero-phase or causal, and one sentence per filter saying what it is for.
• A run log with package versions, the seed, the configuration hash and the wall-clock duration of each step.
• Enough figures to see the recording: a PSD before and after, the ERP before and after cleaning, and the continuous data around one flagged segment.

A report nobody reads is worth little, so make it scannable: the numbers that would make you distrust the subject go at the top.

Seeds, versions and the numbers that move

ICA initialisation, any subsampling, any permutation test (L3.7) and autoreject’s cross-validation folds are all random. Set one seed in the config, pass it into every stochastic step, and record it. Then pin versions: the exact MNE, NumPy, SciPy, autoreject, mne-icalabel and (if used) ASR implementation versions go into the run log, because their defaults change between releases and a default change is a silent pipeline change.

This is more than hygiene. (Robbins et al., 2020) shows that reasonable variations in preprocessing move the results of real EEG studies — the same data, defensibly processed different ways, yields different answers. The response is not to find the one true pipeline but to fix yours in advance, record it completely, and, where a choice is genuinely arbitrary, report the result across the alternatives (the multiverse approach of L6.4).

The data behind this lesson

• The notebook builds the pipeline skeleton and runs it on ds-eegbci (via its CC0 BIDS mirror, OpenNeuro ds004362) and on ds-erpcore P3 (CC BY 4.0, open, per-subject downloadable). TODO(confirm): the author mirrors the ERP CORE entry into the catalog registry and signs off the dataset page (§10.11 item 8).
• ds-eegbci’s documented defective subjects (S088, S089, S092, S100, and often S038 and S104) are the worked example of an exclusion that belongs in the configuration file with its reason, not in a comment.

Explore

This lesson has no widget; the pipeline is the object of study. Before you write any code, do this on paper for your own data:

• write the nine steps in order and, next to each, the one sentence that justifies its placement;
• mark every step that is not linear (interpolation with a changed channel set, rejection, ICA application after component removal) — those are the places where order changes the numbers, not just the intermediate views;
• track the rank down the list: start at the channel count and subtract as you go;
• mark every parameter that will live in the config file, and every parameter you are about to hard-code — then move the second list into the first;
• for each step, name the line it will write into the QC report.

Practice

Pipeline order and reproducibility: run_subject driven by one configuration file, the run log, and a per-subject QC report nb-2-8-pipeline

Level 2 ~3 min

notebooks/L2/nb-2-8-pipeline.ipynb

Downloads from ds-erpcore.

Open in Colab Download Read it here

The notebook builds the pipelines/ package skeleton: a configuration schema, one module per step, run_subject(config, subject) returning cleaned data plus a run log, and a QC HTML report. It runs end to end on a small subset and prints the run log so you can see what each step recorded.

Exercises

Exercise ex-2-8-pipeline-order

Put in order

A colleague sends you these nine pipeline steps in scrambled order. Put them in the canonical order this lesson defends, from first to last.

1. 1. Attach the montage ↑ ↓
2. 2. Re-reference ↑ ↓
3. 3. Reject epochs ↑ ↓
4. 4. Detect bad channels ↑ ↓
5. 5. Load the file and set channel types ↑ ↓
6. 6. Interpolate the bad channels ↑ ↓
7. 7. Fit ICA and apply the unmixing ↑ ↓
8. 8. Epoch ↑ ↓
9. 9. Filter the continuous data ↑ ↓

Check order

Hint

Exercise ex-2-8-justify-order

Free response

Justify each placement in one sentence, and then name the two steps whose order could defensibly be swapped and say what would and would not change if you swapped them.

Your answer

Reveal model answer

Pitfalls

Pitfall

Uncorrected stimulus–trigger latency

Symptom

Components shifted by a constant; latencies differ between labs.

Cause

The trigger records when the stimulus computer sent the stimulus, not when it reached the subject. Between the two sit the monitor’s refresh cycle and its response time (a stimulus sent just after a refresh waits for the next one), graphics buffering, audio-driver and amplifier latency for sounds, and the time the acquisition system takes to register a software marker. Each contributes a delay th…

Detect

• Record a photodiode taped to the screen (or a microphone at the speaker) as an extra channel in a pilot session, and measure the delay from each trigger to the onset of the sensor signal: the mean is the offset, the spread is the jitter. - Check the acquisition metadata and the lab’s documentation for a stated offset; if none is stated, assume the offset is unknown, not zero. - Compare an early…

Fix

• Measure the offset with a photodiode or microphone for each setup, and store it with the dataset (EEG-BIDS has a place for it: the StimulusOnsetDelay style of metadata in the events sidecar, or a documented correction in the dataset description; TODO(confirm) the exact BIDS field the author wants named). - Shift the event onsets by the measured offset when loading (mne.Annotations or the events…

Full entry with example →

Pitfall

ICA after interpolation without rank adjustment

Symptom

Duplicated or degenerate components; ICA fails to converge.

Cause

ICA is a change of basis. It can only find as many independent components as the data has independent dimensions — its rank. A recording’s rank is not its channel count if anything linear has been done to it:

Detect

• Compute the rank explicitly and compare it with the channel count (mne.computerank(inst, rank=‘info’), or the arithmetic above done by hand). A gap you cannot account for is the finding. - Look at the singular values of the channel covariance: a sharp drop to near-zero after k values means the true rank is k. - Inspect the decomposition for near-duplicate pairs (high absolute correlation betwee…

Fix

• Carry the rank through the pipeline as a number, updated at every linear step, and log it (L2.8). Start at the channel count and subtract: one per interpolated channel, one for an average reference, one per projector. - Pass it explicitly: ICA(ncomponents=rank, …), or MNE’s rank argument, rather than relying on a default derived from the channel count. - Prefer to detect bad channels, re-refe…

Full entry with example →

Pitfall

Electrode bridging

Symptom

Two neighbouring channels are near-identical; interpolation "works".

Cause

Conductive gel spreads between two neighbouring electrodes and connects them electrically, so both record the same potential. It is most common with dense caps (short distances), with too much gel, with sweating, and with long recordings in which gel migrates. Because bridged channels are not noisy, the usual bad-channel detectors, which look for variance, amplitude or spectral outliers, do not s…

Detect

• Compute the electrical distance between every pair of channels (the variance of their difference signal over the recording); bridged pairs stand out as near-zero values in an otherwise broad distribution. MNE provides mne.preprocessing.computebridgedelectrodes for this check (TODO(confirm) the citation for the electrical-distance method the author wants listed in the reading list). - Look at a…

Fix

• At acquisition: use less gel, check impedances for suspiciously low neighbouring pairs, and re-prepare bridged sites before recording. - Offline: mark bridged channels as bad before computing an average reference, interpolating, or running ICA; if both channels of a pair are bridged to each other, keep one and interpolate the other only from unbridged neighbours, or drop the pair. - Keep the br…

Full entry with example →

Pitfall

Comparing amplitudes across references

Symptom

Component at Pz differs by half between studies.

Cause

An EEG channel is a difference: the potential at an electrode minus the potential at whatever the amplifier subtracted. There is no absolute voltage to recover. Changing the reference subtracts a different signal from every channel, so every amplitude changes, and by different amounts at different electrodes.

Detect

• Read the methods of both studies for three things: the online reference, the offline reference, and, for an average, the number and layout of channels it was computed over. If any of the three is missing, the amplitudes are not comparable and you cannot make them so. - Re-reference your own data to the other study’s reference and repeat the measurement. That is the only honest comparison, and i…

Fix

• Fix the reference before looking at the effect, and choose it from the component and the literature you need to speak to — not from which value looks best. - State all three facts in the methods: online reference, offline reference, and the channel set an average was taken over. - Reconstruct an absent online-reference channel (as a row of zeros with a montage position) before average-referenci…

Full entry with example →

Pitfall

High-pass cutoffs that distort slow ERPs

Symptom

Artifactual early components; attenuated late components.

Cause

Slow ERP components have most of their energy at low frequencies. A high-pass filter removes that energy, and because the standard offline filter is zero-phase, the removed energy is redistributed symmetrically in time: a large positive deflection is accompanied by negative deflections before and after it that were not in the data. The higher the cutoff (and the sharper the filter), the larger an…

Detect

• Filter a single trace containing a known slow transient (the sandbox’s electrode pop, or a simulated boxcar of the component’s duration) with your chosen high-pass and look at what appears on either side. - Compute the ERP at several high-pass cutoffs (0.01, 0.1, 0.5, 1 Hz) and plot them together; an amplitude that falls steadily with cutoff, or a deflection that grows with cutoff, is a filter…

Fix

• Use a conservative high-pass for ERPs — on the order of 0.1 Hz or lower — with a gentle transition band, and state the cutoff, filter type, order or length, and direction (zero-phase or causal) in the methods. - Treat the cutoff as an analysis choice to be justified, not a default: filter as little as the data allow and remove drift by other means where possible (good recording practice, baseli…

Full entry with example →

Pitfall

Filtering short epochs

Symptom

Edge artifacts inside the analysis window.

Cause

A filter needs samples on both sides of every point it computes. In a short epoch those samples do not exist near the edges, so the implementation pads (with zeros, a mirror image, or a constant); the padded values are not data, and the filter’s response to the padding leaks into the epoch by the length of its impulse response. For a high-pass with a low cutoff that length can exceed the whole ep…

Detect

• Compare the ERP computed from epochs cut before filtering against the ERP from data filtered continuously and epoched afterwards; any difference is edge effect. - Compute the impulse-response length of the filter (FIR: the number of taps; IIR: the settling time) and compare it with the epoch length. - Plot the average of the pre-stimulus baseline across trials: a systematic slope or curvature n…

Fix

• Filter continuous data, then epoch (the canonical order in L2.8). - If data are only available as epochs, use epochs long enough that the analysis window lies more than one impulse-response length from each edge, and state the padding method. - Choose the mildest filter that serves the goal; a lower-order or wider-transition-band filter has a shorter impulse response. - For real-time or single-…

Full entry with example →

Pitfall

Rejection that differs by condition

Symptom

One condition loses 30% of trials; effect appears or disappears.

Cause

A fixed rejection criterion is not condition-neutral, because the conditions are not equally likely to trip it. Rare, attended or task-relevant stimuli provoke more blinks and more movement; conditions differ in trial length, in arousal, in how often the participant responds. So the same threshold rejects more trials in one condition than the other.

Detect

• Build the per-condition rejection table for every subject, every run: trials presented, trials rejected, percentage, and the criterion and channel responsible. epochs.droplog holds all of it. - Plot the percentage rejected per condition against the threshold, and look at the difference between the two curves rather than the total. - Compare the effect computed on the full trial set with the eff…

Fix

• Set the criterion condition-blind: estimate it on pooled trials, never per condition, and fix it before looking at the effect. Pre-specify it alongside the subject-exclusion rule. - Prefer a data-driven threshold (autoreject, fitted on pooled trials) to a hand-picked one, and report what it chose. - Report the per-condition rejection table in the QC report and in the paper — not just the total…

Full entry with example →

Pitfall

Removing brain components that carry the effect

Symptom

Effect shrinks after "cleaning"; components with alpha or posterior topography removed.

Cause

Every component you remove is subtracted from the data. If it carried brain activity, so is your effect.

Detect

• Compute the effect with and without the cleaning step. This is the decisive test, and it is cheap. A cleaning step that moves the effect is a step that needs justifying. - Count removed components per subject and look at the distribution. Large variance across subjects means the criterion is not the same criterion each time. - Review the removed set: any component with a smooth dipolar topograp…

Fix

• Decide the removal policy before you look at the effect: which classes are removed, on what evidence, and what the upper limit on the number removed is. Apply it identically to every subject. - For each removed component, record the class, the evidence from each view, and the classifier probability if available, in the QC report (L2.8). If the justification needs “probably” more than once, keep…

Full entry with example →

In other tools

In other toolsEEGLAB · FieldTrip — names only

The equivalents of what this lesson does, for a reader who works in another toolbox. Function names only: their own documentation is the place to learn how to call them.

EEGLAB

• eeghEEGLAB
• pop_runscriptEEGLAB

FieldTrip

• ft_analysispipelineFieldTrip

Names checked 2026-09-18 against EEGLAB 2026.0.0 (plugins at the versions in EEGLAB’s own plugin list) and FieldTrip 20251218.

Reading

1. Robbins et al. (2020). Sensitivity of EEG results to preprocessing. unverified
2. Delorme (2023). EEG is better left alone (presented as a contested position). unverified
3. Bigdely-Shamlo et al. (2015). PREP. unverified