Reporting standards
Reporting filters, reference, rejection, ICA, epoching, measurement and statistics to COBIDAS-MEEG standard; figures with uncertainty; sharing code and data.
Prerequisites: L6.1 · The multiple-comparisons landscape, L6.2 · Mixed models and trial-level data, L6.3 · Effect sizes, power and precision, L6.4 · Circularity and analytic flexibility, L6.5 · Decoding as inference
1 claim on this page is unverified. TODO(confirm) marks a specific statement the author has not yet checked against a
primary source. Everything else on this page has been reviewed. Treat a marked claim as
provisional and go to the cited source rather than quoting the sentence.
What you will be able to do
- Report filters, reference, rejection, ICA, epoching, measurement and statistics to COBIDAS-MEEG standard
- Produce figures with uncertainty
- Share code and data (BIDS, environment files)
Why this matters
Everything the previous five lessons established is invisible in a paper unless the methods section says it. Whether the measurement window was fixed in advance, whether the spatial filter sat inside the cross-validation fold, whether the reference was chosen before the effect was plotted — a reader cannot infer any of it from the results, and the difference between an analysis with an error rate near α and one with an error rate near three in four lives entirely in those sentences. Reporting is not administration. It is the only channel through which the rigor of Level 6 reaches anybody.
Concepts
The checklist exists, and it is long for a reason
Community reporting standards for EEG and MEG — the COBIDAS-MEEG report ( (Pernet et al., 2020) ) and the earlier publication guidelines ( (Keil et al., 2014) ) — enumerate what a methods section must contain. They are long because the pipeline is long, and because almost every item on them is a value someone could otherwise have chosen after seeing the result.
Work through it by stage.
Participants and ethics. Number, age, sex or gender as recorded, handedness, relevant clinical characteristics and medication state, recruitment, exclusions with reasons and counts, compensation, ethics approval and consent. For patient studies, group-by-group, because pf-group-demographic-confound lives in the gap between “patients and controls were matched” and the actual numbers.
Acquisition. Amplifier class and bandwidth, cap and electrode type, electrode count and naming scheme, how positions were determined (template or digitized), online reference and ground, impedances and the criterion, sampling rate, online filters including any hardware notch, mains frequency, recording environment, and the session structure with block lengths. pf-hardware-bandwidth-ceiling and pf-notch-hole-in-band are both invisible unless the hardware filters are stated.
Events and timing. How triggers were generated, the measured stimulus-to-trigger latency and its jitter, and any correction applied. “Triggers were sent by the stimulus computer” does not tell a reader whether the latencies are comparable with anyone else’s (pf-trigger-offsets).
Preprocessing, in the order performed. Filter type (FIR or IIR), cutoffs, transition bandwidth or order, direction (zero-phase or causal), and what the filter was applied to — continuous data or epochs. Resampling rate and anti-alias handling. Bad-channel criterion, the channels flagged per subject, and whether they were interpolated and how. Reference, stated as what it is, plus the rank consequence of interpolation and average-referencing together (pf-interpolation-rank). ICA: algorithm, the data it was fitted on and any filter used only for the fit, the rank supplied, how components were classified, the criteria, and the number removed per subject with the range.
Epoching and rejection. Epoch window, baseline interval, rejection criteria as numbers, and the per-condition counts kept and lost. Per-condition, because pf-condition-biased-rejection is only visible in the comparison.
Measurement. Component, electrodes, window, measure, and — the item most often missing — where the window and electrodes came from. “From the prior literature”, “from a collapsed localizer”, “from the grand average of the condition difference” are three different analyses with three different error rates (L6.4).
Statistics. Unit of analysis; the model or test, written out; contrast coding; correction for multiple comparisons, over which family and by which procedure; for permutation tests the scheme, the count and the seed; α and tails; degrees-of-freedom method for any mixed model; effect size with a confidence interval; and the raw effect in µV. Non-significant results reported as well as significant ones.
Sharing. Data in BIDS where possible, with a data-availability statement and a licence; analysis code with a version-control reference; an environment file with pinned versions; seeds for anything random; and the derivatives needed to redraw the figures.
The same checklist, read by what each omission hides
The same eight stages, read by what stops being checkable when the item is left out. Every item below is one the stage paragraph above already lists; the right-hand column is what a reader loses without it.
| Stage | The item the stage turns on | What nobody can check without it |
|---|---|---|
| Participants and ethics | The counts, the characteristics, and the exclusions with their rule — and for patient studies, the group-by-group numbers rather than the word “matched” | Whether the groups differ in anything that also drives the effect (pf-group-demographic-confound) |
| Acquisition | Online filters, including any hardware notch, and the amplifier’s bandwidth | Whether a band you report was passed by the hardware at all, and whether a “missing” mains line was notched out upstream (pf-hardware-bandwidth-ceiling, pf-notch-hole-in-band) |
| Events and timing | The measured stimulus-to-trigger latency and its jitter | Whether your latencies are comparable with anyone else’s (pf-trigger-offsets) |
| Preprocessing, in order | Filter type, order or transition band, direction, and what it was applied to — continuous data or epochs; then the reference, and the rank left after interpolating and average-referencing together | Whether the waveform you are shown is the one the data produced (pf-hp-cutoff-erp, pf-filter-epoched-data, pf-reference-changes-everything), and whether a decomposition was asked for more components than the data holds (pf-interpolation-rank) |
| Epoching and rejection | The baseline interval, and the counts kept and lost per condition | Whether the baseline contained the effect (pf-baseline-contamination), and whether the conditions were cleaned to different depths — which the condition comparison then measures (pf-condition-biased-rejection) |
| Measurement | Where the window and the electrodes came from: the prior literature, a collapsed localizer, or the condition difference itself. The lesson calls this the item most often missing | Which of three analyses you are reading, since they carry three different error rates (L6.4, pf-post-hoc-windows) |
| Statistics | The correction for multiple comparisons — over which family, by which procedure — then the permutation scheme, count and seed; α and the tails; the non-significant results | Whether the error rate is anywhere near α (pf-uncorrected-timepoint-tests), whether a cluster result is being described as more than its extent (pf-cluster-inference-misread), and whether the test can be re-run at all |
| Sharing | Pinned versions, the analysis commit, and the seed | Whether anybody — including you, later — can get the same numbers from the same data (pf-unreproducible-archive) |
Every stage now has a pitfall to point at. pf-unreproducible-archive was the last gap and was written for
this table on 2026-10-10; its worked example is this site’s own continuous integration, where a single
undeclared dependency meant a test suite reported nothing useful for its entire life and nobody could tell.
Figures that carry uncertainty
A grand-average difference wave drawn as a single line asserts a precision it does not have. Four habits fix most of it:
- Draw a band. A confidence interval or standard error around the grand average, and — for a within-subject effect — a within-subject-corrected interval, since the between-subject variance in overall amplitude is not part of the effect’s uncertainty. Say in the caption which interval it is.
- Show the subjects. Thin per-subject traces behind the average, or a per-subject scatter beside the bar, turns “the effect” into “the effect and how consistently it appeared”.
- Draw the analysis on the figure. Shade the measurement window, mark the electrodes, put the per-condition trial counts in the caption, and state the reference, the filter and the polarity convention. A figure that cannot be matched to the methods is a figure a reader has to trust.
- Share the scale. One colour scale in µV across topography panels, with the units on the bar; separate scales invite a comparison the data do not support.
And the negative rule: do not draw the significance and then describe it in words that the test does not support. A shaded “significant” region under a cluster test is a description of the cluster’s extent, and the caption has to say so (pf-cluster-inference-misread).
Reproducibility is a property of the archive, not the intention
Three things, in increasing order of how often they are forgotten:
- Data. BIDS gives the directory layout, the sidecar metadata and the event tables a stranger needs; a data-availability statement gives the access route and the licence. Where the data cannot be shared, say why, and share the derivatives that can be.
- Code. The analysis scripts, at the commit that produced the numbers, with the seed. A preregistration’s commit hash (L6.4) is what turns “written before” into a checkable claim.
- Environment. Pinned package versions. MNE, EEGLAB, FieldTrip and every statistics package change behaviour between versions; “analysed in MNE-Python” dates a result without reproducing it.
Write the methods section before you run the analysis. Most of it is the preregistration with the tense changed, and drafting it early has an effect that no checklist review afterwards can reproduce: an item you cannot write down is a decision you have not made, and you will find those while the choice is still free rather than after the result has been seen. What remains afterwards is filling in the counts.
A paragraph to audit
The paragraph below is a composite written for this lesson — not a quotation from any paper. It is deliberately the kind that passes review.
EEG was recorded from 64 scalp electrodes at 500 Hz using an active-electrode cap and a DC amplifier. Data were re-referenced to the average of all electrodes and band-pass filtered between 0.1 and 30 Hz. Bad channels were interpolated. Independent component analysis was used to remove ocular artifacts. Data were epoched from −200 to 800 ms relative to stimulus onset and baseline-corrected. Epochs containing artifacts were rejected. The P3 was quantified as the mean amplitude at Pz. Amplitudes were compared between conditions with a paired t-test, and effects were considered significant at p = .05 or below.
Read it once as a reviewer and once as someone trying to re-run it. The exercises below ask for both readings.
Explore
This lesson has no widget; the object of study is your own next methods section. Before the exercises, take a paper you have read recently — ideally one you found convincing — and try to rebuild its pipeline from the text alone, writing down each parameter you would have to guess. The count is usually between ten and twenty, and the exercise is more persuasive done on a paper you admire than on one you do not.
Practice
This lesson has no notebook. Its practice is the interactive checklist in the exercises below (spec §6 specifies a checklist component here in place of a notebook): work it against a real manuscript — your own, a preprint you are reviewing, or the paper you audit in C6 — and record the result. It is self-graded, because only you can see the manuscript; what it is for is to make the omissions visible while they can still be fixed.
Exercises
Exercise ex-6-6-methods-omissions
Multiple selectRead the composite methods paragraph above. Select every item below that the paragraph does NOT state.
Reporting checklist ex-6-6-cobidas-checklist
Checklist (self-graded)Work this against a manuscript — yours, one you are reviewing, or the paper you audit in C6. Check an item only if the manuscript states it in a form a stranger could act on. Self-graded; nothing leaves your browser.
0 / 23 checked. Self-graded.
Exercise ex-6-6-methods-audit
Free responseAudit the composite methods paragraph above in full. List what is missing, and for each omission say what a reader cannot check because of it — then write the two or three sentences you would add to fix the most serious ones.
Terms used here
Every term this lesson leans on, with the one-line meaning. Follow a term for its full entry.
- BIDS
- A standard folder, file-naming and sidecar layout for neuroimaging datasets, with an EEG extension.
Pitfalls
The archive runs for you and nobody else draft
- Symptom
- The deposited code runs on the author's machine and fails, or silently differs, anywhere else.
- Cause
A working analysis is the code plus the environment it ran in, and only the code gets deposited. Four things go missing, in increasing order of how often:
- Detect
- Re-run it from the archive alone, in a clean environment, on a machine that has nothing installed for unrelated reasons. A fresh virtual environment or container is the whole test. Passing on the development machine proves nothing, because that machine is the one place the gaps are filled in. - Grep your own methods section for the four items above. If any is absent from the archive, it is abse…
- Fix
- An environment file with pinned versions, deposited beside the code. A lock file is better than a requirements list, and either is far better than prose. - A commit hash or tag in the paper, pointing at the state that produced the numbers. - Seeds recorded with the results, not only set in the code. - Data in BIDS where it can be shared, with a data-availability statement naming the access rout…
In other tools
In other toolsEEGLAB · FieldTrip — names only
The equivalents of what this lesson does, for a reader who works in another toolbox. Function names only: their own documentation is the place to learn how to call them.
EEGLAB
pop_exportbidsEEG-BIDS plugin
FieldTrip
data2bidsFieldTripft_reproducescriptFieldTripft_analysispipelineFieldTrip
Names checked 2026-09-18 against EEGLAB 2026.0.0 (plugins at the versions in EEGLAB’s own plugin list) and FieldTrip 20251218.
Reading
- Pernet et al. (2020). COBIDAS MEEG. unverified
- Keil et al. (2014). Publication guidelines for EEG/MEG. unverified