Scalp to Source
Progress

Pitfalls › pf-filter-across-boundaries

filtering Level 1Level 7 pf-filter-across-boundaries

Filtering across discontinuities

Symptom. Ringing at file joins, breaks, or buffer edges.

Symptom

Large oscillatory artifacts, or a slow swing that takes seconds to settle, at specific places in a continuous recording: where two runs were concatenated, where a segment was cut out and the remainder joined, at the start and end of the file, and — in real-time systems — at every buffer boundary. Epochs that happen to fall near those places carry the artifact; averages that include them show a slow deflection with no relation to the stimulus.

Cause

At a discontinuity the signal jumps: from the last sample of one run to the first sample of the next, from a value to zero padding at the file edge, or from the DC offset of one recording to that of another. To a filter a jump is a step, and a step response with overshoot or a long settling time produces ringing (for sharp FIR and high-order IIR filters) or a slow exponential swing (for a high-pass with a low cutoff). A zero-phase filter spreads the artifact both before and after the join. The length of the affected region is the length of the filter’s impulse response — many seconds for a 0.1 Hz high-pass.

Detect

  • Keep a list of boundary events (MNE annotations of type BAD_boundary or equivalent) and inspect the filtered data around each one.
  • Look at the start and end of every file after filtering; compare with the unfiltered data.
  • Compute the impulse-response length of your filter and mark that much data on each side of every boundary as suspect.
  • In real-time pipelines, filter a test signal with a known step and measure the settling time of your buffer scheme.

Fix

  • Filter each continuous run separately, then concatenate; never filter across a join.
  • Where a join is unavoidable, annotate the boundary and exclude epochs that overlap the boundary plus the filter’s impulse-response length.
  • Pad sensibly at file edges (MNE reflects the data by default; state the padding method) and discard the padded region’s worth of data from analysis.
  • Real-time: carry filter state across buffers (scipy.signal.sosfilt with zi, or a streaming implementation) instead of filtering each buffer independently; account for the resulting group delay in the latency budget (L7.3).

Example

Three stacked traces around the point where two recordings were joined. Top: the raw signal with a visible jump in level at the join. Middle: the high-passed signal computed across the join, showing a slow bump spanning about two seconds either side of it, overlaid on the per-run version, which shows none. Bottom: their difference, a smooth deflection confined to the shaded region of half a filter length on each side of the join.

ds-eegbci S001, Fp1: the last 6 s of R01 and the first 6 s of R02 concatenated raw, with a −131 µV level step at the join. A 1 Hz high-pass (MNE’s default FIR, 529 taps) applied across the join spreads the step into a deflection of ±1.65 s (half the filter length) on both sides, up to 82 µV; filtering each run separately, which is what mne.concatenate_raws plus Raw.filter do by default through the EDGE annotations, leaves the join clean. Generated by data/scripts/make_figures.py (ODC-By 1.0).