Dropped samples and clock drift

Symptom

Events line up with the data at the start of a recording and drift away from it toward the end: an ERP that is sharp in the first block is smeared or delayed in the last; a rhythm that should be time-locked to a stimulus is locked only early on; the number of samples in the file does not equal the sampling rate times the duration the protocol says was recorded; the interval between two markers that should be exactly one second grows or shrinks across the file. In multi-stream recordings (EEG plus eye tracker plus stimulus log) the streams agree at one end and disagree by seconds at the other.

Cause

Two mechanisms, often together. Dropped samples: the acquisition buffer overflows (a busy computer, a slow disk, a wireless link) and a block of samples is lost without a gap being written; every later sample is now earlier in the file than it was in time, by the length of the dropped block, and events timestamped by a separate clock no longer match. Clock drift: two devices with their own oscillators (the amplifier and the stimulus computer) run at slightly different rates; timestamps from one drift relative to samples from the other, linearly with time, by parts per million that add up to tens of milliseconds over an hour (TODO(confirm) the drift magnitude the author wants quoted). A nominal sampling rate that is not the true one produces the same drift.

Detect

• Compare the sample count with the sampling rate times the recorded duration (from an independent clock: the file’s start and end timestamps, or the stimulus log).
• Plot the interval between successive periodic events (a fixed-rate stimulus, a clock marker) across the recording: dropped samples show as a jump, drift as a slope.
• Look for discontinuities in the data at the moment of a drop: an abrupt step in slow channels, a phase jump in a rhythm.
• In multi-stream files (XDF), inspect the recorded clock offsets between streams; a linear trend is drift, a step is a drop.
• Run the L0.6 first-look checklist: the sampling-rate and event-count items catch both mechanisms.

Fix

• Prevent it at acquisition: hardware triggers on the amplifier’s own clock, adequate buffers, a wired link, and a periodic hardware sync pulse recorded in both streams.
• For drift with a known reference (a sync pulse or periodic marker), resample or linearly rescale the event times to the EEG clock; LSL-style recordings store the offsets needed to do it.
• For dropped samples that can be located, insert a marked gap (an annotation) rather than silently shifting events, and exclude epochs that overlap the gap; a filter across the gap will ring (see pf-filter-across-boundaries).
• Record what you found and did in the QC log, and state the correction in the methods; if the drop cannot be located, treat everything after it as unaligned.

Example

Real data, no signal processing: the EDF+ headers and annotations of ds-eegbci S088 R03 against S001 R03, as read by MNE. S088’s R03 header reports 128 Hz where S001’s reports 160 Hz (S088’s R01 header says 160 Hz), and its annotation intervals (median rest 1.38 s, task 5.12 s, cycle 6.50 s) differ from S001’s cycle of about 8.3 s; 6.50 s × 160 / 128 = 8.12 s, so the annotations and the header disagree about the clock. This is the catalog’s documented caveat that S088, S089, S092 and S100 carry inconsistent/overlapping event timestamps (community-documented, not stated on the PhysioNet page), and the case the L0.6 exercise asks you to catch. Generated by data/scripts/make_figures.py (ODC-By 1.0).