An adaptive threshold is a moving target
Symptom. Reward rate is the same in every session, whatever the signal did, and 'improvement' has no definition.
Symptom
Ten sessions of neurofeedback, and the participant was rewarded about 60 % of the time in every one of them. The training log says the feedback was “responsive” throughout. The report compares the signal in session one with session ten and calls the change learning; a reviewer asks what the threshold was in each session and the answer is that it was re-set each time so that the participant received reward at a comfortable rate. Nothing in the log can say whether the signal moved, because the criterion moved with it.
Cause
An adaptive threshold — re-fixed from the last thirty seconds, the last block of trials, or the previous session — keeps the participant near threshold by construction. That is its purpose: it makes the display feel responsive and keeps the reward rate in a band the protocol designer chose. But the reward rate is then a property of the rule, not of the signal. A participant whose alpha doubled and one whose alpha never changed receive the same fraction of rewards, and a comparison of “reward” or “time above threshold” across sessions compares the rule with itself. Lesson L7.3 lists the baseline as the third of the five loop decisions for exactly this reason: whether it moves decides whether “improvement” can be defined at all.
The confusion is easy to fall into because the adaptive rule is often the better training choice — a fixed threshold set on a bad day may be unreachable — and the two uses of the threshold (driving the display, and measuring the outcome) are rarely separated in the log.
Detect
- Read the protocol for the threshold rule. “Adjusted to maintain a reward rate of about x %” is the sentence.
- Plot the reward rate across sessions. Flat at the target rate, whatever the signal did, is the signature.
- Check whether the raw feature and the threshold were logged separately. If only the reward events survive, the outcome cannot be recovered.
- Ask whether the pre-registered outcome is defined on the raw signal (µV, a z-score against a fixed calibration) or on the display (reward count, time above threshold).
Fix
- Separate the two jobs. Let the display use whatever rule trains best, and define the outcome on the raw feature against a baseline fixed once, before training, and never re-fixed.
- Log the raw per-block value and the threshold as two columns (the library’s session log does), so the outcome can be computed after the fact even if the display rule changes.
- Pre-specify the learning outcome on the raw signal, with the calibration rule stated, before the first session (L7.14).
- Report the reward rate as a description of the display, never as evidence of learning.
Example

data/scripts/make_figures_p5.py.Top and middle, synthetic: the library’s alpha-schedule stream (frontotemporal-4 channel names at 256 Hz, seed 20260920) with a planted linear +60 % rise in the 8–12 Hz RMS from 30 s to 300 s, through the same chain the L7.13 widget runs (129-tap FIR, 32-sample blocks, mean of the TP9/TP10 block RMS). Fixed threshold: z = +1 of the first 20 s (median, MAD). Adaptive threshold: z = +1 of the last 30 s, re-fixed every block. A real learning effect exists here by construction, and the adaptive rule’s reward rate does not show it. Bottom, real: the twelve ds-eegbci eyes-closed minutes nb-7-3 and L7.14 replay (S001–S012, O1, 160 Hz, ODC-By 1.0) through the same chain and the same two rules — nobody can learn in a replayed minute, the fixed rule’s reward rate follows each minute’s alpha, and the adaptive rule pays every minute at about the same rate. Generated by data/scripts/make_figures_p5.py; the exact rates are in the figure’s sidecar.