CLAIM SUPPORTED registered EVALUATION axis A9 (`subspace_projection`), with its mandatory A12 controls 2 run(s) on disk
Everything in this block was recorded in version control before any run. It is rendered above the result on purpose: a prediction a reader meets after the number it predicted is not a prediction.
A large fraction of frozen-embedding disease discrimination lives in a low-rank speaker-identity subspace, and removing it collapses AUC further than a variance-matched random projection of the same rank does.
Yeh, Sun, Mahapatra, Chandra, Mower Provost, Sisman, 2026, Who is Speaking or Who is Depressed? A Controlled Study of Speaker Leakage in Speech-Based Depression Detection (arXiv:2604.14354): speaker overlap significantly boosts performance, accuracy drops sharply on unseen speakers, and a DANN fails to close the gap; they conclude identity reliance is "a property of current speech representations rather than a model-specific limitation." That paper establishes the effect by measurement. The mechanistic test — estimate the identity subspace, project it out, re-measure — has not been run, and their DANN result predicts it will be hard to remove.
Two-sided, both pre-registered. (a) If projecting out the rank-k speaker subspace collapses disease AUC toward chance while a variance-matched same-rank random projection does not — i.e. D(k) = AUC_rand(k) − AUC_spk(k) has a Holm-corrected 95% CI excluding 0 and AUC_spk(k)'s CI includes 0.5 — the clinical-validity claim for that model/dataset pair is falsified. (b) Conversely, if D(k) is within the noise band for all k while AUC_spk(k) stays well above chance, the hypothesis is falsified and the audited result is strengthened — that becomes the positive control the field currently lacks.
At k = 16: AUC_spk drops 0.10–0.25 from AUC_full; the variance-matched random control drops < 0.05; so D(16) ∈ [0.06, 0.22], positive. Manipulation check: speaker-verification accuracy on the projected embeddings falls from ~0.90 to < 0.30.
| tier (as registered) | EVALUATION |
| family size m / seeds n | m = 14 (7 ranks k ∈ {1,2,4,8,16,32,64} × 2 corpora) · n = 10 → 0.001953 ≤ 0.05/14 = 0.003571 ✓ |
| datasets | SVD (≥ 2 vocalisations/speaker, 2,043 speakers) · Coswara (9 streams/participant). COUGHVID is excluded — no speaker identifiers, so the subspace cannot be estimated. |
| cost | Pure linear algebra on cached embeddings — but "free" was wrong: the confirmatory run took 4h15m on CPU (21 auc_cv calls per repeat × 10 repeats on a 28,509 × 1536 matrix). Cheap in concept, not in wall-clock. |
| min attainable paired p | 0.001953 |
| Holm-tightest threshold 0.05/m | 0.003571 |
| arithmetically satisfiable under R6 | ✓ |
| registry status | CLAIM SUPPORTED, falsifier did NOT fire — see F4 |
Registry status is read from the ## Summary table of IDEA_TABLE.md, which the author maintains as a set. The per-block status cells are maintained one at a time and disagree with it; one source of truth, and this is it.
EVALUATION n = 10 repeats against the pre-registered family (m = 14 (7 ranks k ∈ {1,2,4,8,16,32,64} × 2 corpora) · n = 10 → 0.001953 ≤ 0.05/14 = 0.003571 ✓): min attainable paired p = 0.001953 ≤ Holm 0.05/m = 0.003571. Feasible. Feasibility is a property of the design, not of the result — it says what this run could have shown.
CONTROL CONTROL PASSES -- effect vanishes under shuffling n = 3 × 5 folds SCOPE-LIMITED
Artifact autoresearch_results/V2_speaker_subspace_SHUFFLE.json · written 2026-07-28 21:11 UTC · full run page → · written up as F4 in FINDINGS.md
EVALUATION CLAIM SUPPORTED -- falsifier did NOT fire n = 10 × 5 folds SCOPE-LIMITED
Artifact autoresearch_results/V2_speaker_subspace.json · written 2026-07-28 10:59 UTC · full run page → · written up as F4 in FINDINGS.md
Rendered from autoresearch_results/V2_speaker_subspace.json. Every other run for this hypothesis, including controls, has its own page linked above.
| rank k | AUC, speaker subspace removed | AUC, top-k PCA removed | AUC, variance-matched removed | D vs top-k [95% CI] | variance removed spk / topk / matched | speaker-ID after |
|---|---|---|---|---|---|---|
| 1 | 0.6755 | 0.7320 | 0.7363 | +0.0565 [0.056, 0.058] | 0.206 / 0.345 / 0.271 | 0.251 |
| 2 | 0.6739 | 0.7302 | 0.7320 | +0.0564 [0.055, 0.058] | 0.261 / 0.616 / 0.345 | 0.249 |
| 4 | 0.6446 | 0.7189 | 0.7199 | +0.0743 [0.072, 0.078] | 0.451 / 0.718 / 0.678 | 0.234 |
| 8 | 0.6104 | 0.7025 | 0.7113 | +0.0921 [0.090, 0.094] | 0.700 / 0.812 / 0.798 | 0.207 |
| 16 | 0.5959 | 0.6692 | 0.6747 | +0.0734 [0.071, 0.076] | 0.819 / 0.879 / 0.874 | 0.193 |
| 32 | 0.5661 | 0.6529 | 0.6531 | +0.0869 [0.085, 0.088] | 0.893 / 0.926 / 0.924 | 0.189 |
| 64 | 0.5398 | 0.6287 | 0.6295 | +0.0888 [0.086, 0.091] | 0.940 / 0.959 / 0.958 | 0.187 |
Both controls are built from the data's own principal axes and remove at least as much variance as the speaker subspace, so a positive D cannot be explained by "more signal was deleted".
0.7372 · 0.7391 · 0.7393 · 0.7383 · 0.7388 · 0.7386 · 0.7372 · 0.7361 · 0.7388 · 0.7383
Mean 0.7382 over 10 repeats. Speaker-ID accuracy on unprojected embeddings 0.2780 against a chance rate of 0.0167 — the manipulation check, and it is WEAK: mean-pooled WavLM is not an x-vector, so the identity subspace estimated here is a partial one.