CLAIM SUPPORTED registered EVALUATION axis A5 (representation), with A3 pinned at the strict value; the *leaky* comparison at A3 = `random_recording` is the paired reference condition 1 run(s) on disk
Everything in this block was recorded in version control before any run. It is rendered above the result on purpose: a prediction a reader meets after the number it predicted is not a prediction.
Under simultaneously speaker-disjoint and demographically balanced splits, frozen SSL/health embeddings do not reliably beat eGeMAPS.
SpeechDx ranks Whisper-enc (MRR 0.44) > Qwen3-TTS-Tokenizer (0.40) > WavLM (0.38) and states "no current representation generalizes reliably across the clinical speech landscape" — Bhalla, Kieu, Merchant, de Lara, Mariakakis, 2026, SpeechDx: A Multi-Task Benchmark for Clinical Speech AI (arXiv:2606.17339). The audited proposition is the field's derived habit of treating SSL > handcrafted as settled; SpeechDx itself already qualifies it, and the head-to-head against eGeMAPS under both controls simultaneously has not been run with seed statistics.
If eGeMAPS lands within 2σ_seed of the best of {HeAR, WavLM-base+, Whisper-small-enc} on ≥ 2 of 3 corpora under A3 = speaker_disjoint + demographically_matched, A4 = fit_per_fold, the "SSL wins" claim is falsified for this task family.
eGeMAPS trails the best SSL encoder by 0.02–0.06 AUC on SVD (a real but small gap) and by < 0.02 — i.e. inside the noise band — on Coswara and COUGHVID. Direction: the gap shrinks by ≥ 0.03 AUC relative to the same comparison under A3 = random_recording.
| tier (as registered) | EVALUATION |
| family size m / seeds n | m = 9 (3 encoders × 3 corpora, each vs eGeMAPS) · n = 10 → 0.001953 ≤ 0.05/9 = 0.005556 ✓ |
| datasets | SVD · Coswara · COUGHVID |
| cost | openSMILE is CPU-only; embeddings extracted once and cached. No GPU beyond one extraction pass per encoder. |
| min attainable paired p | 0.001953 |
| Holm-tightest threshold 0.05/m | 0.005556 |
| arithmetically satisfiable under R6 | ✓ |
| registry status | CLAIM SUPPORTED on SVD — matching works (age 0.874→0.553); eGeMAPS 0.6496 BEATS WavLM 0.6227, CI [−0.032, −0.022], 10/10 seeds. Not formally closed (falsifier wants ≥2 corpora) — see F7 |
Registry status is read from the ## Summary table of IDEA_TABLE.md, which the author maintains as a set. The per-block status cells are maintained one at a time and disagree with it; one source of truth, and this is it.
EVALUATION n = 10 repeats against the pre-registered family (m = 9 (3 encoders × 3 corpora, each vs eGeMAPS) · n = 10 → 0.001953 ≤ 0.05/9 = 0.005556 ✓): min attainable paired p = 0.001953 ≤ Holm 0.05/m = 0.005556. Feasible. Feasibility is a property of the design, not of the result — it says what this run could have shown.
EVALUATION SSL DOES NOT BEAT HANDCRAFTED n = 10 × 5 folds SCOPE-LIMITED
Artifact autoresearch_results/V1_ssl_vs_handcrafted.json · written 2026-07-28 16:05 UTC · full run page → · written up as F7 in FINDINGS.md
Rendered from autoresearch_results/V1_ssl_vs_handcrafted.json. Every other run for this hypothesis, including controls, has its own page linked above.
| seed | speakers | recordings | age healthy | age patho | age gap | AUC age only | AUC WavLM | AUC eGeMAPS |
|---|---|---|---|---|---|---|---|---|
| 0 | 616 | 10,243 | 34.32 | 35.21 | 0.893 | 0.5655 | 0.6151 | 0.6392 |
| 1 | 610 | 10,402 | 34.43 | 34.96 | 0.531 | 0.5585 | 0.6253 | 0.6335 |
| 2 | 616 | 10,200 | 34.20 | 35.04 | 0.839 | 0.5474 | 0.6240 | 0.6539 |
| 3 | 608 | 10,682 | 34.51 | 35.14 | 0.632 | 0.5787 | 0.6307 | 0.6718 |
| 4 | 608 | 10,160 | 34.25 | 35.13 | 0.882 | 0.5684 | 0.6217 | 0.6519 |
| 5 | 610 | 9,579 | 34.08 | 34.84 | 0.760 | 0.5292 | 0.6139 | 0.6444 |
| 6 | 610 | 10,427 | 34.41 | 35.08 | 0.670 | 0.5490 | 0.6183 | 0.6466 |
| 7 | 608 | 9,968 | 34.41 | 34.83 | 0.420 | 0.5391 | 0.6307 | 0.6602 |
| 8 | 620 | 10,117 | 34.33 | 35.51 | 1.181 | 0.5359 | 0.6277 | 0.6562 |
| 9 | 626 | 10,652 | 34.16 | 35.08 | 0.924 | 0.5623 | 0.6196 | 0.6386 |
eGeMAPS beats WavLM on 10 of 10 seeds. Paired difference -0.02693 (95% CI [-0.03172, -0.02151]) — the interval excludes zero, and it excludes it on the losing side for the SSL model.
Encoders registered: HeAR, WavLM-base+, Whisper-small-enc; encoders actually run: WavLM-base+, eGeMAPS(baseline). Scope stated in the artifact: HeAR and Whisper not extracted on this host; COUGHVID excluded per F2. A registered-but-unrun encoder is a hole in the family, not a free pass — it is why the falsifier is not formally closed.
Age matching at a ±3.0-year tolerance collapses the age-only AUC to 0.5534 and leaves a mean age gap of 0.7732 years. The artifact records matching_worked = True. Without that collapse the comparison beneath it would be meaningless, so it is measured every run rather than assumed once.