EVALUATION NOT CLEARED SCOPE-LIMITED svd egemaps
Scope caveat. This run is a pilot slice -- 49 of the 13,535 speakers measured elsewhere on svd. It may satisfy the repeat contract and still be a narrower measurement than a later run on the same corpus; both facts are shown because either alone would mislead.
Paired Wilcoxon over 10 repeats gives a minimum attainable p of 0.001953 = 2/210, against a Holm-tightest threshold of 0.010000 = 0.05/5 for the pre-registered family. Feasible. This is arithmetic on the design, computed before the data: it says what the run could have shown, not what it did.
| head / baseline | speaker AUC | 95% CI | recording AUC | UAR | ECE | per-repeat mean ± std | verdict |
|---|---|---|---|---|---|---|---|
logreg | 0.6172 | [0.437, 0.777] n_boot=2000 | 0.5709 | 0.6560 | 0.204 | — ± — | NOT CLEARED |
linsvm | 0.5259 | [0.353, 0.697] n_boot=2000 | 0.5314 | 0.5250 | 0.088 | — ± — | NOT CLEARED |
mlp | 0.6310 | [0.460, 0.791] n_boot=2000 | 0.5958 | 0.6560 | 0.209 | — ± — | NOT CLEARED |
gbt | 0.7983 | [0.663, 0.925] n_boot=2000 | 0.6910 | 0.6655 | 0.152 | — ± — | NOT CLEARED |
ens_rank3 | 0.7190 | [0.566, 0.855] n_boot=2000 | 0.6394 | 0.6448 | 0.186 | — ± — | NOT CLEARED |
confound age_only | 0.7086 | [0.533, 0.874] n_boot=2000 | 0.7027 | 0.6716 | 0.244 | — ± — | THE BAR |
confound sex_only | 0.4569 | [0.286, 0.630] n_boot=2000 | 0.4552 | 0.6336 | 0.282 | — ± — | baseline |
confound age_sex | 0.6776 | [0.495, 0.853] n_boot=2000 | 0.6725 | 0.7138 | 0.188 | — ± — | baseline |
confound duration_only | 0.4414 | [0.283, 0.606] n_boot=2000 | 0.4149 | 0.5000 | 0.012 | — ± — | baseline |
confound intensity_rms_only | 0.5155 | [0.359, 0.689] n_boot=2000 | 0.4655 | 0.5000 | 0.087 | — ± — | baseline |
confound age_sex_duration_rms | 0.6759 | [0.495, 0.852] n_boot=2000 | 0.6723 | 0.7138 | 0.179 | — ± — | baseline |
ECE is expected calibration error: the gap between stated confidence and observed frequency. It is reported here because clinical usability is a probability question, not a ranking question (CLAUDE.md §4.3). A head can rank well and still be badly calibrated — compare the ensemble's ECE against the plain logistic head.
| head | recording-level delta [95% CI] | speaker-level delta [95% CI] | paired Wilcoxon over repeats | cleared? |
|---|---|---|---|---|
logreg | -0.1318 [-0.289, 0.030], p>0 = 0.0485 | -0.0914 [-0.286, 0.096], p>0 = 0.1570 | W = 0.0, p = 0.001953 | NO |
linsvm | -0.1713 [-0.334, -0.011], p>0 = 0.0185 | -0.1828 [-0.372, 0.010], p>0 = 0.0310 | W = 0.0, p = 0.001953 | NO |
mlp | -0.1069 [-0.261, 0.051], p>0 = 0.0835 | -0.0776 [-0.271, 0.097], p>0 = 0.1945 | W = 0.0, p = 0.001953 | NO |
gbt | -0.0117 [-0.159, 0.151], p>0 = 0.4355 | +0.0897 [-0.076, 0.264], p>0 = 0.8540 | W = 0.0, p = 0.001953 | NO |
ens_rank3 | -0.0633 [-0.218, 0.103], p>0 = 0.2115 | +0.0103 [-0.171, 0.199], p>0 = 0.5440 | W = 0.0, p = 0.001953 | NO |
| head / baseline | AUC per repeated partition |
|---|---|
logreg | 0.5390 · 0.5513 · 0.6173 · 0.5851 · 0.5096 · 0.5423 · 0.5863 · 0.5675 · 0.5672 · 0.5693 |
linsvm | 0.5184 · 0.4929 · 0.5335 · 0.5010 · 0.4607 · 0.5067 · 0.5743 · 0.4460 · 0.5423 · 0.5484 |
mlp | 0.5948 · 0.5779 · 0.6311 · 0.5945 · 0.5330 · 0.5920 · 0.5835 · 0.5611 · 0.5718 · 0.6069 |
gbt | 0.6633 · 0.6958 · 0.6822 · 0.6899 · 0.6169 · 0.6558 · 0.6781 · 0.6667 · 0.6811 · 0.6463 |
ens_rank3 | 0.6290 · 0.6344 · 0.6675 · 0.6521 · 0.5533 · 0.5989 · 0.6407 · 0.5686 · 0.6338 · 0.6309 |
confound::age_only | 0.7230 · 0.7110 · 0.7119 · 0.7163 · 0.6990 · 0.6901 · 0.7099 · 0.6913 · 0.7182 · 0.6924 |
confound::sex_only | 0.5311 · 0.5752 · 0.5377 · 0.5835 · 0.5568 · 0.5581 · 0.5602 · 0.5630 · 0.5690 · 0.5615 |
confound::age_sex | 0.6849 · 0.7005 · 0.6835 · 0.6987 · 0.6267 · 0.6859 · 0.6991 · 0.6661 · 0.6516 · 0.6593 |
confound::duration_only | 0.4697 · 0.4916 · 0.4990 · 0.4955 · 0.4939 · 0.4862 · 0.4967 · 0.4820 · 0.4916 · 0.4936 |
confound::intensity_rms_only | 0.3965 · 0.4404 · 0.4701 · 0.4911 · 0.4288 · 0.5025 · 0.4814 · 0.4224 · 0.4843 · 0.4741 |
confound::age_sex_duration_rms | 0.6760 · 0.6958 · 0.6833 · 0.6950 · 0.6254 · 0.6872 · 0.6976 · 0.6574 · 0.6496 · 0.6593 |
The spread across repeated speaker-disjoint partitions is the empirical noise band. A delta smaller than this spread is not a result whatever its point estimate says.
| artifact | autoresearch_results/bench_svd_egemaps.json · view raw · last written 2026-07-25 23:55 UTC |
| command | run_benchmark.py --corpus svd --backbone egemaps --head all --folds 5 --repeats 10 |
| git_sha | a640ceb19f59b474daa3daede4a81904cf9e60d2 |
| config_hash | 3ffad10ed6634618 |
| generated_utc | 2026-07-25T23:55:48Z |
| embedding_content_hash | 2f46ffc67d70edd20afac757be876ae7efa907423744d8b03d0a319eecac2ee7 |
| wall clock | 3,461.2 s (0.96 h) |
No 7-step reasoning entry exists for this run. The sibling convention renders Diagnosis · Citations · Hypothesis · Prediction · Verdict · Learning on every per-experiment page, sourced from a reasoning_annotations.json. This program never wrote one, and neither does it keep an append-only experiment_log.jsonl; the run history is the set of artifact files listed on the run ledger, ordered only by their write time. The pre-registered prediction for this run does exist — it is on the hypothesis registry, but this run predates the registry and has no pre-registered prediction of its own. Everything else on that convention’s checklist is absent here and is shown as absent rather than quietly dropped.