PARTIAL registered EVALUATION axis A4 (preprocessing-fit scope), A3 pinned at `speaker_disjoint` 1 run(s) on disk
Everything in this block was recorded in version control before any run. It is rendered above the result on purpose: a prediction a reader meets after the number it predicted is not a prediction.
The inflation from fitting the scaler/normaliser before splitting varies by an order of magnitude across corpora, so "we scaled before splitting but it's a small effect" is not defensible on an unmeasured corpus.
Feature scaling induced data leakage quantification in machine learning-based voice pathology detection, Applied Soft Computing (S1568494626007970): 1,000 repetitions per configuration; effect −0.14 to +0.14 pp on SVD but −8.3 to +7.8 pp on VOICED. Leakage can degrade as well as inflate. This is a published measurement on two corpora; the audited proposition is its generalisation to the corpora this program actually uses, and to embedding features rather than handcrafted ones.
If the fit_on_all − fit_per_fold AUC difference has a 95% CI contained within ±0.01 AUC on all of SVD, Coswara, COUGHVID for both eGeMAPS and a frozen SSL representation, the "corpus-specific magnitude" claim is falsified for this pipeline family and preprocessing scope can be de-prioritised as an audit axis.
Effect < 0.01 AUC on SVD (reproducing the published near-null) and > 0.03 AUC on at least one of Coswara/COUGHVID; sign not predicted (the published result shows both directions).
| tier (as registered) | EVALUATION |
| family size m / seeds n | m = 6 (3 corpora × 2 representations) · n = 10 → 0.001953 ≤ 0.008333 ✓ |
| datasets | SVD · Coswara · COUGHVID |
| cost | Free — re-fits a scaler on cached features. |
| min attainable paired p | 0.001953 |
| Holm-tightest threshold 0.05/m | 0.008333 |
| arithmetically satisfiable under R6 | ✓ |
| registry status | PARTIAL — SVD near-null reproduced on embeddings; falsifier NOT evaluable — see F5 |
Registry status is read from the ## Summary table of IDEA_TABLE.md, which the author maintains as a set. The per-block status cells are maintained one at a time and disagree with it; one source of truth, and this is it.
EVALUATION n = 10 repeats against the pre-registered family (m = 6 (3 corpora × 2 representations) · n = 10 → 0.001953 ≤ 0.008333 ✓): min attainable paired p = 0.001953 ≤ Holm 0.05/m = 0.008333. Feasible. Feasibility is a property of the design, not of the result — it says what this run could have shown.
EVALUATION PARTIAL -- near-null reproduced; falsifier NOT evaluable n = 10 × 5 folds SCOPE-LIMITED
Artifact autoresearch_results/V6_preprocessing_leakage.json · written 2026-07-28 14:03 UTC · full run page → · written up as F5 in FINDINGS.md
Rendered from autoresearch_results/V6_preprocessing_leakage.json. Every other run for this hypothesis, including controls, has its own page linked above.
| corpus | backbone | recordings | speakers | AUC, scaler fit on ALL | AUC, scaler fit per fold | delta (leak - honest) | 95% CI on delta | status |
|---|---|---|---|---|---|---|---|---|
| svd | wavlm | 28,509 | 1,679 | 0.7382 | 0.7382 | 0.00004 | [0.0000, 0.0001] | RUN |
| svd | egemaps | 667 | 49 | 0.5315 | 0.5357 | -0.00418 | [-0.0063, -0.0022] | RUN |
| coswara | wavlm | 610 | 72 | 0.4929 | 0.4906 | 0.00236 | [0.0004, 0.0048] | RUN |
| coswara | egemaps | not yet measured | not yet measured | not yet measured | not yet measured | not yet measured | not yet measured | NOT RUN -- no cached embeddings |
| coughvid | wavlm | not yet measured | not yet measured | not yet measured | not yet measured | not yet measured | not yet measured | NOT RUN -- no cached embeddings |
| coughvid | egemaps | not yet measured | not yet measured | not yet measured | not yet measured | not yet measured | not yet measured | NOT RUN -- no cached embeddings |
3 of 6 pre-registered cells ran. The unrun cells are the corpus-specificity arm — the part of this hypothesis that is actually novel. The near-null reproduced on SVD is a reproduction of the audited paper's point, not the new claim, so the falsifier is not evaluable and the finding is PARTIAL. Artifact self-report: all_within_pm_0.01 = True.