Experiment lineage

Every configuration shown in this demonstration, with the parameters that produced it. A result that cannot be traced to a configuration is an anecdote.

All runs

Run	Name	Label	Split	Feats	AUPRC	Accuracy	Recall	Fit	Note
R1	majority class	failure starts within 24 h	n/a	0	0.0200	0.9800	0.0000	0.0s	Predicts 'no failure' for every window. This is the number the room judges first.
R2	headline (naive)	failure starts within 24 h	random 70/30 shuffle	51	0.8701	0.9918	0.6434	0.73s	Presented with no caveat. The accuracy is real; the recall is the story.
R3	leaky (in-window)	the window overlaps a reported failure	random 70/30 shuffle	51	0.9951	0.9987	0.9470	0.42s	Detects a failure already in progress. Near-perfect and worth nothing.
R4	leaky (time-ordered)	failure starts within 24 h	walk-forward, refit every 14 d, 48 h embargo	51	0.0217	0.9158	0.0000	3.44s	Same leaky features, honest split. The gap between R2 and R4 is the split; R4 to R7 is the features.
R5	leak recipe L1 only	failure starts within 24 h	walk-forward, refit every 14 d, 48 h embargo	40	0.0212	0.8623	0.0526	3.13s	Contemporaneous machine state, nothing else.
R6	leak recipe L3 only	failure starts within 24 h	walk-forward, refit every 14 d, 48 h embargo	11	0.0172	0.8448	0.0000	2.06s	Whole-series z-scores and percentile ranks only.
R7	causal	failure starts within 24 h	walk-forward, refit every 14 d, 48 h embargo	89	0.0194	0.9305	0.0000	10.39s	Strictly past-only rolling windows, expanding baselines shifted.
R8	causal + regime	failure starts within 24 h	walk-forward, refit every 14 d, 48 h embargo	116	0.0183	0.9310	0.0000	18.35s	Adds operating-regime segmentation and regime-relative baselines.
R9	logistic regression	failure starts within 24 h	walk-forward, refit every 14 d, 48 h embargo	89	0.0230	0.7732	0.0720	5.66s	A linear model on the same causal features, for the PR chart.
R10	isolation forest	unsupervised; scored against the same label	fitted once on normal windows before 2020-05-01	89	0.0208	0.8789	0.0693	1.28s	Trained on good behaviour only. It does not know what a failure looks like.

Sorting by accuracy would rank the least honest run first. That is why the table shows the label and the split beside every score — those two columns are what make the numbers comparable, or reveal that they are not.

Walk-forward folds

Refit every 14 days with a 48-hour embargo between the end of training and the start of scoring.

Fold	Train n	Train pos	Test n	Test pos	From	To
0	10,132	116	1,582	0	2020-05-01 00:00	2020-05-15 00:00
1	11,663	116	1,588	2	2020-05-15 00:00	2020-05-29 00:00
2	13,214	116	930	260	2020-05-29 00:00	2020-06-12 00:00
3	14,178	378	1,864	0	2020-06-12 00:00	2020-06-26 00:00
4	16,009	378	1,537	0	2020-06-26 00:00	2020-07-10 00:00
5	17,657	378	1,475	99	2020-07-10 00:00	2020-07-24 00:00
6	19,043	477	1,533	0	2020-07-24 00:00	2020-08-07 00:00
7	20,603	477	1,771	0	2020-08-07 00:00	2020-08-21 00:00
8	22,392	477	1,267	0	2020-08-21 00:00	2020-09-01 04:00

Windows excluded, and why

Reason	Windows
failure window / repair recovery	869
feature embargo after event	502
guard band	8

Exclusions are part of the experiment and must be reported with it. Rows inside a failure window, or inside the repair-recovery period afterwards, are neither honest positives nor honest negatives; silently keeping them is how a leaderboard score stops meaning anything.

Reproducing this

cd Demo2
python scripts/build_dataset.py     # raw CSV  -> windowed features + labels
python scripts/build_models.py      # features -> 10 runs, sweeps, tests, drift

Deterministic under seed 20260809. The raw CSV is downloaded from the UCI Machine Learning Repository (CC BY 4.0, DOI 10.24432/C5VW3R) and is not redistributed in this repository.

Computed by scripts/build_models.py in 86.4s at 2026-08-09T19:08:15+00:00 · scikit-learn 1.6.1 · seed 20260809 · source: MetroPT-3 (Air Production Unit of a Metro do Porto train), CC BY 4.0, DOI 10.24432/C5VW3R. Nothing on this page is hardcoded.