Operating regimes and causal features

The compressor is off, starting, offloaded or loaded, and a deviation means something different in each. Features computed without regard to regime average across four different machines.

Precision–recall, honestly scored

Out-of-sample windows only, walk-forward with a 48-hour embargo. The prevalence floor is 0.0200, which is where a useless model sits.

Runs

Run	Model	Features	AUPRC	ROC-AUC
R7	causal	89	0.0194	0.158
R8	causal + regime	116	0.0183	0.191
R9	logistic regression	89	0.0230	0.476
R10	isolation forest	89	0.0208	0.394

Several of these sit at or below the prevalence floor, and two have ROC-AUC below 0.5 — worse than a coin. That is not a bug in the code; it is the dataset telling the truth. Read the next panel before drawing a conclusion.

Per-event detection — where the value actually is

Window-level average precision says these models are useless. Episode-level detection says something different. Both are true, and the difference is the entire argument about metric selection.

Event	Start	Scoring	Peak score percentile	Lead time of peak (h)
F1	2020-04-18 00:00:00	in the training block	not scored	—
F2	2020-05-29 23:30:00	out of sample	94.4%	43.8
F3	2020-06-05 10:00:00	out of sample	95.7%	46.8
F4	2020-07-15 14:30:00	out of sample	95.5%	35.7

The number that matters The unsupervised isolation forest, trained only on normal operation and never shown a failure, ranks all three out-of-sample events above the 94th percentile of its own score distribution, with hours of lead time. Its AUPRC is 0.0208 — a figure that would get the project cancelled in a review that only read one number.

What this asset can and cannot support

F1 (18 April) falls inside the initial training block and is the single historical event the first model is allowed to learn from. F2, F3 and F4 are scored strictly out of sample. Of those three, only F4 has a genuine precursor: duty roughly doubles over the final 24 hours and oil temperature climbs. F2 and F3 have essentially no run-up in the aggregate, and F1 runs the other way because the train was parked. Realistic achievable performance on this unit is one to two of four events with actionable lead time.

Stating this out loud is not a weakness in the demonstration. A supplier who shows four-for-four detection on four events is either leaking or overfitting, and the engineers in the room already know it.

Computed by scripts/build_models.py in 86.4s at 2026-08-09T19:08:15+00:00 · scikit-learn 1.6.1 · seed 20260809 · source: MetroPT-3 (Air Production Unit of a Metro do Porto train), CC BY 4.0, DOI 10.24432/C5VW3R. Nothing on this page is hardcoded.