A model for early failure detection on an air compressor

Trained on seven months of real telemetry from the air production unit of an in-service metro train — the same class of oil-lubricated reciprocating compressor, air-dryer tower pair and intake valve found in an industrial installation. Four air-leak failures are documented in the source data.

Accuracy

99.18%

on a held-out 30% split

Precision

92.0%

of alerts were real

ROC-AUC

0.988

threshold-independent

Training rows

1,516,948

10-second samples

Before we go on Take the number at face value for a moment. It is a real model, really trained, on real industrial data — nothing here is fabricated. Would you deploy it? Write down your answer.

What the system was asked to learn

Asset: MetroPT-3 (Air Production Unit of a Metro do Porto train)
Period: 2020-02-01 00:00:00 → 2020-09-01 03:59:50
Cadence: 10 s (1,516,948 rows)
Sensors: 7 analogue, 8 digital
Documented failures: 4 air leaks
Licence: CC BY 4.0

Modelling grain: 10-minute windows (23,877 usable)
Positive windows: 477 (2.00%)
Excluded: 1,379 windows
Model: HistGradientBoosting, 51 features
Split: random 70/30 shuffle

The number nobody asked for

A model that predicts "no failure" for every window, always, and has learned nothing:

Majority-class accuracy

98.03%

predicting "never fails", every time, forever

Our model's advantage over it

+1.15 pts

the entire value of seven months of engineering

Failures occupy 1.97% of the record. On a problem with that balance, accuracy is a measure of the class balance, not of the model. It is the wrong number, and it is the number that appears on most predictive-maintenance slides.

Computed by scripts/build_models.py in 86.4s at 2026-08-09T19:08:15+00:00 · scikit-learn 1.6.1 · seed 20260809 · source: MetroPT-3 (Air Production Unit of a Metro do Porto train), CC BY 4.0, DOI 10.24432/C5VW3R. Nothing on this page is hardcoded.