Skip to content
Hakan KaracaENGINEERING JOURNAL
AI & Intelligent AutomationEki 8, 20263 min read

Predictive Maintenance: Why Failure Labels Are Harder Than the Algorithm

Predictive maintenance often fails because the organization has data, but not the data needed to answer the maintenance question.

The hardest part of predicting failure is often deciding what “failure” means in the available data.

Predictive maintenance projects frequently begin with a collection of vibration, temperature, current or pressure trends. A model can learn patterns from these signals, but a credible prediction requires a trustworthy target: what is being predicted, over what horizon, and how the event is recorded. Maintenance data are created for work planning and traceability, not always for machine-learning evaluation.

Inspect the label-generating process

A work order may describe a planned inspection, a precautionary replacement, a confirmed defect or a follow-up task. Treating all of them as “failure” creates a noisy target. Conversely, failures that were repaired informally or entered late may be missing from the system. Talk to maintenance staff and map how an observed condition becomes a diagnosis, a notification and a recorded event.

Beware of leakage

Feature leakage occurs when the training data contain information that would not be available at prediction time. A maintenance code entered after a failure, a quality disposition linked to the event, or a sensor channel that changes only after the shutdown can make an offline model look excellent. Split data by time and, where relevant, by machine or site; do not randomly shuffle adjacent rows from the same event across training and test sets and call the result a prospective evaluation.

Represent operating context

High vibration at full load may be normal and low vibration during an idle state may say little about health. Include operating mode, speed, load, product type, ambient conditions and maintenance state where they influence the signal. Compare the model against a baseline that uses the same context and time horizon.

Define the maintenance action

A probability of failure in 30 days is useful only if the team knows what to do with it. Does the result trigger a planned inspection, a spare-parts check, a work-order recommendation or no action until another measurement confirms the condition? Quantify the trade-off between missed failures, unnecessary interventions and the cost of downtime. A low false-negative rate may be essential for one asset and economically wasteful for another.

Test prospective usefulness

After offline evaluation, run the model in shadow mode: predictions are recorded but do not drive maintenance decisions. Compare them with subsequent observations, inspect false positives and negatives with domain experts, and verify that data quality and performance remain stable. Only then consider a controlled operational trial with clear ownership and rollback criteria.

Predictive maintenance is not a contest to guess the failure date to the nearest hour. It is a decision-support problem under uncertainty. The algorithm matters, but the definition of the target, the timing of information and the quality of the maintenance workflow matter at least as much.

References and further reading

  1. NIST AI Risk Management Framework