The pilot is the easy part
Most AI demonstrations use a fixed dataset, a known model and a short evaluation period. Production is different. Data changes. PLC programs change. sensors are replaced. Products change. Maintenance events alter equipment behaviour.
An industrial ML system therefore needs the same lifecycle discipline that automation engineers already apply to software and machines.
Version more than the model
Model version alone is not enough to reproduce a prediction. Record the training dataset, feature logic, preprocessing, software environment and model parameters.
When a model is retrained, the organisation should be able to answer what changed and why.
Monitor the inputs
A model can degrade even when its code has not changed. If the distribution of an input signal shifts, the model may operate outside the conditions seen during training.
Monitor missing values, ranges, sampling rates, category frequencies and other relevant data-quality indicators.
Monitor the outcomes
Where labels eventually become available, compare predictions with actual outcomes. For anomaly detection, track alert confirmation and dismissal. For forecasting, track error over time. For quality classification, monitor false rejects and missed defects.
Deployment needs an exit strategy
Every production model should have a known rollback path. If a model behaves unexpectedly after a process change, engineers need to be able to disable or replace it without disrupting the control system.
This is one reason to keep AI services outside the deterministic control core when possible.
Industrial MLOps is mostly disciplined engineering
The tooling can include model registries, pipelines and monitoring platforms, but the principle is simple: know what is running, know what data it expects, know when it has changed and know what to do when it is no longer trustworthy.
Data drift and concept drift are different
Data drift means the inputs reaching the model have changed. Concept drift is deeper: the relationship between the inputs and the outcome has changed. A new product formulation can create both. A sensor replacement may create data drift without changing the underlying process.
The distinction matters because the response is different. A sensor issue may require recalibration; a process change may require retraining.
Monitor model behaviour alongside the plant
Track prediction distributions, confidence, missing inputs and error where labels are available. Compare those signals against production events such as maintenance, recipe changes and software releases.
Set explicit trust boundaries
A model can move from trusted to untrusted gradually. Define thresholds or review rules that trigger investigation. For example, a sudden increase in unknown operating conditions can move the application into advisory-only mode.
Model monitoring needs ownership
Someone must receive the alert and know what to do next. Data scientists, automation engineers and operations teams often share this responsibility, so the handoff should be explicit.
Retraining is not automatically the answer
When a model underperforms, first determine why. More training data will not fix an incorrectly defined target or an unreliable sensor. A disciplined diagnosis prevents a cycle of repeatedly retraining the wrong system.
Link drift to engineering events
A sudden change in model behaviour often has a physical or software cause. Check PLC downloads, recipe updates, sensor replacement, process changes, maintenance interventions and supplier changes before assuming the model itself is defective.
Keep a model change log
For every production model, record deployment date, training window, feature version and reason for retraining. Over time this becomes a useful engineering history and makes it easier to explain why performance changed.
Use a staged response to drift
The response does not have to jump directly from “model online” to “retrain now.” An intermediate state can lower alert authority, increase human review or limit the model to monitoring until the evidence is understood.
Drift is part of normal lifecycle management
Manufacturing changes. A good AI system is designed around that fact rather than treating every performance change as a surprise. Monitoring, investigation, controlled retraining and rollback should be planned before the model is deployed.
Model monitoring is an operational service
Someone should own the model after deployment. That person or team needs dashboards for input quality, prediction behaviour, recent performance and deployment history. Without ownership, drift becomes everyone’s problem and therefore nobody’s problem.
Watch for process changes
PLC software updates, recipe changes, sensor replacement and equipment modifications should be considered potential model events. Integrating those engineering change records with model monitoring makes investigations faster.
Use staged responses
When a model shows unusual behaviour, it may be possible to move it to advisory-only mode, increase human review or temporarily suspend predictions. These intermediate states are often safer than repeatedly retraining without understanding the cause.
Keep training data reproducible
A retraining request should be able to identify the source period, feature version and data-quality rules used to create the dataset. Reproducibility turns model maintenance from trial-and-error into a controlled engineering activity.
Monitoring should connect to change management
Model monitoring becomes much more useful when it is aware of engineering changes. A PLC firmware update, sensor replacement, recipe revision or maintenance intervention can explain a sudden shift in model inputs. Linking these events reduces the temptation to retrain blindly.
Keep a fallback model or rule set
Some applications can retain a simpler statistical baseline or rule-based detector alongside the production model. The baseline provides a reference for detecting unexpected model behaviour and a practical fallback when the primary model is disabled.
Review performance periodically
A model should have a scheduled engineering review, not only an alarm when performance collapses. Periodic review can identify gradual degradation and confirm that the original use case is still relevant to the process.