Engineering Trustworthy Industrial Decisions in the Age of AI
Overconfidence named a failure: a confident wrong answer costs more than a refusal, and almost no AI system built today knows the difference. The Ungoverned Middle took that apart along one seam — decisions, and who answers for them. Causation took it apart along a different seam — reasoning, and whether an inference is grounded in mechanism or merely in pattern.
Three essays so far. So what connects them?
In thirty years around mines, plants, and refineries, I have never heard an operations manager ask what architecture a model uses, or get excited about self-attention. The questions are always the same:
Can I act on this?
Will it help?
Will it hold up in the audit?
Who is accountable if it’s wrong?
Those are the questions the last three pieces were each answering, from different angles.
Industrial AI has never been more capable, and never more in focus. Foundation models, industrial agents, digital twins, optimization: all advancing fast, all being pushed to the plant, as firms move from lighthouse proofs-of-concept toward platform thinking and OT vendors acquire industrial AI firms to keep up.
But will it last, or is this another hype cycle?
Every plant is trying to do the same things it always has: produce more, operate more safely, and use less doing it — higher uptime and utilization, a stronger health, safety, and environment posture, less energy and resources spent per unit of output.
AI will matter only if it helps make those trade-offs better, through decisions grounded in data and domain knowledge, not just fluent ones. The answer will not come just from how capable AI becomes. We already have enough capability to change industrial operations, and it’s becoming more efficient every day.
Capability is not the bottleneck. Trust is.
Trust is the willingness to commit resources, responsibility, or action on the basis of a decision whose uncertainty cannot be eliminated.
Trust cannot be mandated. It cannot be assumed. It has to be engineered. Whether asked of a human or a machine, Can I trust you? has only one acceptable answer in operations: Yes — and here is the evidence, or a track record you can inspect. That answer comes out of engineering, not wishing. That is the premise of this series: Engineering Trustworthy Industrial Decisions in the Age of AI. Two of those words carry all the weight.
The first word: Engineering
Trust is a discipline, not a hope. Engineering exists to turn desirable properties into demonstrable ones — reliability, safety, availability, maintainability, cybersecurity, and now trust. These are not aspirations, but properties that can be specified, designed, measured, tested, and continuously improved.
Every mature engineering field treats its hardest property as something you design in, verify, and evidence — not something that emerges if the team is careful. Process safety engineers do not hope a plant is safe; they run the HAZOP, layer the protections, and audit them.
Industrial AI, today, mostly hopes. Trust is handled with a disclaimer in the footer, a dashboard nobody audits, a pilot that never has to survive contact with operations, or just mandating a human in the loop — which either slows things down or exists more in letter than in spirit. The anti-pattern is always the same: trust treated as something that somehow gets done, downstream of the “real” work of building the model.
Calling this engineering is a commitment: trust gets methods, systems, evidence, and tests — the same machinery every other engineering discipline uses for the properties that matter.
The second word: Trustworthy
“Trustworthy” means nothing until it is operationally defined. Here is the bar this program works to:
An industrial AI system is trustworthy when the engineers and operators who depend on it can rely on it the way they rely on any other piece of engineered equipment. Concretely, that means it is:
- Auditable — you can reconstruct what it did and why, after the fact, to a standard an incident review would accept.
- Explainable — its reasoning can be examined by the people accountable for the outcome, in their terms, not the model’s.
- Repeatable — the same situation produces the same behaviour, or the variation is bounded and understood.
- Predictable — its operating envelope is known, including where it must not be used.
- Testable — every one of the above can be demonstrated with evidence, not asserted.
Notice what appears on neither list: model accuracy or token efficiency. They matter. But they are input to trust, not trust itself.
You cannot buy a trustworthy model. You can only engineer a trustworthy system around whatever models you use.
One clarification before going further. This is not “responsible AI,” and it is not “AI safety.” Those are real and valuable lenses — one rooted in policy and ethics, the other in the behaviour of frontier models. This is an engineering lens, for operational settings where the consequences are physical, the standards are binding, and the accountability is legal.
Why the industrial bar is different
Industrial decisions cross the digital-physical boundary. A recommendation here doesn’t stay a suggestion on a screen: it changes a pressure, a temperature, a maintenance schedule, a safety barrier. A chatbot that answers wrongly wastes a minute; an AI that confidently recommends deferring the wrong inspection can cost lives, licenses, and livelihoods.
Once AI crosses into the physical world, engineering — not prompt engineering — becomes the governing discipline. In industrial operations, a confident wrong answer is categorically worse than a refusal — which inverts the optimization target most AI systems are built around.
The sharpest one-line test I know for whether an AI-assisted decision is operationally ready is this: “Who signs?” If responsibility for the decision cannot be clearly assigned to someone willing to put their name on it, the system is not ready — no matter how good the model is.
The anatomy of a trustworthy industrial decision
You cannot engineer what you have not decomposed. Three dimensions make a decision itself trustworthy: what the AI must know, how it must reason, how it decides.
A. Engineering Knowledge — what the AI knows.
Neglected, you get fluent answers unanchored to the standards, datasheets, and failure history that govern the asset — expert-sounding, knowing nothing that matters. Mining engineering taught the regulations alongside the engineering itself: knowing how to tunnel deep isn’t enough; you must know what’s allowed. Those rules aren’t legal overhead — they encode decades of experience about which risks a mine may take, and which it never may.
B. Grounded Reasoning — how the AI reasons.
Skip this one and correlation gets dressed up as causation — recommending actions that violate physics, standards, or failure mechanisms any SME could recite from memory. We don’t accept that reasoning from humans either: nobody runs a refinery on gut feel, not with lives and billions in equipment on the line. Standards bodies publish procedures for the same reason medical councils prescribe protocols. AI needs that same discipline — grounded data, proven reasoning paths, the right answer repeatably, not just once.
C. Trustworthy Decisions — why engineers should trust the decision.
Neglected, you get recommendations nobody can sign: no auditable path from evidence to action, no answer when the incident review asks why. Most governance stops at canonical data and model lifecycles, never asking whether the recommendation itself is correct, predictable, repeatable. Deterministic expert systems gave that assurance but couldn’t scale; AI scales the recommendation surface enormously, in settings where safety and regulation raise the stakes.
Take a corrosion-under-insulation mechanism near a steam trap: Engineering Knowledge is knowing the mechanism and the standard that governs it. Grounded Reasoning ties the signals to that mechanism, not just the pattern. Trustworthy Decisions is the auditable case for stripping insulation at that exact span.
What happens after a trustworthy decision is made — how to turn it into something operational and scalable — is a separate arc, and a story for a future essay.
One objection is probably already forming. It deserves an answer before moving on.
Where are the models?
A fair challenge: three dimensions, and not one of them says “build better models.” Are we ignoring the actual AI?
No. Models matter enormously — they simply do not occupy the level of abstraction where industrial trust is engineered. Structural engineering is not metallurgy, but no structural engineer is indifferent to steel: the discipline specifies the steel it needs, verifies what it receives, and takes responsibility for everything it builds with it. That is exactly the relationship this framework has with AI models. Frontier model capability is treated as a supplied input, like structural steel. It is a deliberate engineering choice, and it comes with obligations: specify what you need, test what you get, and never build trust on a component you have not verified.
Why this distinction is important: when an AI initiative underperforms, the instinctive response is to swap the model — a new architecture, a new vendor, a different benchmark. Ask instead what is actually constraining the result. It is not always the model. It may be a different constraint somewhere else. An industrial AI program should be evaluated by its impact on that constraint, not by a benchmark score on the model behind it.
Building the Framework
This isn’t a solitary conviction. DNV, the classification society that certifies ships, platforms, and pipelines, is asking the same starting question. So is industry itself: Vimal Kapur, Honeywell’s chairman and CEO, said almost the same thing in a 2026 interview — industrial AI isn’t fundamentally a model problem, it’s a knowledge problem.
Trust is not a feature of the next model. It is an engineering discipline — and it is time we practiced it that way.
If you build, operate, or sign for industrial AI systems, this series is for you — and it will be better with your disagreement than with your applause.
Subscribe via RSS to follow along, or start with the canonical terminology that everything else builds on.