Rethinking MES: A Shared Architecture for Physical AI

More than a decade ago, I worked on a project where the manufacturing execution system (MES) focused on production. We planned to integrate it with an asset performance management (APM) system. APM had a useful asset definition for maintenance, while the quality management system (QMS) mapped the same assets separately around the attributes it needed for quality and process optimization.

Each system held enough master data for its own function. There was no common representation connecting the physical dimensions needed for maintenance with process attributes such as volume and pressure information. We intended to make MES the common ground truth. That effort created another master-data model to build and maintain manually, without grounding it in the equipment’s computer-aided design (CAD) or engineering definition.

I don’t want Physical AI to repeat that pattern. If each new capability arrives with its own bespoke handling of context, evidence and decision logic, the implementations stay narrow and become inconsistent across functions in the same deployment. I’d build MES with Physical AI as a core architectural pillar instead, and make MES own the rules connecting an assessment to execution: which production state it applies to, who can act on it and how the action enters the record.

Here, Physical AI brings learned perception and reasoning about the physical operation into that foundation. MES already preserves execution records: a lot consumed, an operation completed, a tool result received. A learned interpretation also needs its source evidence, model version, uncertainty and conditions of use. For a camera observation, that includes a link to the right part and production order. A prediction can go out of date while production continues. I’d make those properties part of the shared foundation alongside the established transaction rules, with spatial representations, simulation and optimization available wherever a manufacturing function needs physical context or a way to evaluate alternatives.

I began this argument in my earlier article on MES and Physical AI. I meant to return sooner, but got sidetracked into a series on delegating 3D work to agents. That investigation into spatial intelligence brought me back to MES, this time to the foundation its functions share.

Start with the MES foundation

MES already coordinates work, collects evidence and manages exceptions. Established execution models contain considerable context: the ISA-95 job-order representation includes procedure or workflow references and resource requirements. Existing implementations may also provide sophisticated analytics, simulation and decision support.

For this architecture, I group the manufacturing responsibilities as production coordination, work definitions and guidance, material flow, resource readiness, quality decisions, production traceability and operational improvement.

The shared foundation supplies domain meanings, schemas, workflow execution, data and analysis services. Its contracts define how services exchange and interpret information, including the rules for requesting actions and returning results. Enterprise and plant integrations use those meanings and rules to update execution state and initiate workflows.

That foundation must preserve differences across manufacturing modes. One platform may support several modes through configuration, while another serves a particular setting.

These are differences of emphasis. Assembly has procedures and parameters; a process has material requirements. Shared services must preserve those meanings without forcing every operation into the same model.

What I would make native

I would give observations, scenarios and decisions explicit places in the MES domain model. An observation carries its source, time and interpretation; a scenario retains the production state and configuration it evaluated. A decision links the alternatives considered to the evidence, approval and eventual outcome. Manufacturing functions use these records through normal platform interfaces.

By native, I mean the MES product maintains those meanings, interfaces, permissions and lifecycle rules. It also maintains the bindings, the links connecting a production record to its physical representation and observations. The model itself can run in a specialist service or shared plant platform. MES owns how its functions request an assessment, interpret the response and act on it.

For assets, I would make MES the shared operational reference that the earlier project was trying to create. Maintenance could retrieve an asset’s dimensions, while quality could retrieve its process properties. Both would use the same asset identity and recorded installed configuration, reusing maintained links to each property’s source and revision instead of rebuilding those associations for each function. That context would also include the physical representation and the operation using the asset. APM, QMS and other systems could contribute and consume their parts through supported interfaces.

The designed, installed and currently observed asset may differ. CAD supplies design geometry; approved process limits and equipment condition come from their designated engineering and operational sources. The shared reference needs to retain those distinctions, including each attribute’s source, revision and period of use. Otherwise, we’re back to maintaining another copy.

A recurring maintenance task could then resolve the equipment’s recorded installed configuration, including relevant retrofits, through the same reference used by quality and production. A computerized maintenance management system (CMMS) or enterprise asset management (EAM) system could own the maintenance work order while MES coordinates production availability.

A well-integrated MES could already satisfy this definition. Both approaches might show the same forecast and offer the same actions, so I’d compare what happens when a second function needs the capability or a recipe, product configuration or model changes. Does it inherit a maintained contract, or does the team rebuild context, evidence handling and decision logic around another application?

The technical responsibilities below supply that shared behavior. An occurrence is one placed instance of a part, such as bolt position 7 rather than the bolt’s part number. Model applicability means the configurations and operating conditions for which a model is qualified. Both need to be represented explicitly if guidance, quality and planning are to interpret the same result.

Specialist teams could own the process model, solver, scene or inference runtime, with source systems retaining their designated records. Programmable logic controllers (PLCs), distributed control systems (DCSs) and tool controllers would retain their control and interlock responsibilities.

There is a cost to making capabilities shared: version coordination, service availability and model support become platform obligations. An occasional analysis may be better served by an existing integration. My case for a native foundation is strongest when several functions repeatedly need the same context and decision services. Even then, an unavailable scenario service should not block an unrelated material-consumption transaction, and each installation should enable only the capabilities it needs.

One batch decision across several functions

Take an anaerobic batch of sugar-to-ethanol fermentation with yeast. MES coordinates the recipe, ingredient lots, sampling, exceptions and production record. The control system maintains its assigned operating conditions.

Suppose residual sugar is falling more slowly and ethanol is accumulating more slowly than expected, while the temperature loop still meets its setpoint. That loop is doing its assigned job, but the batch may still miss the expected transfer window. Before committing the downstream recovery vessel, the process engineer needs to decide whether to obtain another sample and reassess the timing.

A golden batch, or an envelope of successful batches aligned to process phase and starting conditions, could help identify the departure that warrants investigation. Slower progress alone doesn’t establish yield loss: conversion yield is ethanol produced per unit of fermentable sugar consumed. The immediate decision here is whether the transfer timing remains credible.

The workflow could request eight- and sixteen-hour outlooks using the recipe, initial charge, process history and available laboratory results. Those horizons need validation for the actual process.

Batch B-026 · F-201 → V-202 · recipe R3 / C

Consider a hypothetical assessment for batch B-026 in fermenter F-201. Snapshot S18 identifies recipe R3 revision C, the observations available at that time and their timestamps. A released process model evaluates the likely completion window and its uncertainty, and the assessment preserves those inputs even as production continues.

Quality uses that outlook to assess another sample. Production coordination examines the reservation for downstream vessel V-202. Material flow examines whether preparation of the next charge should move. They refer to the same assessment, then apply their own constraints and authority. A process model supplies the completion outlook; a resource model can examine the resulting vessel conflict.

Now a new laboratory result arrives before the proposed response is approved. It creates a newer state, S19. The workflow checks which assumptions the result affects and, if it changes the completion assessment, routes the recommendation for recalculation or review. Once someone with authority for that change approves it, the workflow commits the response against the assessment it relied on and links the eventual outcome for review.

The batch Operator and Engineer concept views show the same batch record from two roles: sampling and execution tasks for the operator, and the outlook, model applicability and response options for the engineer.

I’d want this behavior even with a conventional process model; nothing in the S18/S19 handling depends on the model being learned.

The same foundation at an engine station

For discrete manufacturing, take a mixed-model automotive engine line planned around a roughly 50-second takt. At the cylinder-head tightening station, each engine serial number resolves to its build configuration, mBOM, routing, approved tightening sequence and tool program.

Several bolts can share a part number while occupying different locations. The view for the current order should resolve the relevant occurrence and procedure, then show the result attached to that position. The tool controller performs the tightening; MES coordinates the work and retains its required torque, angle and time evidence. A camera can help locate the bolt, but cannot establish that the joint achieved the required clamp load.

Engine E-004218 · V2 · position 7 · result R-007

In a separate engagement a few years ago, we modeled visual work instructions and built master data around their mappings. Showing where a part belonged meant preparing a view and associating it with the instruction. Top and side views meant more content. Every new part or assembly had to go through that process, and keeping the mappings aligned became difficult.

With maintained links between product geometry, part positions and operations, a revised assembly could supply views for the affected builds. Guidance, inspection context and rework would reuse those links, with view-generation rules and approved procedures maintained alongside them. Every new view would then start from the current configuration.

Now add learned perception to this station. The build record says what should be on the fixture, and the tool record says what the tool did at each position. The camera supplies a separate visual check against that expected configuration. Suppose camera C-02 and released model P4 produce observation O-017, matching the visible bolt pattern on engine E-004218 against the expected head geometry for variant V2 and locating each position, including bolt 7. MES supplies the engine identity from production tracking. The observation records the source frame, capture time, camera calibration, model version and uncertainty. When the match holds, guidance can use it to highlight the position, quality can review the source image beside the tool result, and rework can use the same association to find the joint.

The harder case is when O-017 disagrees with the record. A different head might have been fitted or a bolt could be missing. The recorded configuration could also be wrong, the camera-to-part mapping could be out of date, or the model could have made a poor match. The discrepancy needs investigation before that observation can guide the work. The observation shouldn’t overwrite the build record, and the record shouldn’t quietly discard the observation.

The workflow holds location-dependent guidance and routes the conflict to someone with authority to resolve it. The source image, model version and recorded configuration appear side by side. I’d want the platform to supply that behavior once, so each camera project doesn’t invent its own. A camera or fixture change would also trigger a check of the affected calibration and position links before the model is reused.

Returning to the case where the position match has been verified, suppose bolt 7 has a rejected tool result, R-007. R-007 carries the tightening evidence, and O-017 places it on the physical joint. The station keeps its hold while the operator checks the evidence and follows the authorized disposition.

Explore the engine-station Operator and Engineer concept views. Select a bolt to see its tool result alongside the station hold, then switch roles to inspect the proposed rework options.

An engineer could compare rework at the station with diversion to an approved bay, considering procedure, tool availability, buffer capacity and downstream sequence. The line’s 50-second takt makes clear visual cues useful; the actual station and flow constraints determine how long the engineer has to decide on rework.

A learned robot policy could operate behind an unchanged MES task interface. That replacement alone is a weak reason to redesign MES; my case rests on the shared context and decisions around execution, including work with no robot.

Models and agents inside the decision window

The shared runtime needs different models for different questions. Discrete-event simulation represents arrivals, completions, breakdowns and resource release. It can examine an engine rework buffer or a queue for a vessel. Physical and process models address behavior such as mechanical contact, heat transfer, mixing and conversion kinetics. A combined scenario can pass a predicted batch-completion time into a flow model to evaluate the downstream effect.

Repeated full solves may exceed the decision’s time or compute budget. A surrogate model, an approximation evaluated over a qualified operating range, can make those comparisons more practical. The scenario service should choose by the question, operating range, decision-relevant accuracy and end-to-end response time. A full solver that meets the workload can remain the default. I explored that tradeoff in When Does a Digital Twin Need a Surrogate?.

The result identifies its model, assumptions and uncertainty. Changed geometry, recipes or operating regimes may require fresh evaluation. When the approximation is unsuitable, the workflow routes the question to a fuller calculation, human review or a defined fallback if no reliable answer can arrive in time.

For an unfamiliar batch exception, an agent could assemble relevant observations, identify missing evidence and coordinate services to compare approved alternatives. It would return the state used, evidence gaps and candidate responses, drawing on the qualified process models for the forecast. That is useful when the investigation can’t be fully specified as one fixed sequence. Ordinary workflows can handle known cases, holds and transaction posting; they also apply the approval rules to an agent’s proposed response.

Model development belongs outside the live transaction path. For the engine camera, a spatial scene could support synthetic inspection data with varied lighting, camera poses, configurations and relevant defect representations. The lifecycle links the dataset and released model to the task and evaluates it against representative real observations. Synthetic examples are development data; evidence that a production unit passed inspection must come from that unit’s qualified inspection process.

What would establish the value

I haven’t measured the benefit of this architecture. I’d compare it with a competent MES and its existing integrations, using the same data and models where possible, so a better forecast isn’t mistaken for a better foundation.

First, introduce a scenario capability into quality, reuse it in production coordination, then change a relevant recipe or configuration. Compare the effort to update the shared links and service rules, the duplicated transformations and the handling of an assessment made obsolete by a later observation. Check what continues to operate when a model service is unavailable. If the existing platform handles that reuse and change with less burden, the case for further redesign weakens.

The opening asset problem needs its own test. Take one asset and a few attributes used by maintenance, quality and production, then change its recorded installed configuration. Can those functions obtain the correct revision, with source authority and history intact, without separately rebuilding the definitions? That would test whether the shared physical reference improves on the manually maintained master data we created.

For the camera case, I’d deliberately introduce an observation that conflicts with the recorded configuration. Check that both records survive, the affected guidance is held and the authorized resolution reaches every function using that context.

Model quality needs separate evidence. For fermentation, historical replay should expose only information available at each decision point and compare the eight- and sixteen-hour outlooks with later outcomes, including error, uncertainty, false alerts and response time. Replay can test prediction; it can’t establish the benefit of an intervention that was never taken. Perception needs real-image checks for incorrect position matches, missed observations and false accepts; surrogates need comparison with their reference models and observed outcomes where available, including behavior outside their intended range.

Disclaimer: The views and opinions expressed in this account are those of my own and do not represent those of my employer, NVIDIA.