The edge box hit 92% GPU utilization after the vendor enabled dynamic batching. Scrap rose. Root cause: inference p99 crossed the 200 ms feature window the soft sensor assumed. The PLC kept controlling; the model kept commenting on the past.
Plant-floor edge AI demos often sell “on-prem inference.” This brief is narrower: batching and queueing that violate the control-cycle deadline.
Deadline before utilization
Feature time vs result time. If features are timestamped at t0 and the result arrives after the PLC move at t1, you do not have a closed-loop assist—you have a diary.
Batching as jitter. Larger batches raise throughput and destroy tail latency. Tail latency is what hits the cycle.
Backpressure honesty. Dropping or holding inferences must be explicit. Silent queue growth invents late setpoints through operator trust.
What to put on the traveler
- Hard latency SLO (p50/p95/p99) tied to the loop’s sample time—not “feels real-time.”
- Batch size and max queue depth as MoC-controlled parameters.
- “Stale result” discard rule in the PLC/OPC path, not only in the ML service logs.
- Alert when discard rate rises; do not only alert on GPU temperature.
This is not general edge deployment folklore and not vision lighting drift. It is timing integrity for AI that claims to sit beside the loop.
Do not raise batch size to green a GPU dashboard if p99 already eats the cycle.