A quality model that was stable for a year does not always fail because the process drifted. Sometimes the labels drifted under it. An MES upgrade, a plant-standardization project, or a "cleanup" of reason codes merges SCRATCH_EDGE, SCRATCH_FACE, and HANDLING into OTHER. Operators still know the difference. The database does not. Every row after the cut is a different language wearing the same key.
Training across that cut without a versioned taxonomy is how you get a model that looks calibrated on paper and mis-routes lots in the hold queue.
What actually changed
Supervised scrap and defect models assume a stable label space. Precision and recall are meaningless if class C3 in January is not class C3 in June. Merges are the obvious break. Splits are quieter: a new code appears, old codes go unused, and the model never sees enough examples of the new name until after the first customer escape.
Free-text "reason detail" fields do not rescue you if the primary code is what MES reports to analytics and what the model was trained on. Detail text also changes spelling and language by shift.
The gate before any retrain
Freeze a taxonomy version ID on every training row: code list hash, effective date, and plant. Refuse to train across versions unless you have an explicit mapping table reviewed by quality engineering—not by the person who wanted fewer dropdown options.
After a merge, do not backfill historical rows into the new code. Keep history in the old ontology and map only for reporting rollups. Models that need fine classes should train on the fine era or on a carefully remapped subset with known loss of information documented.
What the dashboard should show
Track monthly: fraction of scrap in OTHER / UNKNOWN, count of codes used versus codes defined, and date of last taxonomy change. When OTHER spikes after a release, stop celebrating "simpler MES" and open a data-quality defect.
Confusion-matrix improvements that appear the week after a code merge are often an artifact of coarser labels, not better detection. Ask which classes disappeared before you ship the new weights.
The short rule: the model cannot be more honest than the dictionary. Version the dictionary like firmware.