Two inspectors can agree that a scratch is awkward and still disagree on reject. The usual fix is to pick a side so the labeling vendor can close the ticket. That closed ticket becomes a confident training example. Multiply by a few hundred borderline frames and the network learns a coin flip dressed as ground truth.
Inter-annotator agreement measures whether the guide is stable. Label-noise briefs own corrupted tags and easy-case bias. SOP-drift notes own definitions that moved while the dataset did not. Few-shot rare-defect pages own scarcity. This one is the workflow gap those pieces assume: an uncertain class with adjudication, so hard cases are not forced into false certainty.
Put uncertain in the schema before volume labeling starts. Tie each defect family to a written acceptance clause—size, location, product class—and show gold examples of accept, reject, and defer. Route uncertain items to a named QA owner; do not let overnight labelers invent policy. Track the uncertain rate by SKU, camera angle, and lighting. A spike is often a lighting or SOP problem, not a model architecture problem. Keep a frozen hard-case panel of adjudicated borders for evaluation; if promotion metrics ignore it, production will rediscover it as escapes and false scrap.
When the taxonomy revises, decide which historical uncertain and flipped labels must be relabeled. Silent reuse of the old hard set under a new clause is how “we measured agreement last quarter” becomes fiction.
A plant that forbids “I don’t know” in the label tool will still hear “I don’t know” from the line—only later, and more expensively.