News
The Confirmation Rate Nobody Owned
A detection station went live on a machined casting line and, on its first full day, rejected 46 percent of what passed through it. The station was working. Its false call rate was high because it was new, and the plan for that was the usual one: the operators confirm the rejections, the confirmations become labels, the model improves.
Twelve weeks later the model had not improved, the false call rate was unchanged, and the operators had developed a habit of releasing parts without looking. The detection was fine. What failed was a number nobody had been assigned: the defect confirmation rate.
Week One: The Station Was Busy
The first week felt like a success because the station was doing something visible. Parts were being flagged, images were being stored, and the screen showed a stream of judgements.
The number that mattered was on the same screen and nobody read it. Of the parts the station judged, most were waiting for a person to agree or disagree, and the queue was growing faster than it was being cleared. On a sample day the platform recorded thirteen parts inspected, a rejection share around 46 percent, and a defect confirmation rate of 7.7 percent.
That is what a confirmation backlog looks like when it starts. It does not announce itself. It appears as a percentage in a corner of a dashboard, and it stays small enough for weeks that nobody treats it as a trend.
Week Four: Confirmation Became Optional In Practice
By the fourth week the queue had a shape. Parts were flagged, moved to a holding area, and waited. The person nominally responsible for defect confirmation was also running the line, and the defect confirmation work was the only task on their list with no fixed time and no consequence for being late.
So it was done when there was time, which meant it was not done. Parts were released on a glance, or on the argument that the station was new and probably wrong, which was true often enough to be a self-fulfilling justification.
This is the point where an installation quietly changes category. It stops being an inspection station and becomes a delay in the line that everybody has learned to route around.
Week Eight: The Model Had Nothing To Learn From
The retraining plan assumed a stream of defect confirmation images. What it received was a small, skewed sample: the confirmations that did happen were mostly on the parts that were obviously bad, because those were the ones worth stopping for.
The defect confirmation data therefore described the easy cases and almost none of the false calls. Since false calls were the actual problem, the training set was missing exactly the examples that would have fixed it. The model was not failing to learn; it was being fed a description of the wrong question.
Twelve weeks of operation had produced a large image archive and almost no usable labels, which is the most expensive possible outcome: the cost of collecting was paid and the value of the collection was not.
What Confirmation Actually Costs
The reason defect confirmation gets skipped is not laziness. It is that the work is unbounded, and unbounded work loses to bounded work every shift.
Confirming a rejection means comparing the station's marked region against the criterion: a defect of 0.5 mm on this face, a linear indication of a given length, a mark that is inside the allowed pattern for this casting. That takes between twenty seconds and two minutes depending on whether the person has to fetch a limit sample, and the variance is the problem. A task whose duration is unknown cannot be scheduled, so it fills whatever time is left, which on a running line is none.
Two changes make it bounded. Keeping the limit sample at the station removes the fetch. Showing the station's reason with the image removes the re-examination, because the person is confirming a claim rather than inspecting a part. Together they turn a variable task into a fixed one, and a fixed task can be assigned to a shift with a time budget and a queue limit.
The defect confirmation rate then becomes a number somebody is measured on, which is the only kind of number that survives a busy week.
Week Twelve: What Actually Changed
The fix was not technical. It was four decisions, made in one meeting.
First, defect confirmation was assigned to a named person per shift, with a time budget, instead of being a task that belonged to whoever was free. Second, the queue was given a limit: if more than a set number of parts were waiting, the line stopped. That single rule converted confirmation from optional work into a constraint.
Third, the defect confirmation screen was changed to show the station's reason alongside the image, so a person could accept or reject a specific claim rather than re-examine the whole part. Fourth, and most important, defect confirmations were reviewed weekly against the false call rate, so the connection between the two numbers was visible to the people doing the work.
What The Number Means
A defect confirmation rate is a measure of whether the organisation is actually using its own inspection system. Below a certain level, the station's output is not evidence of anything, because nobody has agreed with it.
It is also the leading indicator for whether the model will improve. A station with a high confirmation rate and a mediocre false call rate is a station that will get better. A station with a low confirmation rate is a station that will stay exactly as it is, whatever the retraining schedule says.
The uncomfortable implication is that a learned inspection is only as good as the review discipline behind it, and that discipline is a management decision rather than a feature. Buying the model does not buy the improvement; buying the model and assigning the confirmation does.
What I Still Do Not Know
I do not know what the right confirmation rate is. High enough to feed the model and low enough not to stall the line is a band, not a number, and the band depends on the false call rate the station starts with.
What I can say is that a rate of seven percent is not a band. It is a queue that has been abandoned, and the abandonment is visible in the data long before anyone complains about the station being unreliable.