Deploying A Vision Model: From 3,000 Labelled Images To A Line Station

Overview

A vision project is often described as a model training exercise. In production it is a data exercise followed by a deployment exercise, and the model is the small part in the middle. Projects that fail usually fail at one of the two ends rather than in the training.

The dataset problem is that real defects are rare. A line producing good parts all day supplies very few examples of the defects it is meant to catch, and the examples it does supply are unlabelled and inconsistently described. The deployment problem is that a model which performs well on a validation set has to run on hardware in a cell, within a cycle time, on images that look slightly different from the ones it was trained on.

This article describes the sequence from a first dataset to a running station, including the part that most projects underestimate: what happens in the months after go-live. Big Bird Industrial (formerly Shimada Big Bird Industrial) has been building cleaning and deburring systems for automotive powertrain components since 2001, and the figures here come from production inspection programmes rather than from benchmark datasets.

1. The Dataset Is The Project

The practical constraint on a defect detection project is the number of labelled examples of each defect class, and it is a constraint that cannot be removed by choosing a better architecture.

Why Real Defects Are Scarce

A process that is under control produces few defects, which is the point of the process. Collecting a thousand examples of a specific pore on a specific surface can take months, and during that time the tooling, the casting supplier or the part design may change, which makes the older examples less representative.

What Synthesis Can And Cannot Do

Defect generation algorithms place plausible defects onto images of good parts, and they produce exact labels because the generator knows what it added and where. A set of three thousand images that takes weeks to create and label by hand can be produced in minutes this way, which compresses the path to a first working model from months to weeks.

What synthesis cannot do is replace validation on real defects. Real defects carry surface context, lighting variation and material behaviour that a generator approximates, and a model that has only seen synthetic pores will find synthetic pores. The synthetic set gets the project to a first model; the real set proves it.

2. Training Is The Short Part

Once the dataset exists, the training run is a matter of hours on modern hardware. What takes longer is agreeing what the labels mean.

Label Definition Is A Quality Decision

Two experienced inspectors will disagree on whether a mark is a scratch or a tool mark, and on where a defect boundary lies. That disagreement has to be resolved before training, because a model trained on inconsistent labels learns the inconsistency. Writing the definition down, with reference images for borderline cases, is the step that most often gets skipped and most often determines whether the station is accepted.

The Validation Set Has To Be Different

A validation set drawn from the same collection as the training set measures memorisation rather than capability. The useful split is by part, by shift or by production period, so that the validation images come from conditions the model has not seen.

3. Deployment Changes The Problem

Moving from a workstation to a cell introduces three constraints that the training environment did not have.

Compute And Latency

A cell has a cycle time, and the inference has to fit inside it alongside image acquisition, robot motion and the decision logic. A model that takes two seconds per image is fine on a workstation and impossible on a station that must inspect eight views in twelve seconds. Quantising or pruning the model, or moving inference to an edge device, is the usual answer, and each has to be revalidated because it changes the outputs slightly.

The Camera Is Not The Same Camera

Images from the training rig and images from the production camera differ in colour response, noise and focus. A model trained on one and deployed on the other usually underperforms, which is why the production optics should be fixed before the dataset is collected rather than after.

The Reject Has To Go Somewhere

Deployment is not complete when the model runs. It is complete when a rejected part is routed to a defined position, the result is recorded against the part identity, and the line knows what to do next. A station that only displays a red box is a demonstration.

4. What Happens After Go-Live

The months after commissioning decide whether a station keeps its performance, and they are the part most often left out of the project plan.

Drift

Lamps age, lenses collect dust, a fixture wears and the part sits a millimetre differently. Each change is small and each one moves the images away from the training distribution. The visible symptom is a slowly rising false-call rate, which operators learn to work around, and a station that is being worked around is no longer providing protection.

New Defect Classes

A supplier change or a tool change produces a defect the model has never seen. The station cannot find what it was not trained on, and it will not report an error. The only defence is a periodic review of parts that failed downstream, checking whether the station saw them and passed them.

5. The Handover

A station is handed over properly when the plant can retrain it without the supplier. That means the images are retained, the labels are documented, the training pipeline is reproducible and someone on site has run it once.

Without that, every new defect class becomes a service call, and the station's coverage erodes because nobody can extend it. The value of a platform is measured here rather than in the accuracy of the first model.

6. The Failure That Looks Like Success

The most misleading outcome is a station that passes its acceptance test and quietly stops being used. It happens when the false-call rate is tolerable only in the first month, when the review position is understaffed, or when the reject rate is low enough that operators route around the station rather than fix the process.

The defence is to measure the station in operation rather than at handover: how many parts it passed, how many it flagged, how many of those flags were confirmed, and how many failures reached downstream. Those four numbers say whether the station is doing the job.

A fifth number is worth tracking as well: how often the station was bypassed. Every station has a maintenance switch and a manual override, and the frequency with which they are used is the most honest measure of whether the operators trust the result. A station that is bypassed twice a shift is providing paperwork rather than protection.

Those numbers are worth reviewing monthly rather than annually, because a trend that is moving is cheaper to correct early. A false-call rate that has risen from three to eight percent over a quarter is a lamp or a fixture, and a miss that appears in the downstream review is a new defect class. Both are recoverable if they are seen while the station is still in daily use.

Frequently Asked Questions

How Many Images Are Needed To Start?

A first model can be built from a few thousand images of good parts plus a synthetic defect set, which is enough to prove the concept and to size the optics. Production acceptance needs real defects from the process.

Can The Model Run On The Camera?

On some smart cameras, for simple classes. Complex multi-class detection usually runs on an edge device or a small industrial PC, because the model size exceeds what the camera can host.

How Often Should The Model Be Retrained?

Not on a fixed interval. Retrain when a new defect class appears, when the false-call rate has risen beyond what the line tolerates, or when the optics or fixture have changed.

What Happens When The Model Is Uncertain?

It should reject and route to review rather than decide. A confidence threshold that forces a decision on ambiguous images converts a detection problem into a wrong decision.

Conclusion

A vision deployment is a data project at the front, an engineering project in the middle and a maintenance project afterwards. The model is the part that takes least time.

The projects that hold their performance are the ones where the labels were defined before training, the production optics were fixed before the dataset was collected, and the plant can retrain without calling the supplier. Everything else tends to pass acceptance and then quietly stop being used.