Machine learning for part identification from photos: what 100 look-alike parts showed

The approach is supervised learning. You give the model photos that your team has already labeled with part numbers, about 20 per part in our test. It learns what separates one part from another, then answers on new photos with three things:

• A part number, chosen from the parts it was trained on.
• A confidence from 0 to 1.
• A needs-review flag when the confidence is below the cutoff you set.

The alternatives are weaker for this job. Keyword rules have nothing to work with, since a photo has no text to match. Browsing a catalog by eye works, but it is slow and depends on who is at the counter. A general AI model asked to name a part can describe what it sees ("a 2×3 brick"), but it has never learned your part numbers or the small differences your team uses to tell them apart. Our benchmark did not test a general AI model, so we make no accuracy claim about one. The limit that applies to every trained model also applies here: it can only name parts it was trained on.

A note on the phrase "part number recognition AI": the model in this test recognizes the part from its shape. It does not read a printed or stamped number.

We needed a public, citable stand-in for an industrial parts catalog, so we used the Images of LEGO bricks dataset (v1.1, Gdańsk University of Technology). Many parts differ only by a stud count or a length, which is the same problem a distributor has with near-identical fittings.

• Parts: 100 part numbers in 25 families.
• Training photos: 2,000 (20 per part).
• Test photos: 1,000 (10 per part), none sharing a source photo with a training image.
• Cleanup: 2,725 near-copy photos dropped so the test could not be passed by memory.

Two model kinds in StayCharted AI Model Trainer (AMT) were trained on the same photos:

Why the AI Image Classifier Model won here

We expected the Dedicated AI Image Model, which is fine-tuned end to end on the photos, to do better. It did not. It scored 75.3% against 87.8% and took 74 minutes to train. The AI Image Classifier Model pairs a ready-made vision model with a small classifier that learns your parts. It trained in under a minute with no GPU, and its own validation during training (88.7%) was close to what it then scored on unseen photos (87.8%). That match matters, because it means the report you see after training is a fair guide to what you will get.

With 20 photos per part, the lighter model was the right choice. We say so because you should start there too, and only try the Dedicated model if your own report shows a reason to.

An 87.8% overall score sounds like it needs a person checking every answer. It doesn't, because the model's confidence separates the answers you can trust from the ones you can't. Pick a cutoff, and everything below it goes to a person in the Review Queue.

At the default 0.5 cutoff, 323 photos went straight through and 322 were right. The other two thirds went to a person, but those photos held 99.2% of the model's mistakes. At 0.3, about 65% went straight through (639 of 651 right, 98.2%).

The Dedicated AI Image Model at its default cutoff of 0.9 sent 47% straight through at 97.9% right. The AI Image Classifier Model at 0.4 sent about the same share (48%) at 99.2% right. The lighter model gave cleaner straight-through answers at the same volume.

The "projected after review" column assumes reviewers fix every flagged mistake. We did not measure human review, so treat it as a ceiling. More on choosing a cutoff is in how to use AI confidence scores with human review.

The mistakes look like the ones a new hire makes. The most frequent were a 1×4 plate taken for a 2×8 brick (4 of 10 photos) and a 2×3 brick taken for a 1×3 brick (2 of 10). Of the 122 mistakes, 72 crossed families.

Distinctive families were 90% or better. Plain bricks and panels, where parts differ mostly by size, were the hardest. Across the 100 parts, 48 were named right on all 10 test photos, while five were right on only 4 or 5 of 10. Per-part results like these are what tell you where to add photos, such as close-ups that show the length, or a reference object for scale.

Training a model pays off when:

• You have many look-alike parts. Dozens or hundreds of part numbers that differ by a size or a hole pattern are hard for people and impossible for rules.
• The part number matters. Ordering the wrong replacement, crediting the wrong return or putting stock in the wrong bin costs money.
• You want part of the work to go straight through. A confidence you can trust lets the easy third (or more) skip the counter entirely.
• You need the same answer every time. A trained model gives the same answer for the same photo, whoever takes it.
• Volume is steady. Field service, a returns dock, or one model across your parts stores all repeat the same lookup many times a day.

It is probably not worth it when you have a handful of very different parts, or when a wrong answer costs nothing because someone checks every item anyway. If your parts already carry a readable label or barcode, scanning that is simpler.

If you are classifying part descriptions rather than photos, or want both together, see parts classification with AI, which covers text and combined photo-and-text models.

1. Collect labeled photos. Use photos your team has already matched to part numbers. Our test used 20 per part. See how many examples you need.
2. Upload them to AMT. Photos can come from a file upload or S3-compatible storage. AMT checks the data before training. No code is needed; the steps are in train an image classifier without code.
3. Train an AI Image Classifier Model first. Read the report: accuracy overall and per part, the confusions, and the share of answers above each confidence cutoff with how many of those are right.
4. Pick your cutoff from that report, not from our numbers.
5. Put it to work. Fill new photos from files or through the API, and send low-confidence answers to the Review Queue. Reviewers' answers train the next version.

Image models need the Vision add-on on the Essentials or Business plan; the Dedicated AI Image Model needs Business with Vision. See pricing, the product, and image classification for more, or start free.

This was one run per model on one public dataset. LEGO parts are cleaner than worn industrial parts, and the photos were cut out on white backgrounds, so expect different numbers on shop-floor photos. The model gives one answer per photo; we did not measure whether the right part was among its top few candidates. Your own parts, photos and lighting will set your accuracy, and the report after training tells you what it is.

The Review Queue requires Essentials or Business. Retrain and publish a new version to use reviewer corrections in future answers.