Machine learning for food quality inspection: what a photo model gets right and what it misses

Most food QA teams already have a grading standard. It lives in a spec sheet, in reference photos pinned by the sorting table, and in the heads of experienced graders who know where "slight bruising" ends and "reject" begins. The hard part is applying that standard the same way across shifts, sites and new hires.

A supervised model learns that standard from examples. You give it photos your team has already graded (good or bad, grade A or B, sound or a named defect type), and it learns what separates them. Then it applies that learned grading standard to new photos across shifts.

This is image classification for quality control. Each photo gets one category from your list, a confidence from 0 to 1, and a needs-review flag. The model does not draw boxes around defects or measure them; it labels the whole photo. It works on photos you send it as files, through the API, or from cloud storage, so it fits incoming inspection, lot sampling, supplier claims and audit photos. It is not line-speed machine vision hardware, and the benchmark did not measure speed.

Food quality inspection accuracy depends mostly on how visible the difference between grades is. The benchmark used two public datasets at opposite ends of that range.

• FruitNet: six fruits (apple, banana, guava, lime, orange, pomegranate), each photo labeled good or bad. 1,800 training photos and 600 test photos, half good and half bad.
• VisA cashews: single cashews labeled OK or defective. 360 training photos (60 defective) and 240 test photos (40 defective).

Two models were trained on each. The AI Image Classifier Model trains in about 0.3 minutes with no GPU. The Dedicated AI Image Model fine-tunes an AI model on the training photos on a GPU, which took 41.5 minutes for fruit and 18.3 minutes for cashews.

Fruit quality grading AI: both models scored 98.7% on this dataset

On fruit, both models scored 98.7%, so the faster, cheaper model was as accurate as the fine-tuned one. Per fruit, the AI Image Classifier Model scored 100% on guava, lime and pomegranate, 99% on banana, 98% on apple and 95% on orange. When the difference between grades is clearly visible in the photo, you do not need the larger model.

One caveat matters here. For apples, guavas and pomegranates, the dataset's good and bad photos were taken in different years, so a model could partly learn the photo session instead of the fruit. Treat the fruit results as possibly optimistic, and test on your own photos from separate sessions.

Food defect detection machine learning: judge it by the defects it catches

The cashew result is the more useful lesson. The AI Image Classifier Model was right on 90.0% of photos, which sounds fine until you see that it found only 16 of 40 defective cashews. It called all 200 sound cashews sound and every defect it flagged was a real defect, but most defects went straight through. With 200 sound nuts and 40 defective ones, a model can score high overall while missing most of what QA exists to catch.

The Dedicated AI Image Model found 26 of 40 defects (65%), called 199 of 200 sound cashews sound, and 96% of its "defect" calls were correct. That is a real improvement, and it is still a model that missed 14 defects. It learned from only 60 defective examples, and defects smaller than a few pixels at 512 pixels across may not be visible to either model.

For subtle or rare defects, the practical takeaways are:

• Look at defects found (recall on the defect class) and balanced accuracy, not overall accuracy.
• Use the Dedicated AI Image Model when defects are small or varied.
• Add more examples of each defect type, and photograph at a resolution where the defect is visible.
• Keep a review step, covered next.

Every answer comes with a confidence. Answers below your cutoff are flagged and go to the Review Queue, where a person decides, and those decisions train the next version. The benchmark reports how much checking each cutoff creates, and the projected accuracy if every flagged error is corrected. These are projections, not measured outcomes with human inspectors.

On fruit, checking about 2% of photos is enough to move projected accuracy from 98.7% to 99.5%. On cashews, the default cutoff caught a third of the mistakes, and a 0.99 cutoff caught 80% at the cost of reviewing about a quarter of photos. For the AI Image Classifier Model on cashews, the default cutoff of 0.5 flagged nothing; raising it to 0.9 sent 51.7% of photos to review and caught 96% of mistakes.

The choice is a business one: how many defects can reach a customer, against how many photos a person can look at. The test report in AMT shows this trade-off for your own model before you rely on it. More on setting cutoffs in AI confidence scores and human review.

Rules work when the standard is easy to write down. Food grading rarely is: color thresholds shift with lighting, a bruise on one variety is a russet mark on another, and the line between a cosmetic blemish and a reject is something graders learn by example. A trained model learns from the same examples, so it picks up the unwritten parts of your standard that a rulebook leaves out.

It also gives you something rules and general-purpose AI prompts do not: a test result on your own held-out photos, per category, before you use it, plus a confidence you can set a cutoff on. Our food benchmark did not compare against prompting a general AI model, so we make no accuracy claim there. Other benchmarks on our benchmarks page do make that comparison for text tasks.

A model is not always worth it. If you grade a few dozen photos a week, if your categories are broad and obvious, or if a wrong grade costs little, a person with a reference card is fine. A model earns its place when volume is high, when several people or sites need to apply one standard, when you need a record of how each item was graded, or when you want most items to go straight through and only the uncertain ones to reach a grader.

1. Collect graded photos. Use photos your team has already judged, ideally from several days, lots and lighting setups. Upload them as a ZIP with one folder per grade, or as a spreadsheet of photo links and grades.
2. Train and read the test report. AMT holds back photos the model never sees, then reports accuracy overall and per grade, the confusions between grades, and how many answers clear each confidence cutoff and how many of those are right.
3. Pick the model to match the defect. Start with the AI Image Classifier Model for clear grades. Try the Dedicated AI Image Model when defects are small or rare.
4. Set a cutoff and use the Review Queue. Send uncertain photos to a grader, and let their answers train the next version.
5. Connect it to your process. Fill files of photos, call the API from your inspection app, or classify photos in S3-compatible storage.

Picture models need the Vision add-on on Essentials or Business, and the Dedicated AI Image Model needs Business with Vision. See pricing and the product overview, or read how to train an image classifier without code. For other inspection work, see visual inspection, image classification and condition-based monitoring for equipment photos. You can start free.

The Review Queue requires Essentials or Business. Retrain and publish a new version to use reviewer corrections in future answers.