How accurate is AI HS code classification? We measured it on 6,788 CBP rulings

U.S. Customs and Border Protection publishes its classification rulings: a product, and the 10-digit HTS code CBP decided it belongs under. We trained two models on 5,621 rulings from March 2021 to December 2025. Then we asked them for the code of 1,167 later rulings, from December 2025 to August 2026, that neither had seen. That is the same order a company works in: learn from what was decided before, then classify what comes next.

• An exact 10-digit code from one line of text is hard. The models got it right on 22% and 25% of rulings. Each model read only CBP's one-line subject, such as "deck tiles from Brazil".
• The right neighborhood is much easier. The Dedicated AI Model put 74% of rulings in the right chapter and 39% in the right 6-digit HS subheading.
• The surest answers are the ones to use. The AI Classifier Model was at 0.9 confidence or more on 6.7% of rulings, and 96% of those were right to all 10 digits. Everything below that cutoff goes to a person.
• The models do different jobs. Use the AI Classifier Model to send the sure ones straight through. Use the Dedicated AI Model to give your team or broker a first suggestion on every line.
• No accuracy figure, ours included, makes a code legally right. The importer stays responsible for what is declared. A model prepares the work for a person; it doesn't replace them.

Why by date. A random split would put rulings from the same week, often for the same importer and the same product line, on both sides. The test would then reward remembering. Splitting by date asks what a company actually needs: classify next month's products from last year's decisions.

Why a hard test. Two things make it harder than a company's own history:

• New codes. 27% of test rulings have a code that never appears in training. A model trained on history can't suggest a code it has never seen, so we report the other 73% separately too.
• Short text. One line is less than most product masters hold. In 147 test rulings the subject was word for word one already in training, and in 39 of those (27%) CBP gave a different code. The material, use or construction that decided it was in the ruling, not the subject line.

People ask for rulings when a product is hard to classify. Many everyday products never need one.

• StayCharted AI Classifier Model: a ready-made AI reads the meaning of each subject, and a classifier learns the codes from the training rulings. It trains on a CPU.
• StayCharted Dedicated AI Model: a language model fine-tuned on the same rulings, on a GPU.

Both were trained, published and filled through AMT exactly as a customer would use it: upload a spreadsheet, train, fill a file with the code column blank. Each answer comes with a confidence. Neither model was given anything except the training rulings: no tariff schedule, no chapter notes, no instructions.

Share of the 1,167 test rulings where the model's code matched CBP's to that many digits:

On the 849 rulings whose code appears in training, the 10-digit figures are 30.5% and 34.5%.

• The Dedicated AI Model is ahead at every level. The gap is small at 10 digits and wide at the chapter. On the same rulings, it was right at 10 digits where the AI Classifier Model was wrong 106 times, and the other way round 72 times. At the chapter it was 293 times against 50.
• When it is wrong, it is usually close. Three in four of its answers are in the right chapter. That is what a broker or specialist needs to pick up a product quickly.

For context, not a ranking. A 2025 research paper (ATLAS) tested models on 200 CROSS rulings, each with a fuller product description that an AI wrote from the ruling. A fine-tuned Llama 3.3 70B got 40.0% at 10 digits and 57.5% at 6. GPT-5 Thinking got 25.0% and 55.5%. Different rulings and richer text, so the figures can't be compared directly with ours. They do show that the exact 10-digit code is hard for every kind of model.

This is the number to plan around. Pick a confidence cutoff. Answers at or above it go straight through, and everything below goes to a person.

AI Classifier Model

Dedicated AI Model

Two things to read from these:

• The AI Classifier Model knows when it knows. At 0.9 it answered 78 of 1,167 rulings without a person, and three were wrong. Raise or lower the cutoff and you choose how much reaches your team.
• The Dedicated AI Model's confidence is less sharp. It is right more often overall, but its highest confidences are not as reliable (only 15 rulings were at 0.9). Use it to suggest a code on every line, and let a person confirm.

Combining the two adds little. Taking the AI Classifier Model's code when it was at 0.8 or above, and the Dedicated AI Model's otherwise, gave 25.4% at 10 digits and 39.4% at 6. That is barely more than the Dedicated AI Model alone.

• Your own history may be easier to learn than this test. Your catalog repeats: new colors, sizes and suppliers of products you've already classified. Your descriptions carry material and use, not one line. Most companies use a few hundred codes, not 2,014. Train on your own filed codes and read the report: AMT measures accuracy on rows the model never saw, per code, before you rely on it.
• Plan on a person, and choose how much they see. The cutoff tables above are what AMT's report shows for your own model. Set the cutoff, and everything below it goes to the Review Queue, or to the list you send your broker. Their answers become training rows for the next version.
• Suggest the chapter even when the code is uncertain. A specialist who starts from the right chapter, or the right heading, works faster than one starting from a bare description.
• The model doesn't apply the law. It learns patterns from past decisions. It doesn't read the General Rules of Interpretation, section and chapter notes, or the ruling text. A suggested code is not a ruling, and only CBP issues binding rulings.

• Data. The rulings were pulled from CROSS's public search, newest first, as far back as the search goes (about April 2021). A ruling was kept when it gives exactly one 10-digit HTSUS code, after leaving out Chapter 99 numbers (Section 301 and other additional duties, which sit beside the classification). Rulings that classify several products were left out rather than guessed, and so were revoked and modified rulings. Codes are written with dots (8451.90.3000), as rulings show them.
• Training and filling. Both models were trained and used on StayCharted's production service, in a benchmark workspace, through the same screens a customer uses. The test file had the code column blank; the filled file was compared with CBP's codes.
• Scoring. A code counts as right to N digits when its first N digits match CBP's. The cutoff tables count the rulings at or above each confidence and how many of those were right.
• Margin. These are single-run results on 1,167 test rulings. The confidence-cutoff subsets are smaller: 78 rows for the AI Classifier Model at 0.9, and 15 for the Dedicated AI Model. Their percentages are more sensitive to individual errors.
• Not yet measured. The models with the full ruling text rather than the subject line. We'll publish that as a second round.

• CBP Customs Rulings Online Search System (CROSS)
• ATLAS: Benchmarking and Adapting LLMs for Global Trade via Harmonized Tariff Code Classification (2025)

How accurate is AI HS code classification?

It depends on how many digits you need, and on the text the model reads. From a one-line description of a CBP ruling, our models got the exact 10-digit US code right on 22% to 25% of later rulings, the 6-digit HS subheading on 30% to 39%, and the chapter on 53% to 74%. Their surest answers were right far more often: 96.2% for the 6.7% of test rulings at or above 0.9 confidence with the AI Classifier Model.

Can AI classify HTS codes to 10 digits?

It can suggest them, with a confidence. The 10-digit line depends on details a short description often leaves out, so expect a person to confirm most of them. Send the high-confidence ones through and the rest to your team or broker.

Is there an accuracy guarantee?

No. A suggested code is a starting point, and the importer stays responsible for the code declared to customs. What we give instead is a measurement: accuracy on rows the model never saw, per code, for every version of your model.

Is the benchmark data public?

Yes. CBP rulings are U.S. government works in the public domain, searchable on CROSS (rulings.cbp.gov).

Why are vendor accuracy claims higher than these numbers?

We can't say how others measured. Accuracy depends on how many digits count, how rich the product text is, whether the test products are new or repeats, and whether the test was split by date. We report every digit level, a split by date and a one-line text, so the numbers are hard to inflate.

Does a model trained on our history do better than this?

It may, if your products repeat and your descriptions are fuller. But measure it: train on your filed codes and read the report before relying on it.

Is our product data used for anyone else's model?

No. Each model is trained on your examples for your use and isn't pooled with anyone else's data.