Machine learning for warranty claim classification: why your coded claims beat failure code names

A trained model learns from examples, not from a definition. You give it rows of claim text with the code your team chose, and it learns which words, phrases and meanings went with each code, including the habits nobody wrote down: which complaints your engineers file under the instrument cluster and which under the wider electrical system, or when a door complaint belongs to latches rather than body structure.

Keyword rules break on the way people actually write ("stalls," "died at the light," "lost power on the highway"). A general AI model given code names knows what "automatic emergency braking" means in general, but it has never seen how your team draws the line between that and a collision warning. A model trained on your coded history has.

In StayCharted AI Model Trainer (AMT), there are three text model kinds:

• Classifier Model (no AI): learns words and phrases. Every plan.
• AI Classifier Model: a ready-made AI language model reads the meaning, and a private classifier learns your codes. Every plan, and Free includes one.
• Dedicated AI Model: an AI language model fine-tuned on your own examples. Business plan.

The vehicle complaint classification benchmark uses NHTSA's public consumer complaints. The test set is 5,000 complaints received from January 2025 onward. The trained models learned only from older complaints: 19,902 from 2023 to 2024 for the 21 vehicle systems, and 19,281 from 2020 to 2024 for the subsystems (72 labels in training, 70 of them in the test). Splitting by date matters, because a model has to code complaints written after it was trained, just as it would in production.

We also ran the built-in classify functions in Snowflake and Databricks on the same 5,000 complaints, giving them the category names. The full comparison is in text classification in Snowflake and Databricks.

At the system level, prompting with names is a reasonable approach. Snowflake AI_CLASSIFY with a task description and one example per label reached 84.2%, slightly ahead of our AI Classifier Model at 84.1%. The Dedicated AI Model was still highest at 87.7%, but the gap is small enough that 21 broad systems alone would not justify training.

The subsystem level is where the approaches split. Names-only results fell to 52.7% and 56.1%, while the Dedicated AI Model held at 85.9% (4,296 of 5,000 right).

Automotive failure code classification is hard for the same reason it is valuable: the codes sit close together. On the benchmark, the weak spots for every model were neighboring categories. These are subsystem results for our two trained models:

The fine-tuned model gained most where categories overlap and only past decisions show where the line falls, such as the instrument cluster versus the rest of the electrical system. It did not win everywhere: the AI Classifier Model was better on body structure and on automatic emergency braking. Collision warnings stayed hard for both models, which is what you would expect when drivers describe a warning and a braking event in the same sentence. When the Dedicated AI Model got a subsystem wrong, it still had the right vehicle system about a third of the time (248 of 704).

Your own code list probably has more of these close pairs than NHTSA's component list does. Labor operation codes, causal part codes and condition codes are full of rules that live in your engineers' heads. A code name cannot carry those rules, but a few years of coded claims can.

Accuracy across all claims is only half of what a quality team needs. The other half is knowing which answers you can accept without a second look. Every AMT answer comes with a confidence, and the test report shows, for each cutoff, the share of rows above it and how many of those were right.

At 0.90, the Dedicated AI Model let 71.5% of subsystem complaints through at 94.3% right, and sent the other 28.5% to review. At 0.95, it kept 62.6% at 95.7%. Databricks returns a confidence, but in this test it was rarely high and, when it was, often wrong. Snowflake returns no confidence, so you would need a separate rule to decide what a person checks.

Answers below your cutoff go to the Review Queue. Reviewers' answers train the next version, so the share that goes straight through can grow as the model sees more of your edge cases. More on choosing a cutoff is in AI confidence scores and human review.

We want you to read these results with their limits in view:

• It used NHTSA complaints, not warranty claims or repair orders. Complaints are written by drivers, often in a hurry. Your claims and technician lines are written by people who know the vehicle, so your results may differ in either direction, especially where codes overlap.
• Text only. The models saw no make, model or year.
• NHTSA's filing was treated as the only right answer, even where a complaint could fit two components.
• One dataset, default settings, with no tuning for the task. Blank answers counted as wrong.
• Newer complaints were harder in three of the four trained-model comparisons, a reminder to retrain as vehicles and wording change.

The honest test is your own data. AMT holds back rows the model never saw during training and reports accuracy per code on them, so you see the result before you rely on it.

Training is not always worth it. If you sort claims into a handful of broad groups, a general AI model given the names may be close enough: at 21 vehicle systems, names-only functions scored 82.6% and 83.3%. If a wrong code costs little and nobody trends the data, keyword rules or a manual pass may do. Machine learning pays off when you have many close codes, your own rules for applying them, enough volume that review time matters, and a history of coded claims to learn from.

Many quality teams keep claims in a warehouse. On Essentials and Business, you connect Snowflake or Databricks in AMT Settings, choose which tables AMT may read, set a schema for results and run the grant SQL it gives you. Snowflake connects with a key pair AMT generates (no password), and Databricks uses a service principal.

You then train a model from a table of past coded claims and fill new rows from a table. Answers land in one results table in your warehouse with the key, code, confidence, needs-review flag, and model and version, so your SQL can join them back to the claims. AMT only reads the source tables you chose and writes the results table. Rows sent for training or coding are processed by StayCharted, so this is not computation entirely inside your warehouse; check your data-handling rules first. The Snowflake and Databricks guide walks through the setup.

You can also fill CSV, Excel or JSON files, use the Google Sheets add-on, call the API (Essentials and up), or use the Power Automate connector. For OEMs that want every dealership to use one model, see AI warranty claim coding for dealer networks.

1. Export coded history. Claim or complaint text plus the code your team assigned. The last two years is a sensible start.
2. Upload to AMT. It checks the data, flags personal information and can mask it before training, on every plan.
3. Train and read the report. Accuracy overall and per code, the codes it confuses, and the share above each confidence cutoff with how many are right.
4. Pick a cutoff and fill. Confident answers go straight through; the rest go to the Review Queue.

Start with the AI Classifier Model on Free. If the report shows close codes where it struggles, the Dedicated AI Model on Business is the one that held 85.9% on subsystems. Plans are on the pricing page, and the product page covers every way to use a model. For complaint and claim filing as a ready workflow, see vehicle quality complaint classification.

The Review Queue requires Essentials or Business. Retrain and publish a new version to use reviewer corrections in future answers.