GET STARTED · CHOOSING YOUR APPROACH
Should you use an AI assistant or train a classifier?
Both can categorize information. The best starting point depends on your examples, your categories, and how often you repeat the work.
Which situation sounds like yours?
| Your situation | A practical starting point | Example |
|---|---|---|
| You’re still figuring out your categories. | Ask Claude or ChatGPT to suggest a structure, then review it. | “Look at these customer comments and suggest useful groups.” |
| You have clear rules but few labeled examples. | Give an assistant the definitions and test its answers. | Separate messages into complaints, questions and compliments. |
| Your categories have boundaries specific to your team. | Train on reviewed examples and compare the results with an assistant. | Your team labels “I can’t log in to cancel my subscription” as Cancellation, even though it mentions account access. |
| You already have a history of approved decisions. | Try a trained classifier that learns from those examples. | Use last year’s reviewed supplier catalog to suggest categories for this month’s products. |
| The same task repeats every week. | Compare accuracy, consistency, review effort and cost per item at your volume on a representative batch. | Categorize 10,000 support tickets each week into the same 40 queues. Both approaches can run through an API. |
| Each item needs investigation or several actions. | Use an assistant or agent to coordinate the work; a classifier can handle the category decision. | Read a complaint, retrieve the order, use your model to suggest a queue, and draft a response. |
Example: 5,000 supplier products to categorize
Your next catalog contains 5,000 products. Each needs one of your approved categories.
- No reviewed catalog yet? An assistant can help draft categories and label an initial sample for your team to check.
- Already have 20,000 correctly categorized products? Those are potential training examples. Keep a representative test set separate, train on the rest, and compare both approaches without supplying the test answers.
- Choose using the results. Which makes fewer mistakes? Which categories cause trouble? How many products need review? What does each workflow cost to run and maintain?
Review matters for both approaches. Neither a confident answer nor a high overall score guarantees that a particular product is correct.
Why not just teach the assistant your categories?
You can give an assistant your definitions and some examples. For a one-off task, that may be all you need. For recurring work, a few practical considerations show up:
- Examples need to be available in context. With 40 categories, five examples each is 200 examples if you include every category. You can reuse instructions, cache prompts or retrieve relevant examples rather than send every example with every item. That context still needs to be maintained.
- Corrections don’t build up by themselves. Your workflow needs to update the instructions or examples when someone fixes a category; this can be automated. In StayCharted, reviewed corrections can become examples for the next training run. They do not change the published model immediately.
- You only know the accuracy if you measure it. Either approach needs a test set and scoring to show how often it is right and which categories it confuses. StayCharted provides evaluation on held-back examples when you train.
- Confidence needs checking against your data. An assistant’s stated confidence is not automatically calibrated to your task. StayCharted’s evaluation helps you choose a review cutoff. Confident mistakes can still pass through, so a cutoff does not identify every error.
- The assistant changes over time. Model versions are updated and retired, so evaluate again when your chosen version changes. StayCharted keeps your published trained version in use until you publish a new one.
- Teams need a shared definition of the task. Different prompts can produce different answers. Standardized instructions help; a shared trained model is another way to apply your team’s category rules.
None of this makes an assistant the wrong tool. These considerations matter most when the work repeats and the categories are your own.
Where does the data go?
Data control can decide which workflow is acceptable before accuracy enters the discussion. Residency concerns where data is stored and processed; sovereignty also concerns jurisdiction and control. Neither is established simply by calling a model “private.”
| Your requirement | What to check | Example |
|---|---|---|
| Keep classification inputs away from an external general-purpose AI provider. | Use AMT directly through file processing or its API, without an assistant connection. StayCharted and its infrastructure providers still process the data. | Classify confidential supplier descriptions without sending each row to Claude or OpenAI. |
| Keep data in an approved country or region. | Check storage, processing, backups and subprocessors. StayCharted currently stores and processes Customer Content in the United States. | An EU-only processing requirement needs an explicitly supported arrangement; US hosting does not meet that requirement. |
| Control who can read data and change models. | Review workspace roles, model access and Activity records against your requirements. | Reviewers correct labels while authorized users manage training and publishing. |
| Meet contractual or regulatory requirements. | Check the DPA, retention and deletion rules, security evidence and any required certifications. A model’s accuracy does not establish compliance. | A procurement team approves the service before uploading confidential contract clauses. |
OpenAI and Anthropic say they do not train on inputs and outputs from their business/API offerings by default. That protection does not mean no processing or retention. Check the particular product, settings and contract. OpenAI business data policies · Anthropic commercial training policy.
With StayCharted: direct file and API classification does not require sending every row to a general-purpose AI service. If you connect Claude or ChatGPT, information the assistant reads is shared with its provider under your agreement with them. Only grant access to the workspaces and models you intend.
Security and your data · Privacy Policy · Data Processing Addendum
What do our benchmarks show?
On banking messages, StayCharted AI Model Trainer’s Dedicated AI Model scored 93.7% against Claude’s 84.7%: fewer than half the mistakes. On contract clauses, 84.9% against 76.9%, about a third fewer mistakes. On medical abstracts, where the categories draw on general medical knowledge, the two were close: Claude 65.7%, the Dedicated model 63.6%.
These runs compared models trained on labeled examples with Claude given category definitions. They did not test an optimized assistant workflow with examples, OpenAI models, or product taxonomies, and did not measure speed or cost. They are a reason to test your own task, not a promise that training always wins.
What StayCharted brings together
You can build evaluation, batch processing and review tools around an assistant API. StayCharted AI Model Trainer brings training examples, accuracy reports, file processing and review into one product, so your team can evaluate a classifier without assembling that workflow.
You can also combine them: let your assistant explain results and coordinate work while it calls the model you trained for classification.
Watch the product tour → · Compare model types → · Connect Claude or ChatGPT →
Common questions
Can Claude or ChatGPT classify business data?
Yes. They can classify using category definitions and examples. Their APIs can support batch workflows and evaluation. Test the actual setup on representative data before choosing.
When should I train a classifier?
Try training when you have reviewed examples, team-specific category boundaries and recurring work. Compare it with an assistant on the same held-out examples, including mistakes, review effort and operating cost.
Do I need thousands of examples to start?
There is no single number that guarantees a useful model. The number of categories, their overlap and the diversity of examples matter. With few labeled examples, an assistant can help prepare a sample for your team to review.
Does StayCharted send classification data to Claude or OpenAI?
Direct AMT file and API classification does not require a general-purpose AI service. If you connect an assistant, information it reads through the connection is shared with its provider under your agreement with that provider.
Is a trained classifier automatically more secure or compliant?
No. Security and compliance depend on the service, deployment, access controls, contracts and your requirements. Review the data flow and governing documents for either approach.