How to train an AI classifier by chatting with Claude or ChatGPT

You can ask a general assistant to sort a few messages into Billing, Access and Bug. That can be useful for a first pass. But your team may have rules the assistant does not know: a refund request mentioning a login problem still belongs in Billing. A new prompt or conversation may lead to a different interpretation.

A trained classifier learns from examples your team already categorized. StayCharted tests it on separate examples and reports accuracy overall and by category. A published model version stays in use until you publish another. This makes the category decision repeatable without rewriting a prompt for every file. Test results do not guarantee future accuracy.

The assistant remains useful: it helps prepare data, explain reports and make corrections. Compare an assistant with a trained classifier.

Add StayCharted AI Model Trainer from Claude’s connector directory or its ChatGPT listing. Sign in to StayCharted and choose the workspace you intend to use.

The workspace Owner must allow assistants, and a Builder must enable access for the model. Both controls matter: connecting an assistant does not grant access to every model. Your connection acts with your own role. Follow the AI assistants setup guide for the current screens.

Start with a narrow request: “List the models I can access in this workspace.” Check that the workspace and model are the ones you intended.

For support routing, prepare a spreadsheet with a ticket description and its final approved queue. For invoice coding, use the line description and its approved category. Include representative examples of each answer you want the model to learn.

Ask: “Help me create a support-ticket classifier from my labeled file.” Upload the file using the link the assistant provides. The file goes through StayCharted’s upload flow; you do not need to paste thousands of rows into the conversation.

Ask it to confirm the input columns and answer column before proceeding. Remove identifiers that would let a model memorize a customer instead of learning the task. If several rows belong to the same customer or thread, choose a test split that keeps related rows together. See training best practices.

Ask: “Check the training data and explain anything I should fix before training.” Review missing answers, inconsistent categories and the sensitive-information checks for text. Detection is not exhaustive; your team still decides what data is appropriate to use.

Then ask: “Explain the training options and allowance needed before you train.” The assistant should show the proposed action and wait for your confirmation. Available model types and picture training depend on your plan; pictures require Vision.

After training, ask: “Summarize the report, including each category’s accuracy and the weakest categories.” Inspect the held-out test results and any mistakes. Fix data problems or collect better examples before relying on a weak model. Training and publishing are separate decisions: approve publication only after reviewing the report.

On Essentials and Business, StayCharted can group unlabeled rows and suggest categories for you to review. Ask your assistant to help you start from those rows and explain the groups. You still choose the category names and check the examples.

Suggested groups are a starting point, not verified answers. Write a labeling guide explaining difficult boundaries. For example, state whether a refund request always belongs in Billing even when another issue is mentioned. Approve the labels before training and test on independently reviewed examples.

Ask: “Use my published Support ticket routing model to categorize these new records.” For a large file, use the upload link and retrieve the filled file through the resulting link. The model returns categories and confidence scores. Files, Google Sheets and API integrations can use the same published model outside a conversation.

On Essentials and Business, answers below your model’s cutoff wait in the Review Queue. Ask your assistant to help you inspect the waiting answers. Confirm or correct them, then use reviewed examples when retraining. Confidence is a useful review signal, not a guarantee that an answer is correct.

The assistant can explain a proposed correction, but your team owns the category decision. Keep an eye on changes in incoming data and compare new versions on a suitable test set.

The workspace switch, model switch and your role limit access. File upload and download use StayCharted links, but the assistant can read data you request through the authorized connection, including model reports and permitted rows. Its provider receives what it reads under your agreement with that provider.

Training and publishing require your confirmation. Actions are recorded in Activity. Use the workspace and model controls to remove access when it is no longer needed. See Security and your data and the Privacy Policy.

A classifier is for repeated category decisions across your team’s records. It does not automatically extract document fields, forecast a number or manage a live camera pipeline.

For camera monitoring, object detection and industrial alerts, evaluate a computer-vision platform such as Roboflow. Compare StayCharted and Roboflow. For direct API development rather than an assistant connection, read Claude API vs OpenAI API.

Start with one repeated decision, a file of reviewed examples and a clear measure of success: which categories matter, what mistakes cost and which answers a person should check.