Know when your AI agent should hand over to a person, from the chats your team already took over

Escalation rules usually start as keywords ("cancel", "lawyer", "speak to a human") and a generic sentiment score. Neither captures your business:

• a frustrated customer asking an easy question may be fine with the agent;
• a calm one asking about a contract change needs a person at once;
• "I've already tried that" means something different on your product than anywhere else.

Your chat history already shows where your team decided to step in, and where the agent handled it well. That's the rule, written in examples.

It reads the latest turns of the conversation, newest first:

Customer: Still not working, this is the third time.
Agent: Let's try resetting the integration…
Customer: My invoice is wrong and my account was charged for 40 seats.

It answers in your categories, for example:

• Keep with the agent
• Hand over: billing
• Hand over: technical
• Hand over: at risk of cancelling
• Sales opportunity

With a confidence. Hand over when it's confident a person is needed; keep going when it's confident the agent has it. Use a human fallback for uncertain cases. Never let a prediction override a request to speak to a person or a mandatory escalation rule.

1. Export past conversations with their outcome

For each conversation, take the turns before the decision and what happened:

• taken over by a person, and which team;
• or resolved by the agent with a good rating.

Your chat platform's export or your support tool's ticket history usually has both.

Only include text from before the hand-over. Anything the person wrote afterwards gives the answer away.

2. Train

Upload the file and train an AI Classifier Model. The report shows how often it agreed with your team on conversations it never saw, outcome by outcome. Inspect both unnecessary handovers and missed escalations. Review disputed labels with your support team rather than assuming the model is right.

3. Ask it on every turn

After each customer message, your agent platform sends the latest turns to the model through the API, and gets back the decision and a confidence. Agents built on Claude or ChatGPT can call it as a tool through the StayCharted MCP connector.

4. Retrain on new outcomes

Every month, add the newest conversations and retrain. Evaluate the new version before publishing; retraining does not guarantee better escalation decisions.

Keep turns from the same conversation together when preparing an independent evaluation, so related messages do not leak across it. API access requires Essentials or Business, and every classified turn counts toward your allowance. Your platform implements the handoff.

• It doesn't talk to the customer. It decides when to hand over; your agent or platform does the handover.
• It reads the beginning of the text you send, so send the latest turns first.
• It doesn't replace your must-escalate rules. Keep hard rules for things like legal threats or safety, and use the model for the judgement calls between them.

Use reviewed examples that reflect your current rules. Check errors by category and sample confident answers as well as uncertain ones. Confidence is a model score, not a guarantee. Keep mandatory safety and access rules in your application. Best practices for training and review →.