Classifying text in Snowflake and Databricks: getting the most from AI_CLASSIFY and ai_classify, and when to add a trained model

Snowflake AI_CLASSIFY

SELECT ticket_id,
       AI_CLASSIFY(
         body,
         [{'label': 'Billing', 'description': 'charges, refunds, invoices'},
          {'label': 'Login', 'description': 'passwords, two-factor, locked accounts'},
          {'label': 'Bug'}],
         {'task_description': 'Classify a customer support ticket by what the customer needs.',
          'examples': [{'input': 'I was charged twice this month',
                        'labels': ['Billing'],
                        'explanation': 'a duplicate charge'}]}
       ):labels[0]::STRING AS category
FROM support_tickets;

• Describe each category in a few words (up to 25 words each), and add a one-line task description. Both help.
• Add a real example per category from rows your team already categorized. In our test, examples lifted accuracy from 82.6% to 84.2% on 21 categories.
• Keep the list short where you can. Snowflake's documentation notes that more than about twenty categories may reduce accuracy. Group categories first, then classify within a group.
• Watch for blanks. A call that fails returns NULL rather than stopping the query.
• Mind the cost. Category names, descriptions and examples are sent with every row, and billed as tokens on every row. Count tokens on a sample first with AI_COUNT_TOKENS.
• On a trial account, AI functions are off until a payment method is added.

Databricks ai_classify

SELECT ticket_id,
       r:response[0].value::STRING AS category,
       r:response[0].confidence_score::DOUBLE AS confidence
FROM (
  SELECT ticket_id,
         ai_classify(
           body,
           '{"Billing": "charges, refunds, invoices", "Login": "passwords, two-factor, locked accounts", "Bug": ""}',
           MAP('version', '2.1', 'enableConfidenceScores', 'true',
               'instructions', 'Classify a customer support ticket by what the customer needs.')
         ) AS r
  FROM support_tickets
);

• Pin version 2.1 and turn on confidence scores: they're what let you send only the unsure rows to a person.
• Use label descriptions and the instructions option for context, as with Snowflake.
• Up to 500 labels are allowed; for very large taxonomies, Databricks suggests narrowing the candidates first.
• Check compute and regional support. It is not available on SQL Classic. On Databricks Runtime, the documented minimum is 15.4 LTS; check current regional requirements.

These examples use a support_tickets table with ticket_id and body. Check them on a small sample in your own account before a full run. Inspect NULL/error results as well as successful classifications; a confidence score is not a guarantee of correctness.

Whichever function you use, test it the same way before it drives a report or a workflow:

1. Take a few thousand rows your team has already categorized, ideally the most recent.
2. Run the function on them without the answers, exactly as you will in production.
3. Compare overall and per category. A good overall figure can hide categories that are almost always wrong.
4. If you have confidence scores, check the share of rows above each cutoff and how many of those are right. That tells you how much can go straight through.

Keep test rows separate from the examples used in prompts or training. Count failed or blank answers in the results, and inspect frequent confusions.

We tested public NHTSA vehicle complaints at two levels: 21 vehicle systems and 72 offered subsystem labels, with 70 subsystems represented in the test. Each level contains 5,000 complaints received from January 2025 onward. StayCharted trained on 19,902 older system examples and 19,281 older subsystem examples. All answers were compared with NHTSA’s filing; blanks count as wrong.

Databricks scores are not available yet, so this table does not rank Databricks against either platform. Its SQL example above explains the documented function, not a measured result.

What we took from it:

• On 21 broad categories, Snowflake did well. Adding a task description and examples raised the score from 82.6% to 84.2%, close to the StayCharted AI Classifier Model’s 84.1%. The Dedicated model scored 87.7%—3.5 percentage points higher using the displayed rounded figures.
• On closely related subsystems, the measured gap was larger. Snowflake scored 52.7% with labels and a task description; the Dedicated model scored 85.9%, or 1,660 more correct answers out of 5,000. This comparison did not test Snowflake with subsystem examples. It cannot establish how every prompt or model configuration would perform.
• Confidence adds a review decision. At a 0.90 cutoff, the Dedicated model covered 71.5% of subsystem complaints with 94.3% accuracy within that share. The remaining 28.5% would need review. Human-review outcomes were not measured. Snowflake AI_CLASSIFY returns labels without a confidence score; Databricks 2.1 supports confidence scores, but we have not measured its coverage here.

The Snowflake runs were scored on 30 October 2026. Blank answers numbered 52 for the system labels-only run, 41 for the system run with examples, and 14 for the subsystem run. The Dedicated subsystem result is the v1.0.65 rerun with no blank answers. Examples for Snowflake came from the training data, not the test rows. Category lists and examples are different from training on all earlier complaints; this is a comparison of the tested workflows.

Data preparation, time split and full trained-model results →

Keep the built-in function when the categories are few, broad and say what they mean, and when an occasional wrong answer is cheap.

Consider a model trained on your own decisions when:

• You have many categories, or close ones: dozens of product types, failure codes, account and department combinations, ticket queues.
• The categories are your company's own, with rules nobody wrote down, so names and descriptions can't carry them.
• You need to automate part of the work, which takes a confidence you can trust and a queue for the rest.
• You already have history: a few hundred to a few thousand rows your team has categorized.

The two work together. You can keep the built-in function for broad, low-stakes tagging and use a trained model for the classifications that drive money, compliance or customer outcomes.

StayCharted AI Model Trainer learns from rows your team already categorized. It returns a category, confidence and review flag for new rows. On Essentials and Business, answers below the model’s cutoff go to the Review Queue. Reviewers’ corrections can train the next version; retrain and publish before expecting them to change future answers.

• Export, fill, load back: export CSV rows, train and fill in StayCharted, then load the reviewed results into a warehouse table. Preserve a stable row key so your SQL can join the results back.
• Call the Prediction API: your integration can call a published model from warehouse-connected code with approved external network access. Store credentials securely and handle rate limits, failures and retries in the integration. See the API reference.

Your source tables can remain in the warehouse, but the rows sent for training or classification are processed by StayCharted. This is not computation entirely inside your warehouse. Check your data-handling requirements before sending them.

Start free → · Security and your data →