How to categorize unlabeled data: group, name, review, then train

A supervised classifier learns from pairs: a row and the category your team gave it. With no categories at all, there is nothing to learn from. This is often called the cold start problem.

Labeling by hand is slow, but deciding what the categories should be is usually harder. Two people sorting the same 500 tickets will often draw the lines in different places. If you are still settling those lines, read why taxonomy matters first. If you want to know how many reviewed rows a model needs, see how many examples you need to train a classifier.

On the first-model card, beside uploading a labeled file, there is a third way to create a model: "Start from unlabelled rows". You upload rows with no category and choose one of two starting points.

Group, then name (text clustering into categories)

Grouping similar rows is what people usually mean by text clustering. AMT shows each group with how many rows it holds, the words that set it apart from the other groups and a few example rows, so you can tell what the group is about without reading all of it.

You stay in charge of what becomes a category:

• Name the groups you keep. Each named group becomes a category, and its rows are suggested that category.
• Merge by name. Give two groups the same name and they become one category.
• Drop what you don't need. A group you leave out is dropped.
• Split or ask for more. If a group mixes two things, split it, or ask AMT for more groups.

AMT starts with as few groups as the rows support. Finer categories come when you ask for them or make them yourself. In practice that is a sensible order: agree on a few broad categories first, then split the ones that matter.

Name, then suggest

If your categories are already fixed, for example by a finance code list or a routing plan, give AMT the names and a short description of what each one means. AMT suggests one for every row. Writing the descriptions well is worth the time: a line such as "Billing: questions about an invoice or charge, not about refunds" tells the reader where the boundary is.

Many files are partly labeled: someone started tagging, or an older system filled a column for some rows. Keep those rows in the file. Rows that already have a category count as reviewed, and the rest are suggested from them. That gives AMT your team's own decisions to work from, not only a list of names.

People review the suggestions before training. This step is the point of the feature, not a formality. A group can look clean in its example rows and still contain rows that belong elsewhere, and a suggested category is only as good as the names and descriptions behind it.

A practical way to review:

1. Read the example rows and the words that set each group apart before naming it.
2. Split any group that mixes two meanings you care about.
3. Check a random sample of suggested rows in each category, not only the ones that look obvious.
4. Correct what is wrong. Your corrections become the labels the model learns from.

Once you train, AMT tests the model on held-out rows it never saw and reports accuracy overall and per category, confusions, and the share of answers above each confidence cutoff with how many of those are right. That held-out test is where you get real numbers for your data. If you want to keep improving the labels after the first model, pre-labeling training data covers that loop.

Have people verify the labels in your held-out test set. Testing against unreviewed AI suggestions measures agreement with those suggestions, not accuracy against an independent answer key.

Text and pictures both work. For photos, the same two starting points apply: let AMT group similar pictures, or give it the category names and let it suggest one for each picture. Picture models need the Vision add-on.

"Start from unlabelled rows" is not on the Free plan. AMT reads the rows with a model:

• Essentials: with the AI Classifier Model, on a CPU.
• Business: with the Dedicated AI Model, on a GPU.

Reading counts toward your plan the same way a fill does. Each row counts toward rows filled a month, and each picture toward pictures read a month. Your usage shows a separate line, "reading unlabelled rows", so you can see what the reading used. It runs as a queued job and may not start at once.

As an example of the arithmetic, reading 8,000 unlabeled rows on Essentials uses 8,000 of that month's 25,000 rows filled. Full details are on the pricing page.

Nothing has been benchmarked for cold start. We do not publish an accuracy figure for grouped or suggested categories, and this post doesn't quote one. How good the first suggestions are will depend on your data: how distinct the categories are, how much each row says, and how clearly the names and descriptions draw the lines. Treat every suggestion as a draft for review, and judge the trained model by its held-out test, not by how tidy the groups looked.

• You already have labeled history. Upload the labeled file and train directly.
• You have only a few hundred rows. A person can often label them in an afternoon.
• You need two or three broad categories. Simple rules or a quick manual pass may be enough.

1. Export the rows you want categorized, with any categories you already have left in place.
2. Sign in to StayCharted AI Model Trainer and choose "Start from unlabelled rows" on the first-model card.
3. Pick a starting point: let AMT group the rows, or give it your category names and what each means.
4. Name, merge, split or drop groups, and review the suggestions.
5. Train, read the held-out test results, and set a confidence cutoff. Answers below it go to the Review Queue.

The full feature list is on the product page.