Classifying Outreach Replies Without Guessing

The only categories you need

Interested includes any request for price, availability, a call, a demo, or more information. Not now covers timing, budget cycles and "come back in the new year". Not interested covers a refusal with or without a reason. Unclear covers everything else, including out-of-office, bounces and one-word replies.

Adding categories feels like progress and usually is not. A taxonomy of twelve intention labels produces arguments about the boundaries and no better actions, because only the first two change what you do.

Confidence, and what to do with the middle

Most classification services return a label and a score. Set a high threshold for automatic action, and send everything below it to a manual queue. The queue is not a failure of the system; it is the part that keeps the system safe, and at cold-email volumes it is small.

Never auto-send anything that commits you: a price, a discount, a delivery date, an opt-out confirmation. A classifier can route those to a draft for a person to approve. The cost of a delay is minutes; the cost of an automated wrong commitment is a customer relationship.

• High confidence and interested: create a task, do not auto-reply.
• High confidence and not interested: suppress from this campaign only.
• High confidence and not now: schedule a revisit date.
• Low confidence: a person reads it, always.
• Any opt-out language: suppression regardless of confidence.

Measure the errors that matter

Track false negatives on interest: replies a person later found to be buying signals that the classifier missed. That number, not overall accuracy, tells you whether the system is costing you deals. A classifier with 80 percent accuracy and no missed buyers is better than one with 95 percent and a habit of missing short replies.

Also track the reverse: replies classified as interested that were not, because those waste sales time and inflate the pipeline. Both numbers come from a sample of manual review, which is why a small random sample should always be read even when the classifier is confident.

Worked example: the reply that said "send it"

A team automated classification across three mailboxes. A reply that said only "send it" was classified as unclear and queued, where it sat for two days. The buyer had meant "send the proposal", and by the time a person replied they had gone with another supplier. The classifier had not made an error it could see; it had made a judgement on two words.

The fix was not a better model. Short replies were reclassified as interested by rule, because at cold-email volume almost nobody writes two words unless they want something. The manual queue stayed, but anything under five words went straight to a person.

The same team then measured what they had been missing: about one interested reply in forty had been sitting in the unclear bucket. That was the real cost of the classifier, and it had been invisible because nobody measured the errors that mattered.

Label, typical trigger, and the safe action

The asymmetry to design around: a missed buyer is expensive and a mislabelled refusal is cheap, so bias the system towards escalation.

Set it up, then prove it

• 1Define the four labels with one example each before touching a classifier.
• 2Set the confidence threshold for automatic action high, not convenient.
• 3Route everything below it to a queue someone actually reads.
• 4Never automate a price, a commitment or an opt-out confirmation.
• 5Sample the confident results weekly and count missed buyers.