Approval Rate Is a Vanity Metric

For months I walked into every review meeting leading with the same number: the approval rate of my sales agent. The agent drafts a reply, a human reads it, the human sends it without changing a word. That percentage was my headline.

It was a real number. It was honestly measured. And it was quietly steering the entire project in the wrong direction.

Approval rate is a QA metric wearing a business metric's clothes

Approval rate answers exactly one question: how often does the human agree with the machine?

That's a useful question. It belongs on the dashboard I stare at when I'm debugging the agent's writing — is the tone drifting, is the knowledge base stale, is some template misfiring on a case it was never meant to touch. It's a QA signal. It's how I know the model is behaving.

It says nothing about whether the company is better off.

The trap is that approval rate has every surface property of a business metric. It's a percentage. It goes up when things get better. It's cheap to compute, easy to chart, and it makes everyone in the room nod. So it gets promoted out of the debugging dashboard and into the steering meeting. And once a number is in the steering meeting, it stops describing the project and starts directing it.

The number goes up when you make the agent worse

Here's the part that took me too long to see.

I can raise approval rate any time I want. I just narrow the agent. Let it handle only the clean, obvious, low-stakes replies and route everything with a wrinkle to a human. Approval rate climbs toward the high nineties. The chart looks fantastic.

And the business gets nothing. The hard cases — the ones eating the team's actual hours — are all still sitting in a human's inbox. I've optimized the metric by shrinking the agent's job until it can't fail.

That's not a hypothetical. That's the gravitational pull the metric creates. I spent weeks tuning phrasing and templates to move approval up by a couple of points, and those points bought the company nothing. Nobody closed faster. Nobody handled more leads. I was polishing the QA number because the QA number was what I'd promised to report.

The metric didn't just fail to help. It cost me weeks.

A 50% approval rate can be an excellent agent

Say the agent drafts a hundred replies a week and a human sends half of them untouched. Fifty percent. On a slide, that reads like failure.

Now ask the only question that matters: compared to what?

If that team was writing thirty replies a week by hand before the agent existed, then a "failing" 50% agent just handled fifty. It nearly doubled the throughput of the function. The other fifty drafts weren't wasted either — a human edited them instead of staring at a blank compose window.

Who decided 50% is bad? Bad against what baseline? The metric never says. It's a ratio with no denominator that anyone cares about.

If your metric would read the same whether the agent handled ten leads or a thousand, it is not a business metric.

That's the test I use now. Approval rate fails it instantly. So does accuracy, so does "quality score," so does most of what makes it onto agent dashboards by default.

What I report instead

The switch was from measuring the agent to measuring the system it lives in. Four numbers, all of which have a denominator someone in the room actually cares about:

• Throughput. How many leads got handled this week, versus before the agent existed. Not how well — how many. This is the number that made my case in a single line where months of approval-rate charts hadn't.
• Velocity. How long a deal takes from first reply to signature. If the agent is real, this compresses. On the deal flow I built the agent for, [fill in: cycle time before → after].
• Coverage. What share of inbound gets any response at all, inside a window that still matters. Most teams discover their real problem here — not that replies are mediocre, but that a chunk of leads never got one.
• Hours returned. What the team stopped doing. This one is soft and I report it anyway, because it's the number the people doing the work actually feel.

Approval rate still exists in my world. It lives where it belongs: on the QA dashboard, next to the error logs, where I check it when something smells wrong. It is a tool for me to debug with. It was never a tool for the business to decide with.

Why this keeps happening

I don't think anyone reports approval rate out of vanity, exactly. It's the number that's easiest to get.

Your agent framework hands you approval rate on day one, for free, in the box. Throughput means going to the CRM, agreeing on what "handled" means, and finding a believable before-picture. Velocity means someone has to define the stages honestly. Coverage usually means admitting that nobody knows how many inbound leads quietly died last quarter.

So we report the metric the tool gives us instead of the metric the business needs, and then we're surprised when leadership doesn't feel the value. They're not being obtuse. We handed them a number about the model when they asked a question about the company.

The principle

Measure the system, not the model. The agent is a component; nobody buys a component. If a metric can't move when the business moves — and can't fall when the business stalls — it isn't reporting on the business, and it doesn't belong in the room where decisions get made.

If you're running an agent right now, try this before your next review: take your headline metric and ask whether it would look any different if the agent's volume dropped by 90%. If the answer is no, you have a QA number on the wrong slide. Go find the denominator.