Ask most teams running Agentforce how their agent is performing, and you’ll get a number like “92% response rate” or “agent handled 4,000 conversations last month.” Those numbers tell you almost nothing. An agent can respond to every single message and still be wrong a third of the time, confidently, in ways that look fine in a transcript and cost you a customer three days later.
I’ve sat in enough go-live reviews now to notice the pattern. The dashboard shows green. Containment is up. Average handle time is down. Everyone’s happy, because the metric being tracked is “did it answer,” not “was the answer correct, grounded, and the right thing to do.” Those are three different questions, and most Salesforce orgs I’ve looked at are only measuring the first one.
Why “did it answer” is the wrong starting point
An agent that answers every query has solved a UX problem, not a business one. It’s trivially easy to get a high response rate out of Agentforce: keep the topics broad, keep the instructions permissive, let the model reach for an answer rather than escalate when it’s unsure. The number goes up. So does the rate of plausible-sounding nonsense reaching your customers.
The honest version of this metric is closer to “did it answer correctly, with grounding it could point back to, and take the right action if one was needed.” That’s harder to measure and much more useful. A contact-centre team I worked with early in their Agentforce rollout had a 97% response rate and a customer satisfaction score that was quietly dropping. Nobody could explain the gap until someone actually read fifty transcripts end to end instead of trusting the dashboard. The agent was answering. A good chunk of the answers were wrong.
The metrics that actually tell you something
Task completion against a ground truth set
Build a set of representative conversations with known correct outcomes before you go live, not after. For each one, define what “correct” looks like: the right topic matched, the right action taken, the right answer given. Run this set every time you change a prompt, an instruction, a topic, or an action’s input schema, and diff the results. If you’re not doing this, you’re flying on vibes every time someone edits a system prompt in Prompt Builder, and those edits happen more often than anyone wants to admit.
Groundedness, not just fluency
Track what fraction of factual claims in agent responses can be traced back to a retrieved source, a Knowledge article, a record field, a Data Cloud object, rather than generated from the model’s general training. This is the single best predictor of hallucination risk I’ve found, and it’s rarely on anyone’s dashboard. If your retrieval is returning the wrong article half the time, no amount of prompt tuning fixes the answers downstream.
Action success rate, measured at the system, not the chat window
An agent can tell the customer “I’ve updated your address” while the underlying Apex action silently failed validation and rolled back. If your success metric is “the agent said it worked,” you’ll never catch this. Instrument the actions themselves. Log every invocation, every input, every outcome, and reconcile what the agent claimed against what actually happened in the org. This is boring instrumentation work and it matters more than any prompt change you’ll make this quarter.
Escalation correctness
Measure both false negatives and false positives here. An agent that never escalates is dangerous. An agent that escalates everything slightly ambiguous is expensive and defeats the point of building it. The useful number is: of the conversations that should have gone to a human, how many actually did, and of the ones that got escalated, how many genuinely needed to be.
Override and correction rate
When a human agent picks up a conversation the AI started, how often do they have to walk it back versus just carry on with it? A high override rate on specific topics is the clearest signal you’ll get about where the agent’s instructions, grounding, or action design need work. Break this down by topic, not just an aggregate number, because the aggregate will hide exactly the pocket of failure you need to see.
Build the measurement before you need it
The mistake I see most often isn’t choosing the wrong metrics. It’s not having any of this in place until something’s already gone wrong in production. By then you’re debugging with incomplete logs and a customer complaint instead of a dataset you control.
Before an Agentforce agent goes anywhere near a customer, you want three things sitting in place: a golden dataset of representative conversations with defined correct outcomes, instrumentation on every action that records inputs and actual results separately from what the agent said happened, and a recurring human audit, even a small one, of real transcripts rather than just the summary metrics. None of this is exotic. It’s the same discipline you’d apply to any integration that writes data into production, except agent QA tends to get skipped because the output is a sentence instead of a database row, and sentences feel less risky than they are.
What leadership should actually be asking for
If you’re the technical lead on an Agentforce build, push back on any reporting that stops at “agent handled X conversations.” Ask for the breakdown by topic. Ask what fraction of answers are grounded versus generated. Ask what the override rate looks like on the three most common intents. These questions take longer to answer properly, and that’s exactly why they’re worth asking. A response rate tells you the agent is busy. It doesn’t tell you whether it should be trusted with the next ten thousand conversations, and that’s the only question that actually matters once you’re past the pilot.
Leave a Reply