Tag: Observability Layer

  • Agent observability: what to monitor once Agentforce is making decisions in production

    Agentforce doesn’t just answer questions anymore. Give it the right actions and it updates records, raises cases, triggers flows, and replies to customers without a human in the loop. That’s a different risk profile than a chatbot that reads back knowledge articles, and most Salesforce teams are still monitoring it like one.

    I’ve sat through more than one go-live retro where the agent technically worked, every demo passed, every test case went green, and three weeks later someone in support asks why forty cases got auto-closed with the wrong resolution code. Nobody noticed because nobody was watching for it. The agent wasn’t broken. It was doing exactly what it was told, just slightly wrong, over and over, quietly.

    Why your existing monitoring doesn’t cover this

    Debug logs tell you a flow ran. They don’t tell you why the agent picked that flow over three other actions it had access to, or what it decided a customer meant before it acted on it. Standard Salesforce monitoring is built around deterministic systems: a trigger fires, a field changes, you can trace cause to effect in a straight line. An agent’s decision path isn’t straight. It reasons across a prompt, grounds against your data, picks a topic, picks an action, and the only artifact left behind is usually the outcome, not the reasoning.

    That gap is where things go wrong without tripping a single existing alert.

    The failure mode that actually bites teams

    It’s rarely a hard crash. Hard crashes get noticed fast. The failure mode that costs you is the agent that’s technically functioning but drifting: answering slightly outside its intended scope, grounding on stale data, routing to the wrong queue often enough to matter but not often enough to show up in a weekly spot check. Three months in, someone finally asks “has anyone actually been reviewing this?” and the honest answer is no.

    What to actually watch

    Four categories matter more than the rest, and most orgs only build dashboards for one of them.

    Decision accuracy, not just completion

    Did the agent pick the right topic and the right action for the request, not just whether an action fired at all. “Action executed successfully” and “action executed correctly” are two different metrics, and Salesforce’s own logs will happily tell you the first without touching the second. You need a sample review process, human-reviewed, on a regular cadence, checking a slice of transcripts against what the right call actually was.

    Escalation and deflection rates, tracked together

    A rising deflection rate looks great on a leadership slide. It looks a lot less great if escalations to a human are quietly climbing at the same time because the agent is resolving the easy 80% and dumping the hard 20% back on a queue with no context attached. Track both numbers next to each other, not in separate reports that nobody cross-references.

    Grounding and data freshness

    If the agent is pulling from Data Cloud or a knowledge base to answer, you need visibility into what it actually grounded on for a given response, not just that grounding happened. Stale records, duplicate profiles, an out-of-date knowledge article that should’ve been retired last quarter. All invisible until you go looking at the source the agent actually cited.

    Cost and latency per conversation

    Every agent interaction has a token cost and a response time, and both move as usage scales or as someone tweaks a prompt. I’ve seen a single prompt change meant to improve tone quietly double average response length and the per-conversation cost along with it. Nobody noticed for a billing cycle. Put this on the same dashboard as the business metrics, not buried in a separate admin console nobody opens.

    Building the actual observability layer

    You don’t need a new platform bolted on day one. Start with what Salesforce already gives you and get disciplined about using it.

    • Turn on and actually review Agentforce’s built-in conversation transcripts and action logs, not just when something breaks.
    • Build a lightweight Apex or Flow logging layer around custom actions that captures the inputs the agent passed in, not only the result, so a bad decision is traceable back to what triggered it.
    • Stand up a dashboard that puts decision accuracy, escalation rate, and cost per conversation on one screen, reviewed weekly by someone with the authority to change the prompt or the action design, not just report the number.
    • Define what “wrong” looks like before go-live, in writing, with examples. Teams that skip this end up arguing about what counts as a failure after the data’s already ugly.

    None of this needs to be sophisticated to be useful. A spreadsheet with fifty reviewed transcripts a week beats a beautiful dashboard nobody reads.

    What good observability actually buys you

    It’s not a compliance exercise. A team that reviews agent decisions weekly catches a bad prompt change in days instead of discovering it from a customer complaint three months later. It also gives you the evidence to go back to the business and say the agent is handling 40% of a queue correctly, not just handling 40% of a queue. Those are different claims, and only one of them survives scrutiny when leadership asks for proof.

    The teams that get burned by Agentforce in production aren’t usually the ones who built a bad agent. They’re the ones who built a decent agent and then stopped watching it the moment it shipped.