Exploring How RevOps Teams Understand AI Agent Actions and Decision Context in Production

b25urvashi · 2026-08-04T13:55:15.588Z

Hi everyone, Urvashi here. I’ve spent most of my career in engineering leadership across cloud platforms and enterprise technology, and I’m now exploring a new problem around AI agents in production. I’m particularly interested in how RevOps teams are using AI agents that can actually take actions across CRM, email, enrichment, support, and other systems. One question I’m exploring is: when an AI agent does something unexpected, how do teams figure out why it happened — what context it used, what decisions it made, and which action ultimately led to the outcome? I’m doing some customer discovery around this and would love to learn from people here who are already deploying AI into real RevOps workflows. Looking forward to connecting with everyone!

tom704
·
·
Hi Urvashi
tom704
·
·
In claude managed agents, you can instruct them to keep a logbook of their actions in their memory and why they did certain things
tom704
·
·
also in general their logging/monitoring is quite nice
tom704
·
·
i wrote a post about this on my linkedin
b25urvashi
·
·
That's interesting — thanks. When you use the logbook approach, how reliable is it for investigating an unexpected outcome? For example, if an agent makes a wrong decision, can you reliably reconstruct the actual context/retrieval, model/tool calls, retries and state that led to it, or is the logbook more of the agent's own explanation of what it remembers doing? Happy to read your post.
tom704
·
·
The logbook is indeed more for debugging the reasoning of your agent
tom704
·
·
if you want to monitor tool calls, context etc
tom704
·
·
I would suggest Langfuse, Mastra
tom704
·
·
However, if you use Claude Managed Agents this can be observed out-of-the-box by looking at the session of the agent
b25urvashi
·
·
That makes sense. So the logbook is more about the agent's reasoning, while Langfuse/Mastra give you the execution trace. Where I'm curious is what happens when you need to investigate a genuinely bad outcome in a longer-running agent workflow — for example, figuring out exactly which context/retrieval or tool result caused the agent to take the wrong path, and then reproducing or replaying that execution. Do the existing tools give you enough to do that today, or is that still somewhat manual?
tom704
·
·
In essence agents will always be probabilistic, so reproducing it exactly the same is hard/impossible. However, this gives all the best context you can use. You don't have to debug it manually yourself, you can give this context to AI to help you find the issue
b25urvashi
·
·
ok, since goal is not necessarily to reproduce the exact same outcome, but to have enough execution evidence to understand what happened. When you give that context to AI to investigate the issue, how do you establish the causal chain across a longer workflow — for example, which earlier context, retrieval, tool result, or state transition ultimately led to the bad decision. Is that something the current observability tools handle well, or does the AI still have to infer the chain from the available traces?
tom704
·
·
You still have to infer that from the traces. However, it's a matter of time these tools will start giving suggestions of where it went wrong
tom704
·
·
I think Sentry.io is making first steps in here
b25urvashi
·
·
Yes, that's the direction I'm thinking about as well. Observability is increasingly becoming the input to AI-assisted diagnosis rather than the end goal. I'll take a closer look at what Sentry is doing here. One thing I'm also curious about is whether these systems will eventually move beyond identifying likely causes to actually helping teams prevent the same failure from recurring — for example, by identifying patterns across executions rather than just analyzing one trace at a time. Thanks, this is really helpful perspective.

Exploring How RevOps Teams Understand AI Agent Actions and Decision Context in Production

21 comments

Exploring How RevOps Teams Understand AI Agent Actions and Decision Context in Production

21 comments