b25urvashi

Jonatan, this is a really interesting distinction — especially the “plausible quiet failure” point. The idea of capturing the decision context and triggering conditions at the moment of execution makes a lot of sense. Curious: when you’ve seen teams implement this, what signals have actually been useful for detecting those “nothing errored, but something changed” cases?

Agree. If every execution generates a list of “possible issues,” the signal quickly gets buried in noise. What would make an AI-generated diagnosis trustworthy enough for your team to actually act on it — high confidence only, evidence from multiple executions, correlation with business impact, or something else?

Yes, that's the direction I'm thinking about as well. Observability is increasingly becoming the input to AI-assisted diagnosis rather than the end goal. I'll take a closer look at what Sentry is doing here. One thing I'm also curious about is whether these systems will eventually move beyond identifying likely causes to actually helping teams prevent the same failure from recurring — for example, by identifying patterns across executions rather than just analyzing one trace at a time. Thanks, this is really helpful perspective.

ok, since goal is not necessarily to reproduce the exact same outcome, but to have enough execution evidence to understand what happened. When you give that context to AI to investigate the issue, how do you establish the causal chain across a longer workflow — for example, which earlier context, retrieval, tool result, or state transition ultimately led to the bad decision. Is that something the current observability tools handle well, or does the AI still have to infer the chain from the available traces?

That makes sense. So the logbook is more about the agent's reasoning, while Langfuse/Mastra give you the execution trace. Where I'm curious is what happens when you need to investigate a genuinely bad outcome in a longer-running agent workflow — for example, figuring out exactly which context/retrieval or tool result caused the agent to take the wrong path, and then reproducing or replaying that execution. Do the existing tools give you enough to do that today, or is that still somewhat manual?

That's interesting — thanks. When you use the logbook approach, how reliable is it for investigating an unexpected outcome? For example, if an agent makes a wrong decision, can you reliably reconstruct the actual context/retrieval, model/tool calls, retries and state that led to it, or is the logbook more of the agent's own explanation of what it remembers doing? Happy to read your post.