Hi everyone - quick question for folks who have experimented with agents in GTM flows (beyond enrichment and basic automation) - has anyone been playing around with eval or verification harnesses and getting some good results? Basically if you find a need to double check any agent output before it writes back to your CRM or sends out outbound. Would love to learn the agent flavor of the month!
yes, verification is the whole game here. we've been looking at this from the skills angle — most teams don't even have a framework for what "good AI output" looks like before it hits the CRM. the reps who are best at this tend to have an explicit checklist: does the enrichment match what's on LinkedIn, is the signal actually recent, does the personalization reference something real. without that, agents just confidently write plausible-sounding garbage. would be curious what eval criteria you've seen work.
Hey Abesh, Yep been experimenting with this a fair bit. The approach that's worked best for us is using a second AI call as a judge layer right after the main agent output - basically asking a separate prompt to score or validate the output before it moves downstream. Simple but surprisingly effective at catching hallucinations and low quality outputs. For CRM writes specifically, adding a confidence threshold column in Clay works well - only rows that pass a certain score get sent through to HubSpot or Smartlead. Anything that fails gets flagged for manual review instead. The key is keeping the eval prompt super specific to what "good" looks like for your use case rather than generic quality checks. If you still need any help, you can always text me!
Thanks for the quick responses!
Ozan D. - so slightly redundant follow up/existential question here: do you think the productivity trade-off here has been good so far with the skills implemented or is it sort of going back to the square one with the pre-AI workflows? What I have seen working usually in my case is, for existing contacts who I have had conversations with - the meeting transcripts and internal CRM notes are usually a good grounding framework. But for pure "cold" outbound, it either requires the manual LinkedIn check paired with some additional website/post research or keeping it less personalized but at least accurate.
Saad A. This is what I have been thinking and seeing a few teams employ as well. Im hoping we move beyond the confidence score but don't have any better alts yet - we have been doing an internal score with our platform and comparing it to hubspot deal scores and usually get better answers, but sometimes its also not always worth the extra effort (case by case basis). I do find that at the end its almost like spending your tokens twice - one for the draft and one for verification
Yeah the double token spend is the real tradeoff - totally valid point. The way I've been thinking about it is you don't always need to verify everything, just the outputs that carry the most downstream risk like anything that writes to CRM or triggers an outbound send. Low stakes enrichment columns probably don't need the second pass. The HubSpot deal score comparison is smart though - using an existing signal as a sanity check rather than a full second AI call is way more efficient. Curious how much drift you're seeing between your internal score and HubSpot's?
drift between scores: so we have been employing a pretty different approach to just RAG based synthesis done by LLMs, so i dont think the comparison is fair (and i dont want to promote my product here haha) but for accounts with a lot of history, the scores are actually better rather than accounts with thin activity or fresh new accounts. Have you been seeing results from your judge eval layer that is worth continuing to invest in this area? As in, have you caught some egregious failures from the first pass that without the second pass there would be a massive mistake?
Yes 100% worth it for outreach copy - caught personalization lines referencing funding rounds that never happened or wrong companies entirely. Those going into sequences would have been bad. For low risk enrichment like job title formatting? Not worth the double spend at all. I only run the judge layer now on anything touching copy or CRM routing fields. Thin accounts being weaker makes total sense - less signal means more guessing from the model, which is exactly where failures spike.
This is exactly the wall I hit. I run an agent that writes LinkedIn teardowns from sourced data, and the failure mode wasn't bad writing it was confident, hallucinated numbers. What worked was a dumb-but-strict validator between the LLM and the output: every number/stat in the draft has to literally appear in the sourced facts, or it's blocked and never ships. Same for claims it shouldn't make. Not fancy regex + a facts allowlist but it caught the exact stuff that would've been embarrassing. Two lessons: (1) keep the harness deterministic, not another LLM "grading" the first that just adds a second thing to hallucinate. (2) Fail closed block and flag for a human, don't auto-"fix." Happy to share the actual validator logic if useful.
