Measuring VoiceBoTs and Humans in Conversations

The guardrails for Discovery, Build and Optimisation

At CXEX, our AutoInsights platform has been measuring human conversations for over six years across Australia, South Africa, the UK, and the US. As our name suggests, we focus on Customer Experience (CX) and Employee Experience (EX), using real-world voice data to unlock insights and drive performance.

The Evolution of Human Conversation Analytics

• Keyword spotting: Too many false positives, no context.
• Machine Learning: A big step forward, scalable, but still couldn’t match human nuance.
• LLMs: Game-changer. Since late 2022, we’ve shifted fast to LLMs, unlocking depth and accuracy in understanding what customers say and how they say it.

VoiceBoTs: Hype, Hope and Hard Lessons

In parallel, the VoiceBoT landscape has exploded. There are now thousands of platforms promising to revolutionise CX through automation. The marketing is impressive — but the execution? Not always.

Take a recent example: a 200-seat healthcare provider opted for a BoT-first approach, dismissing our tech as “old.” Six months and significant budget later, they couldn’t answer the most basic question: “Why are customers calling?” Now they’re starting over.

The lesson? VoiceBoTs don’t magically fix CX. You need data, structure, and insight first.

What Goes Wrong?

• No clear ROI or prioritisation framework: Not everything should be automated.
• No usable training data: Real-world calls are messy, nuanced, and full of hidden signals.
• No feedback loop: Bots are launched, but not optimised — escalations increase, CX and EX both suffer.

Measuring VoiceBoT Performance

Just like with humans, BoTs need measurement. Did the BoT follow compliance steps? Did it handle curveball questions or hallucinate? When and why did it escalate?

We measure:

• Performance outcomes: Was the issue resolved?
• Satisfaction drivers: Was the experience smooth, empathetic, professional?
• Operational impact: How efficient was the journey?

Because the truth is, CX fails when a bot fumbles and the agent inherits the mess. That’s bad for the customer and your employee.

CXEX’s Role: The Link Between Automation & Experience

We don’t build BoTs – but we make them work. Our approach:

1. Discovery

Use AutoInsights to structure the unstructured. Understand why people are calling, what gets resolved, and what doesn’t. This forms the foundation of your automation strategy – grounded in real conversations, not assumptions.

2. Build

Once a BoT is selected, we help train it using your real-world data. Not synthetic scripts or hypothetical intents – actual transcripts, with context and nuance. We also flag what can’t be automated (often due to tech limitations like CRM integration).

3. Optimise

Post-launch, we continuously measure BoT and human interactions: what worked, what didn’t, and how the handover performed. This insight is critical to refine the experience and realise ROI.

We’re not chasing the “gold rush” like 4,500 other players. We’re providing the picks and shovels – the infrastructure that ensures automation delivers on its promise.