Latency

Latency is made up of several steps: detecting that the caller has finished, converting speech to text, generating the reply, synthesising speech and carrying the audio over the telephone network. Every step takes time. Tool calls, such as a calendar query, are added to that.

In conversations between people, pauses are short. If the pause becomes noticeably longer, the conversation feels halting and callers start speaking again. Vendor figures often refer to a single component only. The only meaningful figure is a measurement across the whole chain, on a real phone and over several calls. Shorter latency is not always better: if the system answers too early, it cuts the caller off.