docs / latency-lab
Latency Lab
A caller judges your agent by the gap after they stop talking. Set each stage, read the waterfall, then listen to the gap.
Turn waterfall
Every turn is five stages in a row: network, endpointing, recognition, model first token and speech synthesis. The caller hears only the sum. Move a slider and the waterfall below shows which stage owns the silence.
Pause Player
Plays a caller beep, then your computed silence, then an agent beep. Turn on barge-in to hear the agent cut off when the caller speaks again.
Turn off silent mode and raise the volume first; on iPhone, silent mode can mute Web Audio. The Play button shows a pulsing dot while sound is playing.
Loop: off · Barge-in: off
Ready. Sound plays only after you press Play.
Reading the bands
- ≤ 800 ms
- Snappy. Close to the gap between people in a lively conversation.
- 801–1500 ms
- Acceptable. The pause is noticeable but callers usually wait, especially on task-focused calls.
- > 1500 ms
- Awkward. Expect "hello?", talk-over and hang-ups.
Stages explained
- network
- Round trip between the phone network, your server and the providers.
- endpointing
- How long the agent waits in silence before deciding the caller has finished. Often the largest hidden cost.
- recognition
- Time to finalize the transcript after speech ends.
- model
- Time to the first token from the language model.
- synthesis
- Time until the first audio chunk of the reply is ready to play.
The presets are illustrative starting points, not measurements of any vendor. Enter your own figures from your logs. The bands are rules of thumb, not guarantees.