Speech-to-text, turn detection, the model and synthesis, running as one streaming pipeline over one WebSocket.
$25 of free credit on every new developer account. Sign up, take your token, start talking.
Transcription runs while the caller is still talking. Synthesis does not wait for a finished sentence: the first clause with a few words behind it goes to the voice. And when the caller cuts in, the agent stops on the word they actually heard, not the word it had written.
Illustrative playback. Struck words were generated but never reached the caller. Telephony arrives as 8 kHz mulaw and the browser as 16 kHz PCM, on the same pipeline, and the browser hears 24 kHz PCM back.
Voice activity detection only says there is sound on the line. A fine-tuned WavLM reads the raw waveform and returns the probability that the caller means to take the floor. That number decides, in about fifteen milliseconds. Not the detector, and not loudness.
And when it does stop, it stops on the word the caller actually heard. The client echoes playback marks, so the reply is cut there rather than where the model happened to be writing.
A multimodal language model for intent classification. It takes the caller's audio and an instruction you write in plain language, then decodes the answer as text instead of picking from a fixed set of classes.
* In alpha.
You name the topics the agent must not engage. Every sentence it writes passes two checks on the way out, and one that fails them is aborted before the caller hears it.
A conversation feels live under about a second of silence. Most systems wait for the whole answer before they open their mouth. Here a small model opens the reply while the large model writes the rest, so the first word lands at about half a second and the full answer follows in the same voice.