Until recently, talking to a voice agent could quickly feel artificial. After each answer, callers sometimes had to wait several seconds. The agent might start speaking at the wrong time, struggle with interruptions or follow a conversation that felt too rigid.
That experience is changing. With speech-to-speech, some agents process voice directly and respond aloud. Conversations can feel more natural, with less waiting between turns.
I have tested several agents using this approach and already offer it to my clients. In my tests, the wait is sometimes almost imperceptible, making it easier for callers to move through their requests.
How did a voice agent use text-to-speech?
In a chained architecture, several stages work together: the caller’s speech is transcribed into text; the agent processes the request and may consult a tool; it then prepares a written response, which is converted into speech.
Text-to-speech, or speech synthesis, is that last stage: it reads text aloud. Turning speech into text is called speech-to-text.
This arrangement lets developers work separately on transcription, processing and voice. It can also provide intermediate text for checking or passing to other tools.
Where does the wait during a call come from?
Latency is the time between an action and the system’s response. On the phone, callers notice it most between the end of a sentence and the start of an answer.
Detecting the end of a sentence, transcribing speech, processing the request, consulting a tool, generating speech and transferring data between services can each contribute to that wait. These delays do not always add up in full: some systems begin processing or streaming a response before the previous stage has completely finished.
Even so, a short wait can make a caller wonder whether the agent understood, repeat the question or start speaking just as the answer begins.
What changes with speech-to-speech?
With speech-to-speech, the model receives audio and produces an audio response. It can process the spoken conversation more directly, without necessarily chaining together separate transcription, text processing and speech synthesis components.
Some transitions between components can therefore disappear from the voice pathway. This can reduce the wait before an answer and make turn-taking feel more natural. The model can also make use of some information in the sound beyond the words alone.
Latency does not disappear altogether. The agent still has to listen, process and respond. If it checks availability or records a booking, the response times of the software, network and their connection also matter. The technology does not automatically make every connected tool faster.
In my tests, the difference is quickly noticeable: the conversation flows more smoothly and the wait is sometimes almost imperceptible.
What could this mean for a business?
On Rue Saint-Nicolas, Rue du Palais or elsewhere in La Rochelle, a business may face the same question: how do you answer the phone when you are already helping a customer? A voice agent can offer one answer, provided it is designed around the calls the business actually receives.
A caller might ask: “There are six of us on Saturday, including two children. Is there any availability, and do we need to book in advance?”
An agent designed for the activity can hold a conversation, collect useful information and answer the questions it has been set up to handle. If connected to the appropriate system, it can also check availability and make a booking. It can ask for clarification or transfer the call to a person when needed.
Some of my clients who currently use a “press 1, 2 or 3” menu are beginning to consider agents that can take requests further. The actions available depend on each company’s tools and agreed rules.
Is a phone menu still useful?
Yes. An interactive voice response system, often called an IVR, remains useful when choices are simple, stable and predictable. A short, well-designed menu may be enough to route calls to sales or customer service.
A voice agent becomes interesting when requests vary, several questions are needed or an action must be completed. Both approaches can also coexist. The choice depends on the experience the business wants and what it actually wants the agent to handle.
More convincing voices also call for vigilance
More natural exchanges can make some artificial voices harder to distinguish immediately from a person. If a request is unusual or urgent, verify the caller’s identity through a known channel, do not share confidential codes, and call the organisation back directly.
For a business, trust also means defining the agent’s role, limiting what it can do and offering a way to speak to a person.
Should you switch to speech-to-speech?
Speech-to-speech can make conversations flow more smoothly by reducing some stages in the voice pathway. But a good agent must also understand the requests it is given, access the right information, use authorised tools and know when to hand over.
Through Automatyz, I help businesses in La Rochelle and elsewhere choose and set up a phone reception suited to their needs: a voice menu, an AI agent or a combination of both.
Would you like to find out whether a voice agent could help your business? Let’s talk about your calls and the requests you receive.