Low latency WebSocket real-time generation

This document will help you understand how to use Vocu's millisecond-level low-latency WebSocket real-time generation solution.

With this solution, regardless of the target text length, voice generation delay can be controlled to around 700ms.

Prerequisites

Please check in Vocu Creation Center's Developer Center - Quotas and Limits - Quotas to see if you have the millisecond-level low-latency concurrency quota.
If you don't have this quota, please click Need More Quota and discuss with the sales team to get details on quota and pricing for millisecond-level low-latency real-time generation.

Opening the Turbo Channel

Low-latency real-time generation needs to be done over a WebSocket channel.
First, you need to request the relevant API to open the Turbo generation channel.

Note that once Turbo is opened, the role cannot be changed.
You can only use the role selected when the channel was opened, and only v2.0+ roles are supported.

As stated in cancelAfter, you need to connect to the WebSocket address provided by wsUrl within 60 seconds, otherwise the channel will be automatically cancelled and you will need to reopen the Turbo channel.

Connecting to the Channel for Generation

After connecting to the channel, if successful, you will receive a 200 message.

You can use your client to send a JSON message to start generation.

You can use the promptId of any style available in the selected role to generate with another style.

If your synthesis_request is valid, the backend will immediately start the generation task and send audio_chunk messages as soon as they are available.

Here, hex is the hex-encoded PCM audio data, and pcm contains its parameter information.

If you receive an empty hex, it means that the current audio segment has finished generating.

At the end of each generated audio segment, you will receive:

While audio is being generated, you can still send new synthesis_request messages. All generation requests will be queued and processed in order.

Keeping the Connection Alive

After the WebSocket connection is successfully established, please send the following at regular intervals:

The server will respond immediately:

If the connection is idle for too long (~1 minute), it may be closed to free resources.

If you wish to keep the connection alive, send ping messages periodically.

Closing the Channel

If there are no client connections for 60 seconds, the channel will close automatically.

You can also send the following message to close the channel immediately:

When the channel closes, you will receive:

Once the channel is closed, all connections will be terminated immediately and the occupied concurrency will be released.

Managing Turbo Channels via HTTP API

This request lists all currently active Turbo channels in your account.
Example response:

This request closes the channel specified by :id.

Low latency WebSocket real-time generation

Prerequisites#

Opening the Turbo Channel#

Connecting to the Channel for Generation#

Keeping the Connection Alive#

Closing the Channel#