Audio Agent Configuration Guide

This article walks through each parameter in the order of the panels from top to bottom on the configuration page: what it does, its value range and default, and which engine modes it applies to. While retaining general capabilities such as knowledge, databases, and memory, the Audio Agent adds five groups of voice-related settings: Voice Engine, Input, Output, Welcome Guidance, and Call Control.

If you are not yet familiar with the basic concepts of the Audio Agent and its three engine modes, we recommend reading the Audio Agent Overview first.

Voice Engine

At the top of the Voice Engine panel is the engine mode selector, where you can switch between Real-time model, ASR-LLM-TTS, and LLM+TTS. After switching, the models and parameters shown below change accordingly.

Real-time model (REALTIME)

A single real-time voice large model completes "listen — think — speak" end to end. Audio input and output are handled within the same model in a closed loop, with no separate speech recognition and synthesis in between, so latency is lowest and the tone is most natural.

In general, we recommend the Real-time model mode more: it has the lowest latency and the most natural tone, and suits most real-time voice scenarios. OpenAI's upcoming Live model not only supports full-duplex real-time voice but can also call a more capable text large model in the background for complex tasks, and will become the preferred solution for real-time voice going forward.

graph LR
  U([User voice]) --> RT[Real-time voice large model end-to-end processing]
  RT --> A([Voice reply])

The Real-time model consolidates recognition, inference, and synthesis into a single model, so it has the most streamlined configuration:

The Real-time model has no separate ASR / TTS stages, so it does not provide settings such as transcription instructions, voice language, or TTS symbol cleanup (removal / replacement)—these capabilities are internalized within the model.

ASR-LLM-TTS

A three-stage pipeline: voice is first recognized into text, passed to a text large model for inference, and then the reply is synthesized into voice. A model can be chosen for each of the three stages, making this the mode with the highest controllability.

graph LR
  U([User voice]) --> ASR[ASR speech recognition]
  ASR -->|Text| LLM[Text large model inference]
  LLM -->|Reply text| TTS[TTS speech synthesis]
  TTS --> A([Voice reply])

The stage-by-stage configuration is as follows:

For both Remove and Replace, separate multiple symbols with commas; paired brackets (such as (), 《》) can be entered as a single unit.

LLM+TTS

A multimodal LLM that supports voice input understands the voice directly, skipping the standalone ASR stage; the model's text reply is then passed to TTS for synthesis into voice.

graph LR
  U([User voice]) --> LLM[Multimodal large model directly understands voice]
  LLM -->|Reply text| TTS[TTS speech synthesis]
  TTS --> A([Voice reply])

The settings are LLM (including model parameters and identity prompt) and TTS (including Remove and Replace), with the same meanings as in ASR-LLM-TTS.

Input

The Input panel controls "how the Agent listens," centered on VAD (Voice Activity Detection) and attachment recognition.

VAD Control

VAD (Voice Activity Detection) determines how the Agent judges when the user has finished speaking and whether it can be interrupted. The adjustable parameters change with the engine mode and recognition mode; after switching, the form changes accordingly:

• Real-time model: uses the model's built-in VAD, and you can choose a recognition mode on the configuration page—Preset mode or Semantic mode, defaulting to Preset mode.
• ASR-LLM-TTS / LLM+TTS: use the platform's built-in VAD, which behaves the same as Preset mode and does not offer recognition-mode switching.

Preset mode

Segments sentences based on the audio signal: it judges whether the user has finished speaking by the volume threshold and silence duration, with intuitive and predictable behavior. Applies to the Real-time model (when Preset mode is selected) as well as ASR-LLM-TTS and LLM+TTS.

Semantic mode

Supported by the Real-time model only. The model judges whether the user has finished speaking by understanding the semantics, no longer relying on fixed volume and silence thresholds, making sentence segmentation more natural—well suited to spoken conversations with many pauses and a loose delivery.

Attachments

Output

The Output panel controls "how the Agent speaks," including background sound and buffered playback.

Background sound

Overlays ambient sound onto the call to make the voice conversation feel more immersive. We recommend adjusting it to a volume of around 0.2 to avoid drowning out the human voice.

Buffered playback

Welcome Guidance

Sets the opening performance at the start of a call, with separate configuration supported for multiple languages.

Call Control

Call control makes calls more natural and prevents idle, empty calls. By setting up a hang-up mechanism, it prevents unlimited occupancy and saves resources and cost.

Quiet-conversation control

When the silence threshold is triggered, it automatically follows up or plays a prompt to bring the user back into the conversation.

Hang up the call

Hang-up is triggered when either the maximum call duration or the maximum number of silences condition is met.

Shared Capability Panels

The Audio Agent also supports general Agent capabilities such as knowledge, databases, and memory, configured the same way as an ordinary Agent; refer to the related Agent documentation. Note the following:

• The Human Service panel currently shows "Coming soon" in the Audio Agent.
• Panels such as Tools and Workflow are not shown in the Audio Agent.

Configuration Comparison Across Modes

The items shown on the configuration page differ slightly across engine modes; compare them in the table below:

For how to set parameter values so they better fit your business, continue reading Best Practices.