Qwen-Audio-3.0-Realtime-Plus is an Alibaba model for real-time voice conversation. It accepts speech and text, responds with audio and text, and supports a two-way conversation with natural turn-taking. Audio can be sent and received in chunks, allowing the interaction to develop during the conversation.
Several ways to detect turn boundaries are provided: voice activity, semantic completion and explicit control. The manufacturer's mode is selected before the first audio is sent; changing it after a conversation has started requires a new connection. Support for this control depends on the selected method. The semantic mode considers pauses and fillers: a brief hesitation does not necessarily mean that the user has finished their thought. New user speech can interrupt a response being generated so that the conversation continues in light of the next turn. In semantic mode, recognition can be guided by a speech sample of the selected speaker to reduce distraction from other voices and background noise.
Instructions define the assistant's role and conversational style. Vocal delivery can reflect tone, pace and emotion; version 3.0 provides five system voices and voices based on a speech sample. The model supports 11 languages, including Russian, as well as the Chinese dialect variants specified by the manufacturer. In the manufacturer's API, the voice is specified in the first update of the session settings; subsequent attempts to supply a new voice in the same conversation are ignored. These settings require support from the selected access method.
Function calling is supported. The model produces the requested action, the connected application executes it and returns the result to the conversation. This allows a conversation to be based on actual data and continued after an operation has been completed. A voice assistant can use tool results while retaining the context of the discussion. The manufacturer's capabilities also include built-in web search when provided by the selected method. Web search and function calls cannot both be enabled in the same session.
By default, the history contains 20 conversation turns, with a supported range of one to 50. Accumulated audio is limited to 300 seconds; older parts of the history are removed when the limits are exceeded. This differs from the length of an individual turn: the history determines which earlier material remains available as the dialogue continues.
The formal context window is 40,960 tokens. The maximum input length is 16,384 tokens and the maximum output length is 8192. The same input and output limits are specified for reasoning mode, with a separate reasoning-chain limit of 1024 tokens.
Streaming audio uses mono, 16-bit PCM: 16 kHz for input and 24 kHz for output. Text and audio belong to the same conversation. Visual input and image output are not stated for this particular model; it specializes in speech, text and voice interaction.
", "aboutText": "Qwen-Audio-3.0-Realtime-Plus is an Alibaba model for real-time voice conversation. It accepts speech and text, responds with audio and text, and supports a two-way conversation with natural turn-taking. Audio can be sent and received in chunks, allowing the interaction to develop during the conversation.\n\n### Knowing when a speaker has finished\n\nSeveral ways to detect turn boundaries are provided: voice activity, semantic completion and explicit control. The manufacturer's mode is selected before the first audio is sent; changing it after a conversation has started requires a new connection. Support for this control depends on the selected method. The semantic mode considers pauses and fillers: a brief hesitation does not necessarily mean that the user has finished their thought. New user speech can interrupt a response being generated so that the conversation continues in light of the next turn. In semantic mode, recognition can be guided by a speech sample of the selected speaker to reduce distraction from other voices and background noise.\n\nInstructions define the assistant's role and conversational style. Vocal delivery can reflect tone, pace and emotion; version 3.0 provides five system voices and voices based on a speech sample. The model supports 11 languages, including Russian, as well as the Chinese dialect variants specified by the manufacturer. In the manufacturer's API, the voice is specified in the first update of the session settings; subsequent attempts to supply a new voice in the same conversation are ignored. These settings require support from the selected access method.\n\n### Tools and conversation history\n\nFunction calling is supported. The model produces the requested action, the connected application executes it and returns the result to the conversation. This allows a conversation to be based on actual data and continued after an operation has been completed. A voice assistant can use tool results while retaining the context of the discussion. The manufacturer's capabilities also include built-in web search when provided by the selected method. Web search and function calls cannot both be enabled in the same session.\n\nBy default, the history contains 20 conversation turns, with a supported range of one to 50. Accumulated audio is limited to 300 seconds; older parts of the history are removed when the limits are exceeded. This differs from the length of an individual turn: the history determines which earlier material remains available as the dialogue continues.\n\n### Text context and audio\n\nThe formal context window is 40,960 tokens. The maximum input length is 16,384 tokens and the maximum output length is 8192. The same input and output limits are specified for reasoning mode, with a separate reasoning-chain limit of 1024 tokens.\n\nStreaming audio uses mono, 16-bit PCM: 16 kHz for input and 24 kHz for output. Text and audio belong to the same conversation. Visual input and image output are not stated for this particular model; it specializes in speech, text and voice interaction.", "editorialOverlay": { "copy_fields": null, "copy_sha256": null, "copy_source": null, "html_sha256": "fe944a7cd2182aa6d85e1ba32dd33fdfc8909047c46d6484f851ad611e3d1cf7", "module_sha256": "ee9dc1c50b1e6a8fb7fa062030fcb4646b9aaaeb08be513b723a5a92a7290f06", "raw_model_unchanged": true, "schema": "ailoot-native-model-editorial-v1" }, "locale": "en", "methods": [ { "html": "qwen-audio-3.0-realtime-plusA real-time voice conversation model that accepts speech and text and responds with voice or text. Designed for a natural conversational pace and interaction with tools.
This description covers the model’s capabilities. Available in Ailoot inputs, outputs and options are listed for the selected method below.
Qwen-Audio-3.0-Realtime-Plus is an Alibaba model for real-time voice conversation. It accepts speech and text, responds with audio and text, and supports a two-way conversation with natural turn-taking. Audio can be sent and received in chunks, allowing the interaction to develop during the conversation.
Several ways to detect turn boundaries are provided: voice activity, semantic completion and explicit control. The manufacturer's mode is selected before the first audio is sent; changing it after a conversation has started requires a new connection. Support for this control depends on the selected method. The semantic mode considers pauses and fillers: a brief hesitation does not necessarily mean that the user has finished their thought. New user speech can interrupt a response being generated so that the conversation continues in light of the next turn. In semantic mode, recognition can be guided by a speech sample of the selected speaker to reduce distraction from other voices and background noise.
Instructions define the assistant's role and conversational style. Vocal delivery can reflect tone, pace and emotion; version 3.0 provides five system voices and voices based on a speech sample. The model supports 11 languages, including Russian, as well as the Chinese dialect variants specified by the manufacturer. In the manufacturer's API, the voice is specified in the first update of the session settings; subsequent attempts to supply a new voice in the same conversation are ignored. These settings require support from the selected access method.
Function calling is supported. The model produces the requested action, the connected application executes it and returns the result to the conversation. This allows a conversation to be based on actual data and continued after an operation has been completed. A voice assistant can use tool results while retaining the context of the discussion. The manufacturer's capabilities also include built-in web search when provided by the selected method. Web search and function calls cannot both be enabled in the same session.
By default, the history contains 20 conversation turns, with a supported range of one to 50. Accumulated audio is limited to 300 seconds; older parts of the history are removed when the limits are exceeded. This differs from the length of an individual turn: the history determines which earlier material remains available as the dialogue continues.
The formal context window is 40,960 tokens. The maximum input length is 16,384 tokens and the maximum output length is 8192. The same input and output limits are specified for reasoning mode, with a separate reasoning-chain limit of 1024 tokens.
Streaming audio uses mono, 16-bit PCM: 16 kHz for input and 24 kHz for output. Text and audio belong to the same conversation. Visual input and image output are not stated for this particular model; it specializes in speech, text and voice interaction.
GET /v1/realtime?model=qwen-audio-3.0-realtime-plustext, audiotext, audio| Field | Type | Requirement | Values & limits | Default |
|---|---|---|---|---|
Fieldquery.model | Typestring | RequirementRequired | Values & limits qwen-audio-3.0-realtime-plus | Default— |
Fieldsession.input_audio_format | Typestring | RequirementOptional | Values & limits pcm16, pcm | Default— |
Fieldsession.output_audio_format | Typestring | RequirementOptional | Values & limits pcm16, pcm | Default— |
Fieldsession.instructions | Typestring | RequirementOptional | Values & limits Limit applies to: selected session policy | Default— |
Fieldsession.max_history_turns | Typeinteger | RequirementOptional | Values & limits Minimum: 1 Limit applies to: selected session policy | Default— |
Fieldinput_audio_buffer.append.audio | Typebase64_string | RequirementRequired | Values & limits Input content: PCM16 mono; 16 kHz input, 24 kHz output; nonempty even byte length | Default— |
realtime_text_audio
| Parameter | Contract |
|---|---|
Parametermaterial_types | Contracttext, audio |
Parameterrequired | Contractnull |
Parameteroptional_controls | Contractnull |
Parametercontent_type | ContractWebSocket |
Lifecycle: exact retained published method
Method shapes and IDs are exact; supplier capability is separate from admission.
GET /v1/realtime?model=qwen-audio-3.0-realtime-plusAiloot reserves an estimated amount before a request. After the result, it settles the final charge from measured usage. Streaming settlement uses the final usage record. How reservation and settlement work →
# Install the documented client once: python3 -m pip install websockets\nimport base64, json, os, time, wave\nfrom pathlib import Path\nfrom websockets.sync.client import connect\n\nkey = os.environ.get("AILOOT_API_KEY", "")\nif not key.strip():\n raise SystemExit("Set your AILOOT_API_KEY before connecting")\nfile = Path(os.environ.get("AILOOT_TEXT_FILE", ""))\nif not file.is_file(): raise SystemExit("Set AILOOT_TEXT_FILE to your actual UTF-8 material")\nmaterial = file.read_text(encoding="utf-8")\nif not material.strip(): raise SystemExit("Text material must not be empty")\nout = Path("realtime-results")\nout.mkdir(exist_ok=True)\naudio, texts, complete = {}, {}, []\ncurrent_response = None\npending = False\ndeadline = time.monotonic() + 570 # Local client bound; lower server/session limits still apply.\n\ndef receive(ws, log):\n global current_response, pending\n left = deadline - time.monotonic()\n if left <= 0:\n raise TimeoutError("Client session limit; preserve IDs and partial results, do not replay")\n event = json.loads(ws.recv(timeout=min(60, left)))\n log.write(json.dumps(event, ensure_ascii=False) + "\\n")\n log.flush()\n kind = event.get("type")\n if kind == "error":\n raise RuntimeError(json.dumps(event, ensure_ascii=False))\n if kind == "response.created":\n current_response = event["response"]["id"]\n if kind in {"response.audio.delta", "response.output_audio.delta", "response.text.delta", "response.output_text.delta", "response.audio_transcript.delta", "response.output_audio_transcript.delta", "response.text.done", "response.audio_transcript.done"}:\n identity = (event["response_id"], event["item_id"], event["output_index"], event["content_index"])\n if kind in {"response.audio.delta", "response.output_audio.delta"}:\n raw = base64.b64decode(event["delta"], validate=True)\n if len(raw) % 2: raise ValueError("Output PCM16 chunk must have an even byte length")\n audio.setdefault(identity, bytearray()).extend(raw)\n elif kind.endswith(".done"):\n texts[identity] = event.get("text", event.get("transcript", ""))\n else:\n texts[identity] = texts.get(identity, "") + event["delta"]\n print(event["delta"], end="", flush=True)\n if kind == "response.done":\n response = event["response"]\n complete.append(response)\n if current_response is None: current_response = response["id"]\n if response["id"] == current_response: pending = False\n return event\n\ndef wait_for(ws, log, kind):\n while True:\n event = receive(ws, log)\n if event.get("type") == kind: return event\n\ndef send(ws, event):\n ws.send(json.dumps(event, ensure_ascii=False))\n\ntry:\n # One exact model query; no reconnect/retry loop and no provider credentials.\n with connect("wss://api.ailoot.ai/v1/realtime?model=qwen-audio-3.0-realtime-plus", additional_headers={"Authorization": "Bearer " + key}) as ws:\n with (out / "events.jsonl").open("w", encoding="utf-8") as log:\n wait_for(ws, log, "session.created")\n send(ws, {"type":"session.update", "session":{"modalities":["audio","text"], "input_audio_format":"pcm", "output_audio_format":"pcm", "turn_detection":None}})\n wait_for(ws, log, "session.updated")\n send(ws, {"type":"conversation.item.create", "item":{"type":"message", "role":"user", "content":[{"type":"input_text", "text":material}]}})\n send(ws, {"type":"response.create"})\n pending = True\n try:\n while pending: receive(ws, log)\n except KeyboardInterrupt:\n # Cancel the actual current inference, then preserve its returned status.\n send(ws, {"type":"response.cancel"})\n while pending: receive(ws, log)\n if not complete or complete[-1].get("status") != "completed":\n raise RuntimeError("Response did not complete normally; inspect saved events and status")\nfinally:\n manifest = {"responses":complete, "parts":[]}\n for number, identity in enumerate(dict.fromkeys(list(audio) + list(texts))):\n record = {"response_id":identity[0], "item_id":identity[1], "output_index":identity[2], "content_index":identity[3]}\n if identity in audio:\n raw = bytes(audio[identity])\n pcm_file = out / ("audio-" + str(number) + ".pcm")\n wav_file = out / ("audio-" + str(number) + ".wav")\n pcm_file.write_bytes(raw)\n with wave.open(str(wav_file), "wb") as wav:\n wav.setnchannels(1); wav.setsampwidth(2); wav.setframerate(24000); wav.writeframes(raw)\n record.update({"pcm":str(pcm_file), "wav":str(wav_file), "sample_rate_hz":24000})\n if identity in texts:\n text_file = out / ("text-" + str(number) + ".txt")\n text_file.write_text(texts[identity], encoding="utf-8")\n record["text"] = str(text_file)\n manifest["parts"].append(record)\n (out / "results.json").write_text(json.dumps(manifest, ensure_ascii=False, indent=2), encoding="utf-8")\n print("\\nSaved every received event and result in", out)\n# Install the documented client once: python3 -m pip install websockets\nimport base64, json, os, time, wave\nfrom pathlib import Path\nfrom websockets.sync.client import connect\n\nkey = os.environ.get("AILOOT_API_KEY", "")\nif not key.strip():\n raise SystemExit("Set your AILOOT_API_KEY before connecting")\nfile = Path(os.environ.get("AILOOT_AUDIO_FILE", ""))\nif not file.is_file(): raise SystemExit("Set AILOOT_AUDIO_FILE to your raw PCM16 mono 16-kHz recording")\nmaterial = file.read_bytes()\nif not material or len(material) % 2 or material.startswith(b"RIFF"):\n raise SystemExit("Use nonempty signed little-endian PCM16 mono 16-kHz bytes, not a WAV header")\nout = Path("realtime-results")\nout.mkdir(exist_ok=True)\naudio, texts, complete = {}, {}, []\ncurrent_response = None\npending = False\ndeadline = time.monotonic() + 570 # Local client bound; lower server/session limits still apply.\n\ndef receive(ws, log):\n global current_response, pending\n left = deadline - time.monotonic()\n if left <= 0:\n raise TimeoutError("Client session limit; preserve IDs and partial results, do not replay")\n event = json.loads(ws.recv(timeout=min(60, left)))\n log.write(json.dumps(event, ensure_ascii=False) + "\\n")\n log.flush()\n kind = event.get("type")\n if kind == "error":\n raise RuntimeError(json.dumps(event, ensure_ascii=False))\n if kind == "response.created":\n current_response = event["response"]["id"]\n if kind in {"response.audio.delta", "response.output_audio.delta", "response.text.delta", "response.output_text.delta", "response.audio_transcript.delta", "response.output_audio_transcript.delta", "response.text.done", "response.audio_transcript.done"}:\n identity = (event["response_id"], event["item_id"], event["output_index"], event["content_index"])\n if kind in {"response.audio.delta", "response.output_audio.delta"}:\n raw = base64.b64decode(event["delta"], validate=True)\n if len(raw) % 2: raise ValueError("Output PCM16 chunk must have an even byte length")\n audio.setdefault(identity, bytearray()).extend(raw)\n elif kind.endswith(".done"):\n texts[identity] = event.get("text", event.get("transcript", ""))\n else:\n texts[identity] = texts.get(identity, "") + event["delta"]\n print(event["delta"], end="", flush=True)\n if kind == "response.done":\n response = event["response"]\n complete.append(response)\n if current_response is None: current_response = response["id"]\n if response["id"] == current_response: pending = False\n return event\n\ndef wait_for(ws, log, kind):\n while True:\n event = receive(ws, log)\n if event.get("type") == kind: return event\n\ndef send(ws, event):\n ws.send(json.dumps(event, ensure_ascii=False))\n\ntry:\n # One exact model query; no reconnect/retry loop and no provider credentials.\n with connect("wss://api.ailoot.ai/v1/realtime?model=qwen-audio-3.0-realtime-plus", additional_headers={"Authorization": "Bearer " + key}) as ws:\n with (out / "events.jsonl").open("w", encoding="utf-8") as log:\n wait_for(ws, log, "session.created")\n send(ws, {"type":"session.update", "session":{"modalities":["audio","text"], "input_audio_format":"pcm", "output_audio_format":"pcm", "turn_detection":None}})\n wait_for(ws, log, "session.updated")\n for offset in range(0, len(material), 640):\n send(ws, {"type":"input_audio_buffer.append", "audio":base64.b64encode(material[offset:offset+640]).decode("ascii")})\n time.sleep(0.02)\n send(ws, {"type":"input_audio_buffer.commit"})\n wait_for(ws, log, "input_audio_buffer.committed")\n send(ws, {"type":"response.create"})\n pending = True\n try:\n while pending: receive(ws, log)\n except KeyboardInterrupt:\n # Cancel the actual current inference, then preserve its returned status.\n send(ws, {"type":"response.cancel"})\n while pending: receive(ws, log)\n if not complete or complete[-1].get("status") != "completed":\n raise RuntimeError("Response did not complete normally; inspect saved events and status")\nfinally:\n manifest = {"responses":complete, "parts":[]}\n for number, identity in enumerate(dict.fromkeys(list(audio) + list(texts))):\n record = {"response_id":identity[0], "item_id":identity[1], "output_index":identity[2], "content_index":identity[3]}\n if identity in audio:\n raw = bytes(audio[identity])\n pcm_file = out / ("audio-" + str(number) + ".pcm")\n wav_file = out / ("audio-" + str(number) + ".wav")\n pcm_file.write_bytes(raw)\n with wave.open(str(wav_file), "wb") as wav:\n wav.setnchannels(1); wav.setsampwidth(2); wav.setframerate(24000); wav.writeframes(raw)\n record.update({"pcm":str(pcm_file), "wav":str(wav_file), "sample_rate_hz":24000})\n if identity in texts:\n text_file = out / ("text-" + str(number) + ".txt")\n text_file.write_text(texts[identity], encoding="utf-8")\n record["text"] = str(text_file)\n manifest["parts"].append(record)\n (out / "results.json").write_text(json.dumps(manifest, ensure_ascii=False, indent=2), encoding="utf-8")\n print("\\nSaved every received event and result in", out)\nsession events
Keep every complete returned response, file and content part.
Assemble every returned audio delta and text/event result for each actual response ID.
Keepactualsession/response/eventIDs andreceipts.
| Response | Condition | Next step |
|---|---|---|
Response426 · invalid_request_error | ConditionNo WebSocket upgrade | Next stepUse a WebSocket client with Upgrade: websocket. |
Ailoot qwen-audio-3.0-realtime-plus
Use the method paths shown above.