Skip to main content
Version: QTrobot V3

Audio

robot.media plays audio on two independent lanes — foreground (FG) and background (BG) — from files, online URLs, or as a raw streamed PCM signal. These examples assume you already have a connected robot — see Connection if you haven't set one up yet.

How the FG/BG audio system works​

QTrobot mixes two independent audio lanes into the final output — each with its own volume control, and both can play at the same time. That's what makes it possible to play, say, background music under foreground speech, each at its own level.

Each lane also accepts audio from two inputs: a file (or URL) and a raw stream. Within a lane, file playback always takes priority — if a stream is playing on, say, the FG lane and an RPC call asks that same lane to play a file, the file takes over immediately; stream frames arriving during that time are simply dropped. Once the file finishes, the lane automatically goes back to playing whatever the stream sends.

One important detail: TTS speech is always played on the FG stream input. So if you want background music or sound effects to play while the robot is talking, put them on the BG lane (file or stream) — the FG lane is reserved for speech.

After the two lanes are mixed, the result still passes through the robot's overall robot.speaker master volume — so you have three independent volume knobs in total: FG, BG, and master.

Creative possibilities​

Two lanes, each with its own volume and its own file/stream input, open up a lot of interactive storytelling:

  • TTS with a sound effect — have the robot speak (FG) while a short sound effect or musical sting plays on BG.
  • Storytelling with ambient sound — narrate a story with TTS (FG) while background music, rain, river, or forest ambience plays underneath on BG.
  • Proximity effects — narrate a scene where a character hears an animal far away: play the animal sound on BG at low volume, then gradually raise the BG volume as the character "gets closer," using the same set_bg_audio_volume() calls you'll see below.
  • Mood shifts — fade BG volume up or down between scenes to make a story feel calmer or more tense, without interrupting the FG narration at all.

The same two volume knobs can support many more scenarios than these — worth experimenting with once you're comfortable with the basics below.

Setup​

mkdir ~/example
cd ~/example
python -m venv .venv
# or: uv venv .venv

source .venv/bin/activate

pip install luxai-robot
# or: uv pip install luxai-robot

Connect​

from luxai.robot.core import Robot

robot = Robot.connect_zmq(robot_id="QTRD000123")
print(f"connected to {robot.robot_id} ({robot.robot_type})")

See Connection for MQTT, WebRTC, and other connection options.

Volume​

# Set FG volume to 100% and confirm
robot.media.set_fg_audio_volume(1.0)
vol = robot.media.get_fg_audio_volume()
Logger.info(f"FG audio volume after set: {vol:.2f}")

# Set BG volume to 100% and confirm
robot.media.set_bg_audio_volume(1.0)
vol = robot.media.get_bg_audio_volume()
Logger.info(f"BG audio volume after set: {vol:.2f}")

Play a file​

A file path must exist on the robot itself (not on your laptop) for these examples to work.

import time

audio_file_on_robot = "/home/qtrobot/robot/data/audios/QT/5LittleBunnies.wav"

# Play on the FG lane and wait for it to finish
ret = robot.media.play_fg_audio_file(audio_file_on_robot)
Logger.info(f"Done. Result: {ret}")

# Play non-blocking, then cancel it after 5 seconds
h = robot.media.play_fg_audio_file_async(audio_file_on_robot)
time.sleep(5)
h.cancel()

Pause and resume​

import time

audio_file_on_robot = "/home/qtrobot/robot/data/audios/QT/5LittleBunnies.wav"

play_handler = robot.media.play_fg_audio_file_async(audio_file_on_robot)
time.sleep(10)

robot.media.pause_fg_audio_file()
Logger.info("Paused. Waiting 5 seconds...")
time.sleep(5)

robot.media.resume_fg_audio_file()
play_handler.wait()  # wait for playback to finish

Play FG and BG simultaneously​

import time
from luxai.robot.core import wait_all_actions

bg_audio_file_on_robot = "/home/qtrobot/robot/data/audios/QT/5LittleBunnies.wav"
fg_audio_file_on_robot = "/home/qtrobot/robot/data/audios/QT/John_Wesley_Tequila.mp3"

# Lower BG volume to hear FG clearly
robot.media.set_bg_audio_volume(0.7)
robot.media.set_fg_audio_volume(1.0)

h_bg = robot.media.play_bg_audio_file_async(bg_audio_file_on_robot)

# Start FG audio after a short delay to create overlap
time.sleep(5)
h_fg = robot.media.play_fg_audio_file_async(fg_audio_file_on_robot)

# Wait for both to finish
wait_all_actions([h_bg, h_fg])

# Reset BG volume
robot.media.set_bg_audio_volume(1.0)

Play an online file or radio stream​

The URL must be a direct link to an audio file or stream, not a webpage.

import time

online_audio_url = "https://www.soundhelix.com/examples/mp3/SoundHelix-Song-1.mp3"
h = robot.media.play_fg_audio_file_async(online_audio_url)
time.sleep(10)
h.cancel()
online_radio_url = "http://radio3.radio-calico.com:8080/calico"
h = robot.media.play_fg_audio_file_async(online_radio_url)
Logger.info("Press <enter> to stop the radio...")
input()
h.cancel()

Stream raw audio​

import math
import struct
import time
from luxai.magpie.utils.common import get_uinque_id
from luxai.magpie.frames import AudioFrameRaw

def _make_sine_chunk(freq_hz, sample_rate, chunk_frames, phase):
    amplitude = 32767  # max for signed 16-bit
    samples = [
        int(amplitude * math.sin(2 * math.pi * freq_hz * (phase + i) / sample_rate))
        for i in range(chunk_frames)
    ]
    return struct.pack(f"<{chunk_frames}h", *samples), phase + chunk_frames

stream_id = get_uinque_id()
sample_rate = 16000
freq_hz = 440.0  # A4
duration_s = 5.0
chunk_frames = 1024
total_frames = int(sample_rate * duration_s)

writer = robot.media.stream.open_fg_audio_stream_writer()

phase = 0
frame_id = 1
while phase < total_frames:
    frames_this_chunk = min(chunk_frames, total_frames - phase)
    raw, phase = _make_sine_chunk(freq_hz, sample_rate, frames_this_chunk, phase)
    frame = AudioFrameRaw(channels=1, sample_rate=sample_rate, bit_depth=16, data=raw)
    frame.gid = stream_id  # group ID for this stream
    frame.id = frame_id    # frame ID (optional, for tracking)
    writer.write(frame)
    frame_id += 1
    time.sleep(frames_this_chunk / sample_rate / 1.5)  # pace the stream in real-time

Next steps​

Continue with the Video tutorial, or see the full robot.media namespace in the Python API Reference.