Skip to main content
Version: QTrobot V3

Audio

robot.media plays audio on two independent lanes — foreground (FG) and background (BG) — each with its own volume, and both can play at the same time. File playback always takes priority over a stream on the same lane. See the Python Audio tutorial for the full breakdown of how the FG/BG system works if you want the details — here we'll stay practical. These examples assume you already have a connected robot — see Connection if you haven't set one up yet.

Setup​

<script src="https://cdn.jsdelivr.net/npm/@luxai-qtrobot/robot-sdk/dist/qtrobot-sdk.umd.js"></script>
npm install @luxai-qtrobot/robot-sdk

Volume​

await robot.media.setFgAudioVolume({ value: 1.0 })
console.log(`FG audio volume after set: ${(await robot.media.getFgAudioVolume()).toFixed(2)}`)

await robot.media.setBgAudioVolume({ value: 1.0 })
console.log(`BG audio volume after set: ${(await robot.media.getBgAudioVolume()).toFixed(2)}`)

Play a file​

A file path must exist on the robot itself (not on your laptop) for these examples to work.

const FG_AUDIO = '/home/qtrobot/robot/data/audios/QT/5LittleBunnies.wav'

// Blocking — wait for playback to finish
await robot.media.playFgAudioFile({ uri: FG_AUDIO })

// Non-blocking — cancel after 5 seconds
const controller = new AbortController()
const play = robot.media.playFgAudioFile({ uri: FG_AUDIO, signal: controller.signal })
setTimeout(() => controller.abort(), 5000)
try {
  await play
} catch {
  console.log('FG audio file playback cancelled.')
}

Pause and resume​

const FG_AUDIO = '/home/qtrobot/robot/data/audios/QT/5LittleBunnies.wav'

const controller = new AbortController()
const play = robot.media.playFgAudioFile({ uri: FG_AUDIO, signal: controller.signal })

await new Promise(r => setTimeout(r, 5000))
await robot.media.pauseFgAudioFile()

await new Promise(r => setTimeout(r, 3000))
await robot.media.resumeFgAudioFile()

await play

Play FG and BG simultaneously​

const BG_AUDIO = '/home/qtrobot/robot/data/audios/QT/5LittleBunnies.wav'
const FG_AUDIO = '/home/qtrobot/robot/data/audios/QT/John_Wesley_Tequila.mp3'

await robot.media.setBgAudioVolume({ value: 0.7 })
await robot.media.setFgAudioVolume({ value: 1.0 })

const bgPlay = robot.media.playBgAudioFile({ uri: BG_AUDIO })

// Start FG audio after a short delay to create overlap
await new Promise(r => setTimeout(r, 5000))
const fgPlay = robot.media.playFgAudioFile({ uri: FG_AUDIO })

await Promise.all([bgPlay, fgPlay])
await robot.media.setBgAudioVolume({ value: 1.0 })

Play an online audio file​

The URL must be a direct link to an audio file, not a webpage.

const url = 'https://www.soundhelix.com/examples/mp3/SoundHelix-Song-1.mp3'
const controller = new AbortController()
const play = robot.media.playFgAudioFile({ uri: url, signal: controller.signal })
setTimeout(() => controller.abort(), 10000)
try {
  await play
} catch {
  console.log('Online audio playback cancelled.')
}

Send your microphone to the robot (WebRTC)​

The robot.media calls above are all RPC-based, which works the same over MQTT or WebRTC. But what if you want to stream live audio — say, your own microphone — to the robot? By default, the MQTT gateway doesn't forward audio or video streams (bandwidth on a shared broker adds up fast, so stream bridging is opt-in, configured per-topic). WebRTC is the right tool for live audio, since peer-to-peer media tracks were built for exactly this.

This is a simplified version of what live-video-call.html does — registering your browser's microphone as a track that gets sent straight to the robot's FG audio lane:

// 1. Ask the browser for microphone access
const localStream = await navigator.mediaDevices.getUserMedia({
  audio: { echoCancellation: true, noiseSuppression: true, autoGainControl: true },
})

// 2. Connect, registering the local track before the WebRTC handshake completes
const robot = await Robot.connectWebrtcMqtt(broker, robotId, {
  preConnect: (conn) => {
    conn.sendAudioTrack(localStream.getAudioTracks()[0], '/media/audio/fg/stream:i')
  },
})

From this point on, anything your microphone picks up plays through the robot's FG audio lane — exactly like any other FG source, so the same priority/volume rules from Volume above apply. See the full examples/web/webrtc folder on GitHub for the complete two-way call demo, including the robot's own microphone played back to you and camera video alongside this.

Next steps​

Continue with the Video tutorial, or see the full robot.media namespace in the TypeScript/Node.js API Reference.