Skip to main content
Version: QTrobot V3

Video

robot.media plays video on the same two lanes used for audio — foreground (FG) and background (BG) — except instead of volume, FG has an alpha (transparency) control and is blended on top of the opaque BG layer. See the Python Video tutorial for the full breakdown (including why QTrobot's own emotions/idle face use BG, leaving FG free for you) if you want the details — here we'll stay practical. These examples assume you already have a connected robot — see Connection if you haven't set one up yet.

Setup​

<script src="https://cdn.jsdelivr.net/npm/@luxai-qtrobot/robot-sdk/dist/qtrobot-sdk.umd.js"></script>
npm install @luxai-qtrobot/robot-sdk

Foreground alpha (transparency)​

// Fully opaque
await robot.media.setFgVideoAlpha({ value: 1.0 })

// Half-transparent
await robot.media.setFgVideoAlpha({ value: 0.5 })
await new Promise(r => setTimeout(r, 2000))

// Restore to fully opaque
await robot.media.setFgVideoAlpha({ value: 1.0 })

Play a background video file​

A file path must exist on the robot itself (not on your laptop) for these examples to work.

const BG_VIDEO = '/home/qtrobot/robot/data/emotions/QT/kiss.avi'

// Blocking — wait for playback to finish
await robot.media.playBgVideoFile({ uri: BG_VIDEO })

// Non-blocking — cancel after 2 seconds
const controller = new AbortController()
const play = robot.media.playBgVideoFile({ uri: BG_VIDEO, signal: controller.signal })
setTimeout(() => controller.abort(), 2000)
try {
  await play
} catch {
  console.log('BG video file playback cancelled.')
}

Pause and resume​

const BG_VIDEO = '/home/qtrobot/robot/data/emotions/QT/kiss.avi'

const controller = new AbortController()
const play = robot.media.playBgVideoFile({ uri: BG_VIDEO, signal: controller.signal })

await new Promise(r => setTimeout(r, 2000))
await robot.media.pauseBgVideoFile()

await new Promise(r => setTimeout(r, 3000))
await robot.media.resumeBgVideoFile()

await play

Play a video file with its audio track​

const FG_VIDEO = '/home/qtrobot/robot/data/emotions/QT/happy.avi'
await robot.media.playFgVideoFile({ uri: FG_VIDEO, with_audio: true })

Send your camera to the robot (WebRTC)​

Like audio, live video streaming isn't bridged over MQTT by default — WebRTC is the right tool for it. This is a simplified version of what live-video-call.html does — registering your browser's camera as a track sent straight to the robot's FG video lane, where it's alpha-blended on top of whatever the robot's face is currently showing on BG:

// 1. Ask the browser for camera access
const localStream = await navigator.mediaDevices.getUserMedia({
  video: { width: { ideal: 640 }, height: { ideal: 480 }, frameRate: { ideal: 15 } },
})

// 2. Connect, registering the local track before the WebRTC handshake completes
const robot = await Robot.connectWebrtcMqtt(broker, robotId, {
  preConnect: (conn) => {
    conn.sendVideoTrack(localStream.getVideoTracks()[0], '/media/video/fg/stream:i')
  },
})

From this point on, your camera feed plays on the robot's FG video lane — same alpha rules as Foreground alpha above apply. See the full examples/web/webrtc folder on GitHub for the complete two-way video call demo, including receiving the robot's own camera feed back (see Camera).

Next steps​

Continue with the Motor tutorial, or see the full robot.media namespace in the TypeScript/Node.js API Reference.