Version: QTrobot V3

TypeScript/Node.js API Reference

The @luxai-qtrobot/robot-sdk SDK gives you a single Robot object to control QTrobot V3 from Node.js or directly from a browser — face, speech, gestures, motors, media, and microphone, all through one consistent, Promise-based API.

npm install @luxai-qtrobot/robot-sdk

For plain HTML pages without a build step, drop the UMD bundle in directly from a CDN — no bundler required:

<script src="https://cdn.jsdelivr.net/npm/@luxai-qtrobot/robot-sdk/dist/qtrobot-sdk.umd.js"></script>

All exports are available under the global QTRobotSDK object:

<!DOCTYPE html>
<html>
<head>
  <script src="https://cdn.jsdelivr.net/npm/@luxai-qtrobot/robot-sdk/dist/qtrobot-sdk.umd.js"></script>
</head>
<body>
<script>
  const { Robot } = QTRobotSDK

  // Browsers require WebSocket — use ws:// or wss://, not mqtt://
  const robot = await Robot.connectMqtt('ws://10.231.0.1:9001', 'QTRD000320')
  console.log(`Connected to ${robot.robotId}`)

  await robot.tts.sayText({ text: 'Hello from the browser!' })
</script>
</body>
</html>

On this page: Core concepts · Connecting · Plugins · tts · face · gesture · motor · media · speaker · microphone · camera

Core concepts

Promises and cancellation

Every API call is async and returns a Promise — there's no separate blocking/non-blocking pair like some other SDKs. Calls that take real time on the robot (speaking, playing a gesture, playing media) accept an optional signal: AbortSignal, the same pattern used by the browser fetch API, so you cancel by aborting a AbortController:

const controller = new AbortController()

const playPromise = robot.gesture.playFile({ name: 'QT/wave', signal: controller.signal })

// stop it early
controller.abort()

await playPromise // rejects once the cancellation is processed

Failures throw a RobotApiError, so regular try/catch works as expected.

The stream pattern: readers, writers, and callbacks

Continuous data — joint positions, microphone audio, camera frames — uses streams instead of a single request/response call:

Callback — on<Name>(handler, onError?) runs your function for every incoming frame, and returns an unsubscribe function you call when done.
Reader — <name>Reader(options?) returns a TypedStreamReader, usable either as an async iterator (for await (const frame of reader)) or with a direct reader.read(timeoutSec) call.
Writer — open<Name>Writer() returns a TypedStreamWriter you push frames to: writer.write(frame).

// Callback
const unsubscribe = robot.motor.onJointsState((frame) => {
  console.log(frame)
})
// later
unsubscribe()

// Async iterator
for await (const frame of robot.motor.jointsStateReader()) {
  console.log(frame)
}

Both the callback and reader forms deliver the exact same data — pick whichever fits your control flow better.

Connecting

MQTT
WebRTC (MQTT signaling)

import { Robot } from '@luxai-qtrobot/robot-sdk'

const robot = await Robot.connectMqtt('mqtt://10.231.0.1:1883', 'QTRD000320')
await robot.tts.sayText({ text: 'Hello!' })
robot.close()

Parameter	Type	Description
`brokerUrl`	`string`	MQTT broker URL, e.g. `mqtt://10.231.0.1:1883` or `wss://broker:8884/mqtt` (browsers must use `ws://`/`wss://`).
`robotId`	`string`	Robot serial number, e.g. `QTRD000320`.
`options`	`ConnectOptions`	MQTT connection options (TLS, auth, ...) plus `connectTimeoutSec` and `defaultRpcTimeoutSec`.

const robot = await Robot.connectWebrtcMqtt('wss://broker:8884/mqtt', 'QTRD000320')
await robot.tts.sayText({ text: 'Hello!' })
robot.close()

MQTT is used only to bootstrap the WebRTC handshake — once connected, traffic flows peer-to-peer. Each call establishes an independent WebRTC peer connection.

Parameter	Type	Description
`brokerUrl`	`string`	MQTT broker URI used for signaling, e.g. `wss://broker:8884/mqtt`.
`robotId`	`string`	Used as the WebRTC `session_id`, e.g. `QTRD000320`.
`options.mqttOptions`	`MqttOptions`	Options for the signaling broker connection.
`options.webrtcOptions`	`WebRtcOptions`	STUN/TURN servers, codec preferences for the peer connection.
`options.reconnect`	`boolean`	Automatically re-establish the connection if it drops.
`options.connectTimeoutSec`	`number`	End-to-end timeout covering signaling + handshake (default `30`).
`options.preConnect`	`(conn) => void \| Promise<void>`	Called after signaling is ready but before negotiation, to register local media tracks to send to the robot — see camera (plugin).

Close the connection with robot.close(), or use a using declaration (TC39 explicit resource management) for automatic cleanup when it goes out of scope:

using robot = await Robot.connectMqtt('mqtt://10.231.0.1:1883', 'QTRD000320')
await robot.tts.sayText({ text: 'Hello!' })
// robot.close() is called automatically here

Robot properties

Property	Type	Description
`robot.robotId`	`string`	The robot's serial number (e.g. `QTRD000320`).
`robot.robotType`	`string \| null`	Robot model/type string.
`robot.sdkVersion`	`string \| null`	Robot-side SDK version.
`robot.minSdk` / `robot.maxSdk`	`string \| null`	SDK version range supported by the robot.

Plugins

The camera is currently the only plugin available in the TypeScript/Node.js SDK. Once enabled, it appears as robot.camera.

MQTT
WebRTC (MQTT signaling)
Custom transport

await robot.enablePluginMqtt('camera', 'mqtt://broker:1883', 'QTRD000320/realsense')
const intrinsics = await robot.camera!.getColorIntrinsics()

Parameter	Type	Description
`name`	`'camera'`	Plugin identifier.
`brokerUrl`	`string`	MQTT broker URL (can be the same broker as the robot).
`mqttTopicPrefix`	`string`	MQTT topic prefix the plugin is accessible under.
`options`	`ConnectOptions`	Optional MQTT + timeout options.

// Inherits broker + options from the robot's own WebRTC connection automatically
await robot.enablePluginWebrtcMqtt('camera', 'qtrobot-realsense-driver')
const intrinsics = await robot.camera!.getColorIntrinsics()

Requires the robot to be connected via connectWebrtcMqtt. Each plugin gets its own independent WebRTC peer connection, so its media tracks never conflict with the main robot peer. Signaling options (brokerUrl, mqttOptions, webrtcOptions, reconnect, connectTimeoutSec) are all optional — pass them only to override what's inherited from the robot connection.

await robot.enablePlugin('camera', new MyCustomTransport(...))

Disable a plugin with robot.disablePlugin('camera').

tts

robot.tts controls text-to-speech: speaking plain text or SSML, and selecting/configuring engines.

sayText

Speak plain text. Blocks (the returned Promise resolves) until audio playback completes.

sayText(options: TtsSayTextOptions): Promise<unknown>

Parameter	Type	Description
`text`	`string`	Plain text to synthesize.
`engine`	`string?`	Engine id (uses default if omitted).
`lang`	`string?`	Language code, e.g. `en-US`.
`voice`	`string?`	Voice id or name.
`rate`	`number?`	Speech rate multiplier.
`pitch`	`number?`	Pitch adjustment.
`volume`	`number?`	Volume level.
`style`	`string?`	Speaking style (engine-dependent).
`signal`	`AbortSignal?`	Cancel the operation.

await robot.tts.sayText({ text: 'Hello world!' })
await robot.tts.sayText({ text: 'Slower speech', engine: 'acapela', rate: 0.8, pitch: 1.1 })

// Cancel after 2 seconds
const controller = new AbortController()
const p = robot.tts.sayText({ text: 'A very long sentence...', signal: controller.signal })
setTimeout(() => controller.abort(), 2000)
await p.catch(() => {})

saySsml

Speak SSML markup. Same cancellation pattern as sayText.

saySsml(options: TtsSaySsmlOptions): Promise<unknown>

Parameter	Type	Description
`ssml`	`string`	SSML markup string.
`engine`	`string?`	Engine id (uses default if omitted).
`signal`	`AbortSignal?`	Cancel the operation.

await robot.tts.saySsml({ ssml: '<speak>Hello!</speak>' })

Engines, voices, and configuration

Method	Returns	Description
`listEngines()`	`Promise<unknown[]>`	Loaded/available TTS engine ids.
`getDefaultEngine()`	`Promise<string>`	Current default engine id.
`setDefaultEngine({ engine })`	`Promise<unknown>`	Set the default engine id.
`getLanguages({ engine? })`	`Promise<unknown[]>`	Supported language codes for an engine.
`getVoices({ engine? })`	`Promise<unknown[]>`	Supported voices for an engine.
`supportsSsml({ engine? })`	`Promise<boolean>`	Whether the engine supports SSML.
`setConfig({ config, engine? })`	`Promise<unknown>`	Set engine-specific configuration.
`getConfig({ engine? })`	`Promise<Record<string, unknown>>`	Get engine-specific configuration.

const engines = await robot.tts.listEngines()
await robot.tts.setDefaultEngine({ engine: 'acapela' })

const voices = await robot.tts.getVoices({ engine: 'acapela' })
if (await robot.tts.supportsSsml({ engine: 'azure' })) {
  await robot.tts.saySsml({ ssml: '<speak>Hello!</speak>', engine: 'azure' })
}

face

robot.face controls the face display: emotion playback and eye gaze.

showEmotion

Show an emotion animation. Blocks until the animation finishes (or is cancelled via signal).

showEmotion(options: FaceShowEmotionOptions): Promise<unknown>

Parameter	Type	Description
`emotion`	`string`	Emotion name or relative path (with or without `.avi`).
`speed`	`number?`	Playback speed factor (uses config default if omitted).
`signal`	`AbortSignal?`	Cancel the operation.

await robot.face.showEmotion({ emotion: 'QT/kiss' })

const controller = new AbortController()
const p = robot.face.showEmotion({ emotion: 'QT/breathing_exercise', signal: controller.signal })
setTimeout(() => controller.abort(), 3000)

listEmotions / look

listEmotions(): Promise<unknown[]>
look(options: FaceLookOptions): Promise<unknown>

Parameter	Type	Description
`l_eye`	`unknown[]`	`[dx, dy]` pixel offset from centre for the left eye.
`r_eye`	`unknown[]`	`[dx, dy]` pixel offset from centre for the right eye.
`duration`	`number?`	If > 0, eyes reset to centre after this many seconds.

const emotions = await robot.face.listEmotions()
await robot.face.look({ l_eye: [30, 0], r_eye: [30, 0] })

gesture

robot.gesture plays predefined or raw keyframe gestures, tracks playback progress, and records new gestures.

play / playFile

play(options: GesturePlayOptions): Promise<unknown>
playFile(options: GesturePlayFileOptions): Promise<boolean>

Parameter	Type	Description
`keyframes`	`Record<string, unknown>`	Trajectory dict (for `play`).
`gesture`	`string`	Gesture name/path, with or without `.xml` (for `playFile`).
`resample`	`boolean?`	Resample trajectory to a uniform rate (`play` only).
`rate_hz`	`number?`	Resample rate in Hz (`play` only).
`speed_factor`	`number?`	Playback speed multiplier.
`signal`	`AbortSignal?`	Cancel the operation.

await robot.gesture.playFile({ name: 'QT/wave' })

const controller = new AbortController()
const p = robot.gesture.playFile({ gesture: 'QT/bye', signal: controller.signal })
controller.abort() // cancel on demand

listFiles, record, stopRecord, storeRecord

listFiles(): Promise<unknown[]>
record(options: GestureRecordOptions): Promise<Record<string, unknown>>
stopRecord(): Promise<boolean>
storeRecord(options: GestureStoreRecordOptions): Promise<unknown>

Parameter	Type	Description
`motors`	`unknown[]`	Motor names to record.
`release_motors`	`boolean?`	Disable motor torque during recording.
`delay_start_ms`	`number?`	Delay before recording starts (ms).
`timeout_ms`	`number?`	Maximum recording duration (ms).
`refine_keyframe`	`boolean?`	Remove redundant keyframes after recording.
`keyframe_pos_eps`	`number?`	Position epsilon for keyframe refinement (degrees).
`keyframe_max_gap_us`	`number?`	Maximum gap between keyframes (µs).
`gesture` (for `storeRecord`)	`string`	Name/path to save the recorded gesture as XML.

record() resolves when recording stops (timeout, or signal abort which calls stopRecord for you). For interactive use, abort via signal, or call stopRecord() directly, then storeRecord({ gesture }) to save the result.

const controller = new AbortController()
const recordPromise = robot.gesture.record({
  motors: ['RightShoulderPitch', 'RightElbowRoll'],
  release_motors: true,
  timeout_ms: 20000,
  signal: controller.signal,
})
// later, e.g. on a button click
controller.abort()
const keyframes = await recordPromise
await robot.gesture.storeRecord({ gesture: 'my_wave' })

Stream: progress

onProgress(handler, onError?): () => void
progressReader(options?: { queueSize?: number }): TypedStreamReader<Record<string, unknown>>

const unsubscribe = robot.gesture.onProgress((frame) => {
  console.log(frame)
})

// or
for await (const frame of robot.gesture.progressReader()) {
  console.log(frame)
}

motor

robot.motor lists motors, controls torque/homing/velocity, and streams joint state, errors, and commands.

list, torque, and homing

list(): Promise<Record<string, unknown>>
on(options: { motor: string }): Promise<unknown>
off(options: { motor: string }): Promise<unknown>
onAll(): Promise<unknown>
offAll(): Promise<unknown>
home(options: { motor: string }): Promise<unknown>
homeAll(): Promise<unknown>
setVelocity(options: { motor: string; velocity: number }): Promise<unknown>

const motors = await robot.motor.list()
await robot.motor.onAll()
await robot.motor.home({ motor: 'HeadYaw' })
await robot.motor.setVelocity({ motor: 'HeadYaw', velocity: 50 })
await robot.motor.offAll()

Stream: joint state, errors, and commands

onJointsState(handler, onError?): () => void
jointsStateReader(options?): TypedStreamReader<JointStateFrame>
onJointsError(handler, onError?): () => void
jointsErrorReader(options?): TypedStreamReader<Record<string, unknown>>
openJointsCommandWriter(): TypedStreamWriter<JointCommandFrame>

jointsState — outbound JointStateFrame (position, velocity, effort, voltage, temperature) per motor.
jointsError — outbound error flags (overload/voltage/temperature/sensor) per motor, when present.
jointsCommand — inbound writer for sending joint position/velocity commands.

const unsubscribe = robot.motor.onJointsState((frame) => console.log(frame))

const writer = robot.motor.openJointsCommandWriter()
writer.write(new JointCommandFrame(...))

media

robot.media plays audio and video on two independent lanes — foreground (fg) and background (bg) — from files or as live streamed frames. fg and bg expose an identical API; examples below use fg.

File playback

playFgAudioFile(options: { uri: string; signal?: AbortSignal }): Promise<boolean>
pauseFgAudioFile(): Promise<unknown>
resumeFgAudioFile(): Promise<unknown>
setFgAudioVolume(options: { value: number }): Promise<unknown>
getFgAudioVolume(): Promise<number>

await robot.media.playFgAudioFile({ uri: '/data/audio/hello.wav' })

const controller = new AbortController()
const p = robot.media.playFgAudioFile({ uri: '/data/audio/hello.wav', signal: controller.signal })
setTimeout(() => controller.abort(), 3000)

await robot.media.setFgAudioVolume({ value: 0.8 })

playBgAudioFile / pauseBgAudioFile / resumeBgAudioFile / setBgAudioVolume / getBgAudioVolume mirror the fg methods on the background lane.

Video file playback

playFgVideoFile(options: { uri: string; speed?: number; with_audio?: boolean; signal?: AbortSignal }): Promise<boolean>
pauseFgVideoFile(): Promise<unknown>
resumeFgVideoFile(): Promise<unknown>
setFgVideoAlpha(options: { value: number }): Promise<unknown>

await robot.media.playFgVideoFile({ uri: '/data/video/intro.mp4' })
await robot.media.setFgVideoAlpha({ value: 0.8 })

playBgVideoFile / pauseBgVideoFile / resumeBgVideoFile mirror the fg methods (no alpha control on bg).

Streamed audio/video

For audio/video frames you generate yourself, rather than a file on disk:

openFgAudioStreamWriter(): TypedStreamWriter<AudioFrameRaw>
openBgAudioStreamWriter(): TypedStreamWriter<AudioFrameRaw>
openFgVideoStreamWriter(): TypedStreamWriter<ImageFrameRaw>
openBgVideoStreamWriter(): TypedStreamWriter<ImageFrameRaw>

cancelFgAudioStream() / pauseFgAudioStream() / resumeFgAudioStream()
cancelBgAudioStream() / pauseBgAudioStream() / resumeBgAudioStream()
cancelFgVideoStream() / pauseFgVideoStream() / resumeFgVideoStream()
cancelBgVideoStream() / pauseBgVideoStream() / resumeBgVideoStream()

const writer = robot.media.openFgAudioStreamWriter()
await writer.write(new AudioFrameRaw(...))
...
await robot.media.cancelFgAudioStream()

speaker

robot.speaker controls the robot's overall hardware volume — separate from any individual media lane's volume (see media).

setVolume(options: { value: number }): Promise<boolean>
getVolume(): Promise<number>
mute(): Promise<boolean>
unmute(): Promise<boolean>

await robot.speaker.setVolume({ value: 0.8 })
const vol = await robot.speaker.getVolume()
await robot.speaker.mute()
await robot.speaker.unmute()

microphone

robot.microphone reads raw audio from the robot's internal 5-channel microphone array (and an optional external mic), and tunes the array's DSP.

DSP tuning

getIntTuning(): Promise<Record<string, unknown>>
setIntTuning(options: { name: string; value: number }): Promise<boolean>

const params = await robot.microphone.getIntTuning()
await robot.microphone.setIntTuning({ name: 'AGCONOFF', value: 1.0 })

Audio and voice-activity streams

onIntAudioCh0(handler, onError?): () => void
intAudioCh0Reader(options?): TypedStreamReader<AudioFrameRaw>
// ... ch1, ch2, ch3, ch4 follow the same pattern

onExtAudioCh0(handler, onError?): () => void
extAudioCh0Reader(options?): TypedStreamReader<AudioFrameRaw>

onIntEvent(handler, onError?): () => void
intEventReader(options?): TypedStreamReader<Record<string, unknown>>

Channel 0 is the processed/ASR-ready channel; channels 1–4 are raw per-microphone signals. The external mic stream is only published if enabled in the robot's configuration. The voice-activity/direction-of-arrival event stream carries { activity: boolean, direction: number } (degrees, 0–359).

for await (const frame of robot.microphone.intAudioCh0Reader()) {
  process(frame)
}

robot.microphone.onIntEvent((frame) => {
  if (frame.activity) console.log('Voice detected — DOA:', frame.direction)
})

camera (plugin)

robot.camera is the SDK's one plugin namespace so far, exposing the RealSense 3D camera. Enable it first — see Plugins.

Intrinsics

getColorIntrinsics(): Promise<Record<string, unknown>>
getDepthIntrinsics(): Promise<Record<string, unknown>>
getDepthScale(): Promise<Record<string, unknown>>

const intrinsics = await robot.camera!.getColorIntrinsics()
const { scale } = await robot.camera!.getDepthScale()

Image streams

onColor(handler, onError?): () => void
colorReader(options?): TypedStreamReader<ImageFrameRaw>
onDepth(handler, onError?): () => void
depthReader(options?): TypedStreamReader<ImageFrameRaw>
onDepthAligned(handler, onError?): () => void
depthAlignedReader(options?): TypedStreamReader<ImageFrameRaw>
onDepthColor(handler, onError?): () => void
depthColorReader(options?): TypedStreamReader<ImageFrameRaw>

Stream	Description
`color`	Color image (BGR, width × height × 3).
`depth`	Raw depth image (16-bit).
`depthAligned`	Depth aligned to the color frame.
`depthColor`	False-colour depth, colourised for visualisation.

for await (const frame of robot.camera!.colorReader()) {
  render(frame)
}

IMU streams

onGyro(handler, onError?): () => void
gyroReader(options?): TypedStreamReader<unknown[]>
onAcceleration(handler, onError?): () => void
accelerationReader(options?): TypedStreamReader<unknown[]>

Both deliver [x, y, z] — angular velocity in rad/s for gyro, linear acceleration in m/s² for acceleration.

Live media tracks (browser)

robot.extra (core connection) and robot.camera.extra (camera plugin) expose native WebRTC MediaStreamTracks — only available over a WebRTC connection with use_media_channels: true (the gateway default):

receiveVideoTrack(topic = 'video'): Promise<MediaStreamTrack>
receiveAudioTrack(topic = 'audio'): Promise<MediaStreamTrack>
sendVideoTrack(track: MediaStreamTrack, topic = 'video'): void
sendAudioTrack(track: MediaStreamTrack, topic = 'audio'): void

receive*Track resolves once the remote peer's track arrives — attach it directly to an HTML element:

const audioTrack = await robot.extra.receiveAudioTrack()
audioEl.srcObject = new MediaStream([audioTrack])
audioEl.play()

const videoTrack = await robot.camera!.extra.receiveVideoTrack()
videoEl.srcObject = new MediaStream([videoTrack])
videoEl.play()

send*Track registers a local track (e.g. from getUserMedia) to send to the robot — it must be called from preConnect (see Connecting), before the connection is established:

const robot = await Robot.connectWebrtcMqtt(brokerUrl, robotId, {
  preConnect: async (conn) => {
    const stream = await navigator.mediaDevices.getUserMedia({ video: true })
    conn.sendVideoTrack(stream.getVideoTracks()[0])
  },
})

See the TypeScript/Node.js Tutorials for runnable examples, including a full set of browser HTML demos under examples/web/.

Core concepts​

Promises and cancellation​

The stream pattern: readers, writers, and callbacks​

Connecting​

Robot properties​

Plugins​

tts​

sayText​

saySsml​

Engines, voices, and configuration​

face​

showEmotion​

listEmotions / look​

gesture​

play / playFile​

listFiles, record, stopRecord, storeRecord​

Stream: progress​

motor​

list, torque, and homing​

Stream: joint state, errors, and commands​

media​

File playback​

Video file playback​

Streamed audio/video​

speaker​

microphone​

DSP tuning​

Audio and voice-activity streams​

camera (plugin)​

Intrinsics​

Image streams​

IMU streams​

Live media tracks (browser)​