TypeScript/Node.js API Reference

The @luxai-qtrobot/robot-sdk SDK gives you a single Robot object to control QTrobot V3 from Node.js or directly from a browser — face, speech, gestures, motors, media, and microphone, all through one consistent, Promise-based API.

npm install @luxai-qtrobot/robot-sdk

For plain HTML pages without a build step, drop the UMD bundle in directly from a CDN — no bundler required:

<script src="https://cdn.jsdelivr.net/npm/@luxai-qtrobot/robot-sdk/dist/qtrobot-sdk.umd.js"></script>

All exports are available under the global QTRobotSDK object:

<!DOCTYPE html>
<html>
<head>
  <script src="https://cdn.jsdelivr.net/npm/@luxai-qtrobot/robot-sdk/dist/qtrobot-sdk.umd.js"></script>
</head>
<body>
<script>
  const { Robot } = QTRobotSDK

  // Browsers require WebSocket — use ws:// or wss://, not mqtt://
  const robot = await Robot.connectMqtt('ws://10.231.0.1:9001', 'QTRD000320')
  console.log(`Connected to ${robot.robotId}`)

  await robot.tts.sayText({ text: 'Hello from the browser!' })
</script>
</body>
</html>

On this page: Core concepts · Connecting · Plugins · tts · face · gesture · motor · media · speaker · microphone · camera

Core concepts

Promises and cancellation

Every API call is async and returns a Promise — there's no separate blocking/non-blocking pair like some other SDKs. Calls that take real time on the robot (speaking, playing a gesture, playing media) accept an optional signal: AbortSignal, the same pattern used by the browser fetch API, so you cancel by aborting a AbortController:

const controller = new AbortController()

const playPromise = robot.gesture.playFile({ name: 'QT/wave', signal: controller.signal })

// stop it early
controller.abort()

await playPromise // rejects once the cancellation is processed

Failures throw a RobotApiError, so regular try/catch works as expected.

The stream pattern: readers, writers, and callbacks

Continuous data — joint positions, microphone audio, camera frames — uses streams instead of a single request/response call:

• Callback — on<Name>(handler, onError?) runs your function for every incoming frame, and returns an unsubscribe function you call when done.
• Reader — <name>Reader(options?) returns a TypedStreamReader, usable either as an async iterator (for await (const frame of reader)) or with a direct reader.read(timeoutSec) call.
• Writer — open<Name>Writer() returns a TypedStreamWriter you push frames to: writer.write(frame).

// Callback
const unsubscribe = robot.motor.onJointsState((frame) => {
  console.log(frame)
})
// later
unsubscribe()

// Async iterator
for await (const frame of robot.motor.jointsStateReader()) {
  console.log(frame)
}

Both the callback and reader forms deliver the exact same data — pick whichever fits your control flow better.

Connecting

MQTT

import { Robot } from '@luxai-qtrobot/robot-sdk'

const robot = await Robot.connectMqtt('mqtt://10.231.0.1:1883', 'QTRD000320')
await robot.tts.sayText({ text: 'Hello!' })
robot.close()

Parameter	Type	Description
brokerUrl	string	MQTT broker URL, e.g. mqtt://10.231.0.1:1883 or wss://broker:8884/mqtt (browsers must use ws:///wss://).
robotId	string	Robot serial number, e.g. QTRD000320.
options	ConnectOptions	MQTT connection options (TLS, auth, ...) plus connectTimeoutSec and defaultRpcTimeoutSec.

WebRTC (MQTT signaling)

const robot = await Robot.connectWebrtcMqtt('wss://broker:8884/mqtt', 'QTRD000320')
await robot.tts.sayText({ text: 'Hello!' })
robot.close()

MQTT is used only to bootstrap the WebRTC handshake — once connected, traffic flows peer-to-peer. Each call establishes an independent WebRTC peer connection.

Parameter	Type	Description
brokerUrl	string	MQTT broker URI used for signaling, e.g. wss://broker:8884/mqtt.
robotId	string	Used as the WebRTC session_id, e.g. QTRD000320.
options.mqttOptions	MqttOptions	Options for the signaling broker connection.
options.webrtcOptions	WebRtcOptions	STUN/TURN servers, codec preferences for the peer connection.
options.reconnect	boolean	Automatically re-establish the connection if it drops.
options.connectTimeoutSec	number	End-to-end timeout covering signaling + handshake (default 30).
options.preConnect	(conn) => void | Promise<void>	Called after signaling is ready but before negotiation, to register local media tracks to send to the robot — see camera (plugin).

Close the connection with robot.close(), or use a using declaration (TC39 explicit resource management) for automatic cleanup when it goes out of scope:

using robot = await Robot.connectMqtt('mqtt://10.231.0.1:1883', 'QTRD000320')
await robot.tts.sayText({ text: 'Hello!' })
// robot.close() is called automatically here

Robot properties

Property	Type	Description
robot.robotId	string	The robot's serial number (e.g. QTRD000320).
robot.robotType	string | null	Robot model/type string.
robot.sdkVersion	string | null	Robot-side SDK version.
robot.minSdk / robot.maxSdk	string | null	SDK version range supported by the robot.

Plugins

The camera is currently the only plugin available in the TypeScript/Node.js SDK. Once enabled, it appears as robot.camera.

MQTT

await robot.enablePluginMqtt('camera', 'mqtt://broker:1883', 'QTRD000320/realsense')
const intrinsics = await robot.camera!.getColorIntrinsics()

Parameter	Type	Description
name	'camera'	Plugin identifier.
brokerUrl	string	MQTT broker URL (can be the same broker as the robot).
mqttTopicPrefix	string	MQTT topic prefix the plugin is accessible under.
options	ConnectOptions	Optional MQTT + timeout options.

WebRTC (MQTT signaling)

// Inherits broker + options from the robot's own WebRTC connection automatically
await robot.enablePluginWebrtcMqtt('camera', 'qtrobot-realsense-driver')
const intrinsics = await robot.camera!.getColorIntrinsics()

Requires the robot to be connected via connectWebrtcMqtt. Each plugin gets its own independent WebRTC peer connection, so its media tracks never conflict with the main robot peer. Signaling options (brokerUrl, mqttOptions, webrtcOptions, reconnect, connectTimeoutSec) are all optional — pass them only to override what's inherited from the robot connection.

Custom transport

await robot.enablePlugin('camera', new MyCustomTransport(...))

Disable a plugin with robot.disablePlugin('camera').

---

tts

robot.tts controls text-to-speech: speaking plain text or SSML, and selecting/configuring engines.

sayText

Speak plain text. Blocks (the returned Promise resolves) until audio playback completes.

sayText(options: TtsSayTextOptions): Promise<unknown>

Parameter	Type	Description
text	string	Plain text to synthesize.
engine	string?	Engine id (uses default if omitted).
lang	string?	Language code, e.g. en-US.
voice	string?	Voice id or name.
rate	number?	Speech rate multiplier.
pitch	number?	Pitch adjustment.
volume	number?	Volume level.
style	string?	Speaking style (engine-dependent).
signal	AbortSignal?	Cancel the operation.

await robot.tts.sayText({ text: 'Hello world!' })
await robot.tts.sayText({ text: 'Slower speech', engine: 'acapela', rate: 0.8, pitch: 1.1 })

// Cancel after 2 seconds
const controller = new AbortController()
const p = robot.tts.sayText({ text: 'A very long sentence...', signal: controller.signal })
setTimeout(() => controller.abort(), 2000)
await p.catch(() => {})

saySsml

Speak SSML markup. Same cancellation pattern as sayText.

saySsml(options: TtsSaySsmlOptions): Promise<unknown>

Parameter	Type	Description
ssml	string	SSML markup string.
engine	string?	Engine id (uses default if omitted).
signal	AbortSignal?	Cancel the operation.

await robot.tts.saySsml({ ssml: '<speak>Hello!</speak>' })

Engines, voices, and configuration

Method	Returns	Description
listEngines()	Promise<unknown[]>	Loaded/available TTS engine ids.
getDefaultEngine()	Promise<string>	Current default engine id.
setDefaultEngine({ engine })	Promise<unknown>	Set the default engine id.
getLanguages({ engine? })	Promise<unknown[]>	Supported language codes for an engine.
getVoices({ engine? })	Promise<unknown[]>	Supported voices for an engine.
supportsSsml({ engine? })	Promise<boolean>	Whether the engine supports SSML.
setConfig({ config, engine? })	Promise<unknown>	Set engine-specific configuration.
getConfig({ engine? })	Promise<Record<string, unknown>>	Get engine-specific configuration.

const engines = await robot.tts.listEngines()
await robot.tts.setDefaultEngine({ engine: 'acapela' })

const voices = await robot.tts.getVoices({ engine: 'acapela' })
if (await robot.tts.supportsSsml({ engine: 'azure' })) {
  await robot.tts.saySsml({ ssml: '<speak>Hello!</speak>', engine: 'azure' })
}

---

face

robot.face controls the face display: emotion playback and eye gaze.

showEmotion

Show an emotion animation. Blocks until the animation finishes (or is cancelled via signal).

showEmotion(options: FaceShowEmotionOptions): Promise<unknown>

Parameter	Type	Description
emotion	string	Emotion name or relative path (with or without .avi).
speed	number?	Playback speed factor (uses config default if omitted).
signal	AbortSignal?	Cancel the operation.

await robot.face.showEmotion({ emotion: 'QT/kiss' })

const controller = new AbortController()
const p = robot.face.showEmotion({ emotion: 'QT/breathing_exercise', signal: controller.signal })
setTimeout(() => controller.abort(), 3000)

listEmotions / look

listEmotions(): Promise<unknown[]>
look(options: FaceLookOptions): Promise<unknown>

Parameter	Type	Description
l_eye	unknown[]	[dx, dy] pixel offset from centre for the left eye.
r_eye	unknown[]	[dx, dy] pixel offset from centre for the right eye.
duration	number?	If > 0, eyes reset to centre after this many seconds.

const emotions = await robot.face.listEmotions()
await robot.face.look({ l_eye: [30, 0], r_eye: [30, 0] })

---

gesture

robot.gesture plays predefined or raw keyframe gestures, tracks playback progress, and records new gestures.

play / playFile

play(options: GesturePlayOptions): Promise<unknown>
playFile(options: GesturePlayFileOptions): Promise<boolean>

Parameter	Type	Description
keyframes	Record<string, unknown>	Trajectory dict (for play).
gesture	string	Gesture name/path, with or without .xml (for playFile).
resample	boolean?	Resample trajectory to a uniform rate (play only).
rate_hz	number?	Resample rate in Hz (play only).
speed_factor	number?	Playback speed multiplier.
signal	AbortSignal?	Cancel the operation.

await robot.gesture.playFile({ name: 'QT/wave' })

const controller = new AbortController()
const p = robot.gesture.playFile({ gesture: 'QT/bye', signal: controller.signal })
controller.abort() // cancel on demand

listFiles, record, stopRecord, storeRecord

listFiles(): Promise<unknown[]>
record(options: GestureRecordOptions): Promise<Record<string, unknown>>
stopRecord(): Promise<boolean>
storeRecord(options: GestureStoreRecordOptions): Promise<unknown>

Parameter	Type	Description
motors	unknown[]	Motor names to record.
release_motors	boolean?	Disable motor torque during recording.
delay_start_ms	number?	Delay before recording starts (ms).
timeout_ms	number?	Maximum recording duration (ms).
refine_keyframe	boolean?	Remove redundant keyframes after recording.
keyframe_pos_eps	number?	Position epsilon for keyframe refinement (degrees).
keyframe_max_gap_us	number?	Maximum gap between keyframes (µs).
gesture (for storeRecord)	string	Name/path to save the recorded gesture as XML.

record() resolves when recording stops (timeout, or signal abort which calls stopRecord for you). For interactive use, abort via signal, or call stopRecord() directly, then storeRecord({ gesture }) to save the result.

const controller = new AbortController()
const recordPromise = robot.gesture.record({
  motors: ['RightShoulderPitch', 'RightElbowRoll'],
  release_motors: true,
  timeout_ms: 20000,
  signal: controller.signal,
})
// later, e.g. on a button click
controller.abort()
const keyframes = await recordPromise
await robot.gesture.storeRecord({ gesture: 'my_wave' })

Stream: progress

onProgress(handler, onError?): () => void
progressReader(options?: { queueSize?: number }): TypedStreamReader<Record<string, unknown>>

const unsubscribe = robot.gesture.onProgress((frame) => {
  console.log(frame)
})

// or
for await (const frame of robot.gesture.progressReader()) {
  console.log(frame)
}

---

motor

robot.motor lists motors, controls torque/homing/velocity, and streams joint state, errors, and commands.

list, torque, and homing

list(): Promise<Record<string, unknown>>
on(options: { motor: string }): Promise<unknown>
off(options: { motor: string }): Promise<unknown>
onAll(): Promise<unknown>
offAll(): Promise<unknown>
home(options: { motor: string }): Promise<unknown>
homeAll(): Promise<unknown>
setVelocity(options: { motor: string; velocity: number }): Promise<unknown>

const motors = await robot.motor.list()
await robot.motor.onAll()
await robot.motor.home({ motor: 'HeadYaw' })
await robot.motor.setVelocity({ motor: 'HeadYaw', velocity: 50 })
await robot.motor.offAll()

Stream: joint state, errors, and commands

onJointsState(handler, onError?): () => void
jointsStateReader(options?): TypedStreamReader<JointStateFrame>
onJointsError(handler, onError?): () => void
jointsErrorReader(options?): TypedStreamReader<Record<string, unknown>>
openJointsCommandWriter(): TypedStreamWriter<JointCommandFrame>

• jointsState — outbound JointStateFrame (position, velocity, effort, voltage, temperature) per motor.
• jointsError — outbound error flags (overload/voltage/temperature/sensor) per motor, when present.
• jointsCommand — inbound writer for sending joint position/velocity commands.

const unsubscribe = robot.motor.onJointsState((frame) => console.log(frame))

const writer = robot.motor.openJointsCommandWriter()
writer.write(new JointCommandFrame(...))

---

media

robot.media plays audio and video on two independent lanes — foreground (fg) and background (bg) — from files or as live streamed frames. fg and bg expose an identical API; examples below use fg.

File playback

playFgAudioFile(options: { uri: string; signal?: AbortSignal }): Promise<boolean>
pauseFgAudioFile(): Promise<unknown>
resumeFgAudioFile(): Promise<unknown>
setFgAudioVolume(options: { value: number }): Promise<unknown>
getFgAudioVolume(): Promise<number>

await robot.media.playFgAudioFile({ uri: '/data/audio/hello.wav' })

const controller = new AbortController()
const p = robot.media.playFgAudioFile({ uri: '/data/audio/hello.wav', signal: controller.signal })
setTimeout(() => controller.abort(), 3000)

await robot.media.setFgAudioVolume({ value: 0.8 })

playBgAudioFile / pauseBgAudioFile / resumeBgAudioFile / setBgAudioVolume / getBgAudioVolume mirror the fg methods on the background lane.

Video file playback

playFgVideoFile(options: { uri: string; speed?: number; with_audio?: boolean; signal?: AbortSignal }): Promise<boolean>
pauseFgVideoFile(): Promise<unknown>
resumeFgVideoFile(): Promise<unknown>
setFgVideoAlpha(options: { value: number }): Promise<unknown>

await robot.media.playFgVideoFile({ uri: '/data/video/intro.mp4' })
await robot.media.setFgVideoAlpha({ value: 0.8 })

playBgVideoFile / pauseBgVideoFile / resumeBgVideoFile mirror the fg methods (no alpha control on bg).

Streamed audio/video

For audio/video frames you generate yourself, rather than a file on disk:

openFgAudioStreamWriter(): TypedStreamWriter<AudioFrameRaw>
openBgAudioStreamWriter(): TypedStreamWriter<AudioFrameRaw>
openFgVideoStreamWriter(): TypedStreamWriter<ImageFrameRaw>
openBgVideoStreamWriter(): TypedStreamWriter<ImageFrameRaw>

cancelFgAudioStream() / pauseFgAudioStream() / resumeFgAudioStream()
cancelBgAudioStream() / pauseBgAudioStream() / resumeBgAudioStream()
cancelFgVideoStream() / pauseFgVideoStream() / resumeFgVideoStream()
cancelBgVideoStream() / pauseBgVideoStream() / resumeBgVideoStream()

const writer = robot.media.openFgAudioStreamWriter()
await writer.write(new AudioFrameRaw(...))
...
await robot.media.cancelFgAudioStream()

---

speaker

robot.speaker controls the robot's overall hardware volume — separate from any individual media lane's volume (see media).

setVolume(options: { value: number }): Promise<boolean>
getVolume(): Promise<number>
mute(): Promise<boolean>
unmute(): Promise<boolean>

await robot.speaker.setVolume({ value: 0.8 })
const vol = await robot.speaker.getVolume()
await robot.speaker.mute()
await robot.speaker.unmute()

---

microphone

robot.microphone reads raw audio from the robot's internal 5-channel microphone array (and an optional external mic), and tunes the array's DSP.

DSP tuning

getIntTuning(): Promise<Record<string, unknown>>
setIntTuning(options: { name: string; value: number }): Promise<boolean>

const params = await robot.microphone.getIntTuning()
await robot.microphone.setIntTuning({ name: 'AGCONOFF', value: 1.0 })

Audio and voice-activity streams

onIntAudioCh0(handler, onError?): () => void
intAudioCh0Reader(options?): TypedStreamReader<AudioFrameRaw>
// ... ch1, ch2, ch3, ch4 follow the same pattern

onExtAudioCh0(handler, onError?): () => void
extAudioCh0Reader(options?): TypedStreamReader<AudioFrameRaw>

onIntEvent(handler, onError?): () => void
intEventReader(options?): TypedStreamReader<Record<string, unknown>>

Channel 0 is the processed/ASR-ready channel; channels 1–4 are raw per-microphone signals. The external mic stream is only published if enabled in the robot's configuration. The voice-activity/direction-of-arrival event stream carries { activity: boolean, direction: number } (degrees, 0–359).

for await (const frame of robot.microphone.intAudioCh0Reader()) {
  process(frame)
}

robot.microphone.onIntEvent((frame) => {
  if (frame.activity) console.log('Voice detected — DOA:', frame.direction)
})

---

camera (plugin)

robot.camera is the SDK's one plugin namespace so far, exposing the RealSense 3D camera. Enable it first — see Plugins.

Intrinsics

getColorIntrinsics(): Promise<Record<string, unknown>>
getDepthIntrinsics(): Promise<Record<string, unknown>>
getDepthScale(): Promise<Record<string, unknown>>

const intrinsics = await robot.camera!.getColorIntrinsics()
const { scale } = await robot.camera!.getDepthScale()

Image streams

onColor(handler, onError?): () => void
colorReader(options?): TypedStreamReader<ImageFrameRaw>
onDepth(handler, onError?): () => void
depthReader(options?): TypedStreamReader<ImageFrameRaw>
onDepthAligned(handler, onError?): () => void
depthAlignedReader(options?): TypedStreamReader<ImageFrameRaw>
onDepthColor(handler, onError?): () => void
depthColorReader(options?): TypedStreamReader<ImageFrameRaw>

Stream	Description
color	Color image (BGR, width × height × 3).
depth	Raw depth image (16-bit).
depthAligned	Depth aligned to the color frame.
depthColor	False-colour depth, colourised for visualisation.

for await (const frame of robot.camera!.colorReader()) {
  render(frame)
}

IMU streams

onGyro(handler, onError?): () => void
gyroReader(options?): TypedStreamReader<unknown[]>
onAcceleration(handler, onError?): () => void
accelerationReader(options?): TypedStreamReader<unknown[]>

Both deliver [x, y, z] — angular velocity in rad/s for gyro, linear acceleration in m/s² for acceleration.

Live media tracks (browser)

robot.extra (core connection) and robot.camera.extra (camera plugin) expose native WebRTC MediaStreamTracks — only available over a WebRTC connection with use_media_channels: true (the gateway default):

receiveVideoTrack(topic = 'video'): Promise<MediaStreamTrack>
receiveAudioTrack(topic = 'audio'): Promise<MediaStreamTrack>
sendVideoTrack(track: MediaStreamTrack, topic = 'video'): void
sendAudioTrack(track: MediaStreamTrack, topic = 'audio'): void

receive*Track resolves once the remote peer's track arrives — attach it directly to an HTML element:

const audioTrack = await robot.extra.receiveAudioTrack()
audioEl.srcObject = new MediaStream([audioTrack])
audioEl.play()

const videoTrack = await robot.camera!.extra.receiveVideoTrack()
videoEl.srcObject = new MediaStream([videoTrack])
videoEl.play()

send*Track registers a local track (e.g. from getUserMedia) to send to the robot — it must be called from preConnect (see Connecting), before the connection is established:

const robot = await Robot.connectWebrtcMqtt(brokerUrl, robotId, {
  preConnect: async (conn) => {
    const stream = await navigator.mediaDevices.getUserMedia({ video: true })
    conn.sendVideoTrack(stream.getVideoTracks()[0])
  },
})

---

See the TypeScript/Node.js Tutorials for runnable examples, including a full set of browser HTML demos under examples/web/.