Skip to main content
Version: QTrobot V3

Plugins

The previous tutorials (Camera, ASR, Kinematics) each enabled a specific plugin. This page covers the underlying mechanism in more depth — sharing a connection's broker, enabling/disabling on demand, and remote signaling — using the realsense-driver plugin as a running example. These examples assume you already have a connected robot — see Connection if you haven't set one up yet.

Setup​

mkdir ~/example
cd ~/example
python -m venv .venv
# or: uv venv .venv

source .venv/bin/activate

pip install luxai-robot
# or: uv pip install luxai-robot

Connect​

from luxai.robot.core import Robot

robot = Robot.connect_zmq(robot_id="QTRD000123")
print(f"connected to {robot.robot_id} ({robot.robot_type})")

See Connection for MQTT, WebRTC, and other connection options — the MQTT and WebRTC examples below show their own connect_mqtt/connect_webrtc_mqtt calls directly.

Over MQTT​

The robot and its plugins must be running qtrobot-service-hub-gateway-mqtt, which bridges ZMQ services — including plugin services — to an MQTT broker. Each plugin's node_id becomes its MQTT topic namespace.

pip install "luxai-robot[mqtt]"

Shared broker — the simplest and most common case. When connected via Robot.connect_mqtt(), the broker connection is shared with plugin transports automatically — no extra configuration needed:

from luxai.robot.core import Robot

robot = Robot.connect_mqtt("mqtt://10.231.0.1:1883", "QTRD000320")
robot.enable_plugin_mqtt("realsense-driver", node_id="qtrobot-realsense-driver")

intrinsics = robot.camera.get_color_intrinsics()
Logger.info(f"Color intrinsics: {intrinsics}")

robot.close()

Enable, use, then disable — frees the plugin's resources while the main robot connection stays open:

robot.enable_plugin_mqtt("realsense-driver", node_id="qtrobot-realsense-driver")
intrinsics = robot.camera.get_color_intrinsics()

robot.disable_plugin("realsense-driver")
# broker connection is still open here

Stream a frame:

reader = robot.camera.stream.open_color_reader()
frame = reader.read(timeout=5.0)
if frame is not None:
    Logger.info(f"Color frame: shape={frame.value.shape}")

A different broker than the robot — useful when the plugin is managed by a separate gateway or network segment:

robot.enable_plugin_mqtt(
    "realsense-driver",
    node_id="qtrobot-realsense-driver",
    broker_url="mqtt://192.168.2.100:1883",  # separate gateway/broker
)

Context manager — robot.close(), including plugin teardown, happens automatically on exit:

with Robot.connect_mqtt("mqtt://10.231.0.1:1883", "QTRD000320") as robot:
    robot.enable_plugin_mqtt("realsense-driver", node_id="qtrobot-realsense-driver")
    intrinsics = robot.camera.get_color_intrinsics()

Over WebRTC​

Each plugin gets its own independent WebRTC peer connection — with a dedicated data channel and its own media tracks — so plugin streams never conflict with the robot's own streams.

pip install "luxai-robot[webrtc]"

Inherit signaling — when connected via connect_webrtc_mqtt() or connect_webrtc_zmq(), all signaling parameters (broker, options, timeout) are inherited by the plugin peer automatically:

robot = Robot.connect_webrtc_mqtt("mqtt://10.231.0.1:1883", "QTRD000320")

# No broker_url needed — reuses the robot's signaling setup
robot.enable_plugin_webrtc_mqtt("realsense-driver", node_id="qtrobot-realsense-driver")

intrinsics = robot.camera.get_color_intrinsics()
scale = robot.camera.get_depth_scale()
robot.close()

Enable, use, then disable — the plugin's WebRTC peer closes; the robot's own peer stays connected:

robot.enable_plugin_webrtc_mqtt("realsense-driver", node_id="qtrobot-realsense-driver")
intrinsics = robot.camera.get_color_intrinsics()

robot.disable_plugin("realsense-driver")

Stream a frame from the plugin's dedicated media track:

reader = robot.camera.stream.open_color_reader()
frame = reader.read(timeout=5.0)

A different signaling broker than the robot:

robot.enable_plugin_webrtc_mqtt(
    "realsense-driver",
    node_id="qtrobot-realsense-driver",
    broker_url="mqtt://192.168.2.100:1883",  # separate gateway/broker
)

Full mTLS signaling + STUN/TURN — both the robot peer and the plugin peer inherit the same security configuration from the robot connection:

from luxai.robot import MqttOptions, MqttTlsOptions, WebRTCOptions, WebRTCTurnServer

robot = Robot.connect_webrtc_mqtt(
    "mqtts://10.231.0.1:8883",
    "QTRD000320",
    mqtt_options=MqttOptions(
        tls=MqttTlsOptions(
            ca_file="/path/to/ca.crt",
            cert_file="/path/to/client.crt",
            key_file="/path/to/client.key",
        ),
    ),
    webrtc_options=WebRTCOptions(
        stun_servers=["stun:stun.l.google.com:19302"],
        turn_servers=[WebRTCTurnServer(url="turn:turn.example.com:3478", username="user", credential="pass")],
    ),
)

# Plugin peer inherits mTLS + STUN/TURN — no extra config needed
robot.enable_plugin_webrtc_mqtt("realsense-driver", node_id="qtrobot-realsense-driver")

Broker-less, ZMQ-signaled (LAN only) — the plugin peer inherits the ZMQ signaling endpoint automatically:

robot = Robot.connect_webrtc_zmq("tcp://192.168.3.152:5555", "QTRD000320")
robot.enable_plugin_webrtc_zmq("realsense-driver", node_id="qtrobot-realsense-driver")

Context manager — all peers (robot + plugins) close automatically on exit:

with Robot.connect_webrtc_mqtt("mqtt://10.231.0.1:1883", "QTRD000320") as robot:
    robot.enable_plugin_webrtc_mqtt("realsense-driver", node_id="qtrobot-realsense-driver")
    intrinsics = robot.camera.get_color_intrinsics()

Next steps​

See the full plugin mechanism (enable_plugin, enable_plugin_local, enable_plugin_zmq, enable_plugin_mqtt, enable_plugin_webrtc_mqtt, enable_plugin_webrtc_zmq, disable_plugin) in the Python API Reference, or head back to the Tutorials overview to explore TypeScript/Node.js or ROS2.