QTrobot Emotion and Face

QTrobot has an integrated 800x480 standard display in the head which is mainly used for showing facial emotions. However, the use of the display is not limited and developers can use it as standard screen for displaying any visual content or even use it with Linux graphical desktop. The display is connected to QTRP (Raspberry Pi board in the head) via HDMI interface as shown in the image below. QTrobot's display is not touchscreen; simply because we do not want to encourage kids and other users to interact with the robot by touching its face.

afraidangryblowing raspberrybreathing exercisebrushing teethbrushing teeth foamcalmig down exercise nosecalming downconfusedcrydirty facedirty face saddirty face washdisgustedhappyhappy blinkingkissneutralneutral state blinkingpuffing the chredo eekssadscreamshowing smileshytalkingwith a coldwith a cold cleaning nosewith a cold sneezingyawn

Specifications
Screen Size	8.0 (Diagonal) inch
Display Format	800RGB(H)×480(V) dot
Active Area	176.64(H)×99.36(V) mm
Pixel Size	0.0736(H)×0.2070(V) mm

---

Software interface

The qt_robot_interface ROS node is the QTrobot's display driver. This node implements varieties of ROS interfaces including the Emotion interface. Using emotion interface, one can show plenty of default QTrobot facial expressions such as happy, sad, surprised, etc. The facial expresions (a.k.a emotions) are simply video files which are located and categorized in QTRP under ~/robot/data/emotions folder.

Showing default emotions

The qt_robot_interface implements both ROS service and topic interfaces to handle emotions. To display QTrobot's default emotion you can use one of the following methods:

• Using Educator tablet app
• Publishing to /qt_robot/emotion/show topic
• Calling /qt_robot/emotion/show service

Here are some examples for showing QTrobot happy emotion which is under ~/robot/data/emotions/QT folder. Open a terminal on QTPC and run the following commands:

Using Publisher:

rostopic pub /qt_robot/emotion/show std_msgs/String "data: 'QT/happy'"

Using Service call:

rosservice call /qt_robot/emotion/show "name: 'QT/happy'"

Note

As it shown in the above example, you should not specify the emotion's file extension (.avi)!

Emotion interface config

The qt_robot_interface has a config file which is located under /opt/ros/noetic/share/qt_robot_interface/config folder. This config file also incudes configuration of emotion interface such as setting startup emotion and idle emotion to be shown on the robot. Under emotion section you will see following lines:

  emotion:
        path:  '/home/qtrobot/robot/data/emotions'
        display_width: 800
        display_height: 480
        fps_ratio: 2.5
        idle_emotion: 'QT/neutral_state_blinking.avi'
        startup_emotion: 'QT/yawn.avi'
        disable_interface: false

Disabling QTrobot emotion interface

As we have explained above, you can display your custom video file using emotion interface. However, you may have some particular scenario where you want to implement your own image/video player for QTrobot. In this case, you can disable only the emotion part of qt_robot_interface and have the display freely for your own setup. To do that simple modify the /opt/ros/noetic/share/qt_robot_interface/config/qtrobot-interface.yaml file as follows:

  emotion:
        ...
        disable_interface: true

Note

The Behavior interface (i.e. TalkText and TalkAudio) needs emotion interface. So Please notice that if you disable the QTrobot emotion interface, TalkText and TalkAudio will be automatically disabled too.

---

Creating and playing your own emotion

QTrobot emotion interface can play different video files including

• AVI (H.264 X.264 or H.264,)
• MP4 (v2)
• MOV (Apple QuickTime movie)

Despite which format your video files has, it should be always named with .avi extension.

Playing a video file

To display custom video, simply copy it to the default emotions data folder (~/robot/data/emotions) on QTRP and play it similar to any other emotions suing publisher or service call. Let's look at the following example.

• Video name: "myvideo.avi"
• Video path: "~/robot/data/emotions/myvideo.avi"

rostopic pub /qt_robot/emotion/show std_msgs/String "data: 'myvideo'"

The video will be stopped when it finishes. However, you can always interrupt and stop it using emotion stop service call:

rosservice call /qt_robot/emotion/stop "{}"

Tips

To have a better quality and have more optimized video files for QTrobot, it's better you create/resize the video to 800x480 pixels with lower framerate (less than 18fps).

Displaying an image file (JPG, PNG)

The QTrobot emotion interface also support image file format. but the issue with showing it is that images does not have a timeframes and immediately disappears after showing up on the face. For the time being, the best way to show an image for a desired period is to convert it to simple AVI file with the desired delay. You can use the ffmpeg command in Linux to convert an image to a video file as shown here:

ffmpeg -loop 1 -framerate 18 -i image.jpg -c:v libx264 -t 10 -pix_fmt yuv420p myimage.avi

the above command creates myvideo.avi video file which shows the image.jpg for 10 second.

Displaying a GIF file

Same as the above procedure, we need to convert the GIF file into an video file. We can do that using ffmpeg tools as shown bellow:

ffmpeg -r 10  -i image.gif -movflags faststart -pix_fmt yuv420p -vf "scale=trunc(iw/2)*2:trunc(ih/2)*2" mygif.avi

You can change the framerate parameter -r to make the gif animation slower (e.g. -r 4) or faster (e.g. -r 18).

Tips

You can learn more about the ffmpeg powerful tool for creating more interesting contents such as slideshow here: ffmpeg wiki