Version: QTrobot V2

QTrobot Riva speech recognition (offline)

This tutorial requires QTrobot AI@Edge variant!

Level: Advanced
Goal: learn how to use QTrobot Offline speech recognition
Requirements:

In this tutorial we will learn about how to use QTrobot Offline speech recognition using Nvidia Riva.

Install Nvidia Riva ASR

Sign in to your Nvidia NGC portal (sign up if you don't have an account).
Follow the instructions for Generating Your NGC API Key to obtain an API key.
Use your API key to log in to the NGC registry by entering the following command and following the prompts.

Note: The ngc CLI is already installed on the QTrobot, so no need to install it.
```
ngc config set
```

Download the Embedded Riva start scripts to the ~/robot folder using the following command:

cd ~/robot/      
ngc registry resource download-version nvidia/riva/riva_quickstart_arm64:2.14.0

Modify config.sh in ~/robot/riva_quickstart_arm64_v2.14.0/config.sh to disable unnecessary services, keeping only service_enabled_asr enabled. The relevant part of config.sh should look like this:

# ...
# Enable or Disable Riva Services
service_enabled_asr=true
service_enabled_nlp=false
service_enabled_tts=false
service_enabled_nmt=false
# ...
# For multiple languages enter space separated language codes such as ("en-US" "fr-FR").
asr_language_code=("en-US")
# ...

Initialize the Riva ASR container. This may take around 30 minutes, depending on your internet speed.
```
cd ~/robot/riva_quickstart_arm64_v2.14.0
bash ./riva_init.sh
```

Start Nvidia Riva ASR

First, ensure that the Riva ASR Docker service is running. If Riva is not active, it may take 1 to 2 minutes to initialize.

cd ~/robot/riva_quickstart_arm64_v2.14.0
bash ./riva_start.sh ./config.sh -s

Launch qt_riva_asr_app

You need to run the qt_riva_asr_app ROS node if it is not already running. You can check if the node is running using rosnode list.

roslaunch qt_riva_asr_app qt_riva_asr_app.launch

By default, the qt_riva_asr_app is configured to use en-US with Nvidia Riva ASR. To change the language, for example to fr-FR,

first ensure that you have already enabled the fr-FR language during the installation of Nvidia Riva ASR in the previous step.
change the language code to fr-FR in ~/catkin_ws/src/qt_riva_asr_app/config/qt_riva_asr_app.yaml:
```
qt_riva_asr_app:
    language_code: 'fr-FR'
    # ...
```

Create a python project

First we create a python project for our tutorial. let's call it tutorial_riva_vosk and add the required python file:

cd ~/catkin_ws/src
catkin_create_pkg tutorial_qt_riva rospy roscpp -D "Using Riva speech recognition"
cd tutorial_qt_riva/src
touch tutorial_qt_riva_node.py
chmod +x tutorial_qt_riva_node.py

QTrobot speech service

Open the tutorial_qt_riva_node.py file and add the following code:

#!/usr/bin/env python3
import sys
import rospy
from qt_robot_interface.srv import *
from qt_vosk_app.srv import *

if __name__ == '__main__':

    rospy.init_node('tutorial_qt_riva_node')
    rospy.loginfo("tutorial_qt_riva_node started!")

    # define a ros service
    speechSay = rospy.ServiceProxy('/qt_robot/speech/say', speech_say)
    recognize = rospy.ServiceProxy('/qt_robot/speech/recognize', speech_recognize)
    # block/wait for ros service
    rospy.wait_for_service('/qt_robot/speech/say')
    rospy.wait_for_service('/qt_robot/speech/recognize')

    speechSay("Say something after the beep.")
    while not rospy.is_shutdown():
        speechSay('#CAR HORN#')
        rospy.loginfo("Listening...")
        resp = recognize(language='en-US', options='', timeout=0)
        rospy.loginfo("I got: %s", resp.transcript)
        if resp.transcript:
            speechSay("You said %s " % resp.transcript)

    rospy.loginfo("finsihed!")

Tips

You can run the Nivida Riva ASR docker automatically when the QTrobot starts. To do that

first create a startup file in ~/robot/autostart folder:

tocuh ~/robot/autostart/start_riva_server.sh
chmod +x ~/robot/autostart/start_riva_server.sh
nano ~/robot/autostart/start_riva_server.sh

and copy past the follwing content:

# !/bin/bash
source /home/qtrobot/robot/autostart/qt_robot.inc
SCRIPT_NAME="start_riva_server"
LOG_FILE=$(prepare_logfile "$SCRIPT_NAME")
{
cd ~/robot/riva_quickstart_arm64_v2.14.0
bash ./riva_start.sh ./config.sh -s
} &>> ${LOG_FILE}

add run_script "start_riva_server.sh" into ~/robot/autostart/autostart_screens.sh file before the last line as follows:
```
#...
#...
run_script "start_riva_server.sh"
} &>> ${LOG_FILE}        
```

Install Nvidia Riva ASR​

Start Nvidia Riva ASR​

Launch qt_riva_asr_app​

Create a python project​

QTrobot speech service​