Create an interactive memory game using cards
Goal: learn how to create an interactive memory game using QTrobot image recognition
Requirements:
In this tutorial, we implement a simple memory game using QTrobot Studio and QTrobot image recognition software. QTrobot shows a sequence of cards to the user and user should remember them and repeat the same sequence. QTrobot has two cards, one in each hand. One of the card (in robot's right hand) is marked with letter Q and the other one (in robot's left hand) has letter T on it. User has the same set of cards. After showing the cards to the user, QTrobot asks the user to show them one-by-one and in the correct order to the QTrobot. The robot starts with a simple sequence and adds more sequence as the game progress. Here is a short video of the scenario:
How does the QTrobot's image recognition work?
Now let’s see how we can easily implement our scenario in QTrobot Studio. There is ROS image recognition module called find_object_2d which is running on the robot. Using QTrobot's camera, the find_object_2dsimply detect whatever images that are in ~/robot/data/images folder on QTPC of QTrobot. For this tutorial, we are using the following images. These images already exist in the corresponding folder to be recognized by find_object_2d.
If we show one of the above card to QTrobot, the find_object_2drecognize it and publishes the corresponding id (number in the filename) along with other information to /find_object/objects topic. The topic uses a message of standard type std_msgs/Float32MultiArray. The first element in the list is the id of the image which is of our interest for this scenario. For example, if we show the T card image (i.e 2.jpg), the following message will be published. The first element in data field (2.0) is the id of the 2.jpg image.
layout:
dim: []
data_offset: 0
data: [2.0, 553.0, 553.0, 0.582885205745697, -0.03982475772500038, ...
---
Implementation
First, let's see how we can implement the above scenario using image recognition and QTrobot Studio.
First we try to read a message from /find_object/objects topic using ROS Subscriber block and check if the message is not empty. After that, we extend it so that we wait until we read a detected image (Q or T) for some times. If no image detect after dedicated time, we assume that user left the game and we stop the game. Now we have our main building blocks, we implement the logic of the game as it is described in the scenario, and finally we put all the pieces together and adds some speech messages to make our games more interesting.
1. Read the detected image
We can use the ROS Subscriber block to read the message published by find_object_2d as it shown here:
find_object_2d continuously publishes messages independently whether any image detected or not. Therefore, we need to check if actually any image is detected by checking the data field of our message. if the data list is not empty, we have a detected image.
2. Extend the blocks to wait for a detect image and extract its ID
In our scenario we are interested to know if the user has left the game and stop the game correspondingly. We can do this by giving some time limit to the user to shows the cards. If no card detected within that time limit, we assume that user has left the game.
To facilitate this, we can implement a subprogram (also known as a function) to do that. We call it ReadImageID. Our subprogram waits for an image to be detected. If the image is detected it returns the image id (e.g. 1 or 2). If no image is detected within 15 seconds, our subprogram return -1. The following blocks shows the implementation of our subprogram:
Now we have our subprogram we can use it like any other block.
3. Showing sequence of Q and T by QTrobot
As we have explained in our scenario, first QTrobot shows a sequence of Q and T to user. The number of letters in the sequence increases as the game progress. One simple way to implement this is using a list of numbers 1 or 2. We can assume 1 is represent Q and 2 represents T. Here are the steps we can follow to implement that:
- first we create a list which contains one of of the random number 1 or 2.
- then at each step of the game we add one more random number from 1 or 2. For example, our list can look like this as game progress:
step 1: 1
step 2: 12
step 3: 121
step 4: 1211
...
Let see how we can implement it using our blocks:
As shown above, first we create an empty list and called it robot_mem. Then in a repeat loop, we use list insert block to randomly adds one of the number from 1 or two to our robot_mem list. So at first, our list should have only one item and as the game progress, it will have more items. Then we use a ForEach loop block to go through each item in robot_mem list and check if it's 1 or 2, If it's 1 we use Show-says-act block to say Q and play show_right gesture to show the Q card which is in robot's right hand. Otherwise we say T and play show_left gesture to show the T card.
4. Checking the sequence shown by user
After we ask QTrobot to shows the sequence, we ask users to do the same and check the user shown sequence. Above, we have already implemented our subprogram block to recognize the card shown by user. So we just need to go again through our robot_mem list, read the shown card by user and check it with each item in the list.
As soon we see a wrong card shown by user, we say that the user lost the game and we can stop the game. otherwise, we say the name of the card that user has shown. Here is the block implementation of what we have just explained:
5. Put it all together
Now we have all our building blocks for showing the robot sequence and checking the user shown sequence, we can put then together and finalize our scenario.
We add some introductory messages at the beginning of the game to explain to the user the game scenario. Then we wrap all blocks by LuxAI repeat until I press stop block. This repeat blocks keeps our game running until we stop it using Educator Tablet. However, we also want to stop the game if user left the game. In our subprogram, we already implemented the logic to understand if user is still playing or he/she has left the game by adding a timeout to our subprogram. We create a simple variable called game_finished and set it to false. Then we add an condition block (IF) to not repeat the loop whenever whenever game_finished is true. We also add a condition in our If block where we check the user shown card. If our subprogram for detecting card's ID return -1, we know that user has left the game and we set the game_finished variable to true. We also ask QTrobot to react correspondingly. Here is the complete source of our memory game: