Remote Control
Since the Jetbot official Teleoperation program needs a third party Gamepad Controller to control Jetbot, so I create this project based on Flask and HTML/JavaScript, with this project, you can only use mobile phone browser to control Jetbot.
Here we use Flask to build back-end Web server, and use HTML/CSS/JavaScript to build front-end Web GUI.
-
Downloading project repo
$ git clone https://github.com/mayuanjason/mbot.git -
Installing Flask
$ sudo pip3 install flask -
Installing python-dotenv, for managing all the environment variables of this project
$ sudo pip3 install python-dotenv -
Installing Watchdog
$ sudo pip3 install watchdog -
Installing Adafruit Python Library for DC + Stepper Motor HAT (Optinal)
Adafruit-Motor-HAT-Python-Library should be already installed on Jetson Nano if you follow the guide Getting Started With Jetson Nano Developer Kit
$ git clone https://github.com/adafruit/Adafruit-Motor-HAT-Python-Library.git $ cd Adafruit-Motor-HAT-Python-Library/ $ python setup.py install -
Installing traitlets (Optional)
The Motor module used by Jetbot depend on traitlets module，so we need to install traitlets module. If you follow the guide Getting Started With Jetson Nano Developer Kit, traitlets should be already installed on Jetson Nano.
$ sudo pip3 install traitlets
-
Enter mbot directory
$ cd mbot/mbot -
Run the application
$ CAMERA=jetbot python3 app.py -
Viewing the Web GUI on another device on your network
http://x.x.x.x:5000/Where
x.x.x.xis the IP address of Jetbot. For example, if the IP address of Jetbot is192.168.0.106, then enter the following URL on your tablet or mobile phone:http://192.168.0.106:5000/If everything goes well, your web browser should look something like the following:
|--mbot/
|--static/
|--css/
mbot_gui.css
|--images/
|--js/
jquery-3.3.1.min.js // jQuery resource file
mbot_gui.js // front-end JS file
|--templates/
mbot_gui.html // front-end template file
app.py // Web server
base_camera.py // base camera drvier
camera_jetbot // jetbot camera driver
motor.py // TT motor driver
robot.py // robot driver
4.1. app.py
0: # -*- coding: utf-8 -*-
1: from importlib import import_module
2: import os
3: from flask import Flask, render_template, Response
4: from robot import Robot
5:
6: # import camera driver
7: if os.environ.get('CAMERA'):
8: Camera = import_module('camera_' + os.environ['CAMERA']).Camera
9: else:
10: from camera import Camera
11:
12: # Raspberry Pi camera module (requires picamera package)
13: # from camera_pi import Camera
line 3: Load the Flask module into your Python Script。
line 4: Load the Robot module. Robot module depends on Motor module, Download robot.py and motor.py from Jetbot Nano GitHub, and save them into mbot root directory.
line 6~13: Load the Camera module. Please issue $ CAMERA=jetbot python3 app.py command if your run this code on jetbot platform.
16: app = Flask(__name__)
17:
18: robot = Robot()
19:
20:
21: @app.route('/')
22: def mbot_gui():
23: return render_template('mbot_gui.html')
24:
25:
26: @app.route('/teleop/<opcode>')
27: def teleop(opcode):
28: if opcode == 'forward':
29: robot.forward()
30: elif opcode == 'backward':
31: robot.backward()
32: elif opcode == 'left':
33: robot.left()
34: elif opcode == 'right':
35: robot.right()
36: elif opcode == 'stop':
37: robot.stop()
38:
39: return '', 204
line 16: Create Flask instance.
line 18: Create Robot instance.
line 21~23: Register router. Python decorators @app.route('/') binds the root directory / and function mbot_gui() all together. So the function mbot_gui() will be triggered when you access to the URL http://localhost:5000/. And function mbot_gui() will render the template file mbot_gui.html, so you can see the whole web page on your browser.
line 26~39: Register router. But this time, we add variable part(<opcode>) into the URL. Variable <opcode> will be passed into function teleop() when Flask handle request sending from customer. For example, when you access to the URL http://localhost:5000/teleop/forward, function teleop() should be called, and opcode should be set to forward.
There has one defect in funtion teleop(), this fucntion can only control Jetbot to go to 4 directions(12 o'clock, 3 o'clock, 6 o'clock and 9 o'clock), so if you want Jetbot to go 45°（2 o'clock）ahead to the right, it cannot do that. We will imporve this in the following section.
42: def gen(camera):
43: """Video streaming generator function."""
44: while True:
45: frame = camera.get_frame()
46: yield (b'--frame\r\n'
47: b'Content-Type: image/jpeg\r\n\r\n' + frame + b'\r\n')
48:
49:
50: @app.route('/video_feed')
51: def video_feed():
52: """Video streaming route. Put this in the src attribute of an img tag."""
53: return Response(gen(Camera()),
54: mimetype='multipart/x-mixed-replace; boundary=frame')
For this section, please refer to the below two webpages:
57: if __name__ == '__main__':
58: app.run(host='0.0.0.0', port=5000, threaded=True)
host='0.0.0.0' means our web server can listen all the external requests, so if you have mobile phone or other device connected to the same router as the Jetbot, then you should be able to open a web browser on that device and point it to the following URL:
http://x.x.x.x:5000/
Where x.x.x.x is the IP address of your Jetbot. For example, if the Jetbot has local IP address 192.168.0.106, then enter the following URL on your mobile phone or other device:
http://192.168.0.106:5000
If everything goes well, you should see the Web GUI on your mobile phone, then you can use the touch screen to control your Jetbot by tapping on the arrow keys.
4.2. mbot_gui.html
10: <td colspan="1" width="100%">
11: <img src="{{ url_for('video_feed') }}">
12: </td>
Refer to the above webpages.
17: <table width="100%" border="0" cellpadding="0" cellspacing="0">
18: <tr>
19: <td colspan="3" style="text-align: center;">
20: <img id="forward" src="{{ url_for('static', filename='images/arrow-forward.jpg') }}" width="100" alt="" align="middle">
21: </td>
22: </tr>
23: <tr>
24: <td width="33%" style="text-align: center;">
25: <img id="left" src="{{ url_for('static', filename='images/arrow-left.jpg') }}" width="100" alt="" align="middle">
26: </td>
27: <td width="33%" style="text-align: center;">
28: <img id="stop" src="{{ url_for('static', filename='images/stop.jpg') }}" width="100" alt="" align="middle">
29: </td>
30: <td width="33%" style="text-align: center;">
31: <img id="right" src="{{ url_for('static', filename='images/arrow-right.jpg') }}" width="100" alt="" align="middle">
32: </td>
33: </tr>
34: <tr>
35: <td colspan="3" style="text-align: center;">
36: <img id="backward" src="{{ url_for('static', filename='images/arrow-backward.jpg') }}" width="100" alt="" align="middle">
37: </td>
38: </tr>
39: </table>
Add five buttons on the Web GUI: forward, left, stop, right and backward.
4.3. mbot_gui.js
0: $(function() {
1: $('#forward').click(function() {
2: $.ajax({
3: url: '/teleop/forward',
4: type: 'get',
5: })
6: })
7:
8: $('#left').click(function() {
9: $.ajax({
10: url: '/teleop/left',
11: type: 'get',
12: })
13: })
14:
15: $('#stop').click(function() {
16: $.ajax({
17: url: '/teleop/stop',
18: type: 'get',
19: })
20: })
21:
22: $('#right').click(function() {
23: $.ajax({
24: url: '/teleop/right',
25: type: 'get',
26: })
27: })
28:
29: $('#backward').click(function() {
30: $.ajax({
31: url: '/teleop/backward',
32: type: 'get',
33: })
34: })
35: })
Register a single click event handler for the each button (see .click(function(){...})). Web browser will send an AJAX request to our Web server (and set URL to /teleop/forward) when we tapping forward arrow key on the tounch screen, and it will trigger function teleop(opcode) and set opcode to forward, and it will lead Jetbot to run forward.
-
Enter mbot2 directory
(mbot) $ cd mbot/mbot2 -
Run the application
(mbot) $ CAMERA=jetbot python3 app.py -
Viewing the Web GUI on another device on your network
http://x.x.x.x:5000/Where
x.x.x.xis the IP address of Jetbot. For example, if the IP address of Jetbot is192.168.0.106, then enter the following URL on your tablet or mobile phone:http://192.168.0.106:5000/If everything goes well, your web browser should look something like the following:
