how to run a jetbot right after bootup?

[HannahZhangVW]

I would like to have the jetbot car run collision avoidance algorithm automatically after boot up.
Here is what I did:

1. converted "live_demo.ipynb" to a python file "live_demo.py".
2. Put "python /home/jetbot/Notebooks/collision_avoidance/live_demo.py in /etc/rc.local.
   However, the live_demo.py can't run because no pytorch, pytorchvision, jetbot could be import.
   I installed AMR version pytorch and pytorchvision sucessfully.
   how to intall jetbot?
   Why "live_demo.ipynb" don't have import problems when it runs on jupytor?

[jaybdub]

Hi HannahZhangVW,

Thanks for reaching out! Do you know which python interpreter you’re calling?

We install dependencies for Python 3 only. My best guess is your new script is being executed with python 2.

Best,
John

[HannahZhangVW]

Hi John Thank you very much for such rapid reply. We talked briefly during the GTC. I already built a jetbot car. It works amazing well. I believe Jetson Nanno will be another very success product. Just as you guessed, I am using python 2. On the command line $python –version $Python 2.7.15rc1 I used the released image jetbot_image_v0p3p0.zip to boot up. Should I install python 3 manually? Thanks! Hannah From: John [mailto:notifications@github.com] Sent: Tuesday, April 30, 2019 9:19 AM To: NVIDIA-AI-IOT/jetbot <jetbot@noreply.github.com> Cc: Zhang, Hannah (VWoA-ERL) <Hannah.Zhang@vw.com>; Author <author@noreply.github.com> Subject: Re: [NVIDIA-AI-IOT/jetbot] how to run a jetbot right after bootup? (#55) Hi HannahZhangVW, Thanks for reaching out! Do you know which python interpreter you’re calling? We install dependencies for Python 3 only. My best guess is your new script is being executed with python 2. Best, John — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub<#55 (comment)>, or mute the thread<https://github.com/notifications/unsubscribe-auth/AL6PUEIVKFW4ULOBU2FGISDPTBWNNANCNFSM4HJJH5UQ>.

[jaybdub]

Awesome! Nice to talk to you again, glad to hear you're enjoying it!

Python3 is actually already installed, all you have to do is call python3 instead of python.

Hopefully that works, if you run into any issues let me know.

Best,
John

[HannahZhangVW]

Hi John, Thank you very much for your suggestion. I create a symbolic link python to point to python3 instead of pyton2. Using python3, import torch, torchvision and jetbot successfully. I found there is no rc.local anymore. What is the best way to run python script automatically after jetson nanno boot? Thank you very much! Hannah From: John [mailto:notifications@github.com] Sent: Tuesday, April 30, 2019 11:53 AM To: NVIDIA-AI-IOT/jetbot <jetbot@noreply.github.com> Cc: Zhang, Hannah (VWoA-ERL) <Hannah.Zhang@vw.com>; Author <author@noreply.github.com> Subject: Re: [NVIDIA-AI-IOT/jetbot] how to run a jetbot right after bootup? (#55) Awesome! Nice to talk to you again, glad to hear you're enjoying it! Python3 is actually already installed, all you have to do is call python3 instead of python. Hopefully that works, if you run into any issues let me know. Best, John — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub<#55 (comment)>, or mute the thread<https://github.com/notifications/unsubscribe-auth/AL6PUELHOIMWW5ANCZV4SCTPTCIQDANCNFSM4HJJH5UQ>.

[jaybdub]

Hi Hannah,

One way is to create a system service that calls your script and enable it to run at boot. We do this for the stats display on JetBot. We actually generate the system service using this script:

https://github.com/NVIDIA-AI-IOT/jetbot/blob/master/jetbot/utils/create_stats_service.py

That script automatically generates the system service file (just filling in a few parameters like the current user). You can easily create a file like this manually. Say you want to create a service called run_my_python_script which executes the python script /home/jetbot/my_python_script.py. You would:

1. Create a file /etc/systemd/system/run_my_python_script.service

2. Edit the following with text like the following

   Description=Runs my python script
   [Service]
   Type=simple
   User=jetbot
   ExecStart=/bin/sh -c "python3 /home/jetbot/my_python_script.py"
   WorkingDirectory=/home/jetbot
   Restart=always
   [Install]
   WantedBy=multi-user.target

3. Start the service manually by calling this in a terminal

   sudo systemctl start run_my_python_script

4. Enable the service at boot by calling this in a terminal

   sudo systemctl enable run_my_python_script

Please let me know if you run into any issues. Hope this helps!

John

[HannahZhangVW]

Hi John, Yesterday, I trained the collision avoidance with my office desk pictures. In the original train script, it saves the best accuracy epoch of 30 epoch. I changed it to save every epoch. Surprisingly, it not only runs well on my desk, it also runs well in my kitchen which I didn’t take any picture. See the video I record yesterday: https://youtu.be/Z4k6rm935rI Another question, could you please share the training script and live demo script for the car following the Lego lane? Thank you very much! Hannah From: John [mailto:notifications@github.com] Sent: Tuesday, April 30, 2019 11:53 AM To: NVIDIA-AI-IOT/jetbot <jetbot@noreply.github.com> Cc: Zhang, Hannah (VWoA-ERL) <Hannah.Zhang@vw.com>; Author <author@noreply.github.com> Subject: Re: [NVIDIA-AI-IOT/jetbot] how to run a jetbot right after bootup? (#55) Awesome! Nice to talk to you again, glad to hear you're enjoying it! Python3 is actually already installed, all you have to do is call python3 instead of python. Hopefully that works, if you run into any issues let me know. Best, John — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub<#55 (comment)>, or mute the thread<https://github.com/notifications/unsubscribe-auth/AL6PUELHOIMWW5ANCZV4SCTPTCIQDANCNFSM4HJJH5UQ>.

[hachpai]

Hi all!

I'm into the same goal, thanks for those instructions.
Someone could give a trained model for obstacle avoidance? I'm lacking of time to produce the dataset and train it by myself... I've two jetbots and i just need them to avoid each others and evident walls in good light conditions.

[RyanC1681]

Hi,
How to run live demo live_demo.py with downloaded pre-trained model?
I tried python3 live_demo.py, but it didn't spin the wheel or no outputs.
Can I put ipynb in in /etc/rc.local or let it continuous run?
Thanks,

[jaybdub]

@HannahZhangVW

Thanks for sharing! Very impressive!

The road following code will be released soon, but we don’t have an exact date. If you’re eager to get started you can check out the notebooks in the road_following_ai branch.

It’s under development, we still need clean the samples and add documentation.

Please let me know if you have any questions or feedback.

Best,
John

[HannahZhangVW]

Hi John, During running the live mode for collision avoidance, sometimes the camera only return one picture, so that the update function only be called once. When this problem happens, if the first picture is free, the car will go forward forever and if the first picture is blocked, the car will spinning forever. Since it happens 50% chance, I have to reboot all the time. Is there a way to make the camera more stable? Best Regards Hannah From: John [mailto:notifications@github.com] Sent: Wednesday, May 08, 2019 12:36 AM To: NVIDIA-AI-IOT/jetbot <jetbot@noreply.github.com> Cc: Zhang, Hannah (VWoA-ERL) <Hannah.Zhang@vw.com>; Mention <mention@noreply.github.com> Subject: Re: [NVIDIA-AI-IOT/jetbot] how to run a jetbot right after bootup? (#55) @HannahZhangVW<https://github.com/HannahZhangVW> Thanks for sharing! Very impressive! The road following code will be released soon, but we don’t have an exact date. If you’re eager to get started you can check out the notebooks in the road_following_ai branch. It’s under development, we still need clean the samples and add documentation. Please let me know if you have any questions or feedback. Best, John — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub<#55 (comment)>, or mute the thread<https://github.com/notifications/unsubscribe-auth/AL6PUEPF3DSOM65FS7M6MU3PUJ7FVANCNFSM4HJJH5UQ>.

[jaybdub]

Hi Hannah,

Rather than rebooting, the following should be an easier method to recover from this state.

1. Restart notebook

2. Restart camera daemon (from terminal)

   sudo systemctl restart nvargus-daemon

Could you try the following and see if this issue persists:

1. Convert model to half precision (and update preprocessing function)

   model = model.half()
   ...
   def preprocess(camera_value):
       ...
       return x.half()

2. Configure Jetson Nano to 5W mode (from terminal)

   sudo nvpmodel -m1

Please let me know if this works for you.

Thanks!
John

[HannahZhangVW]

Hi John, I am using color tape to layout a simple road for the Jetbot car to run. The road also has 90 degree right turns and left turns. The model’s output should be “arrived”, “go forward”, “left turn”, and “right turn”. I am going to make 4 directories under the “dataset” folder. The 4 directories names are “arrived”, “forward”, left”, and “right” . I will collect images and put them into the corresponding directory. In the training python script, I will modify the “collision avoidance” ‘s training_model.ipynb in the following place: Change from: model.classifier[6] = torch.nn.Linear(model.classifier[6].in_features, 2) To: model.classifier[6] = torch.nn.Linear(model.classifier[6].in_features, 4) In the live_demo.ipynb, I would like to modify collision_avoidance/live_demo.ipynb in the update function (changes shown in green color and red color text). //////////////////////////////////// def update(change): global blocked_slider, robot x = change['new'] x = preprocess(x) y = model(x) # we apply the `softmax` function to normalize the output vector so it sums to 1 (which makes it a probability distribution) # y = F.softmax(y, dim=1) arrived = float(y.flatten()[0]) forward = float(y.flatten()[1]) left = float(y.flatten()[2]) right = float(y.flatten()[3]) direction = max(arrived, forward, left, right) if(direction == arrived) robot.stop() else if (direction == forward) robot.forward(0.4) else if (direction == left) robot.left(0.4) else robot.right(0.4) time.sleep(0.001) Will this plan works or any better suggestion? Hannah Zhang From: Zhang, Hannah (VWoA-ERL) Sent: Friday, May 24, 2019 11:45 AM To: 'NVIDIA-AI-IOT/jetbot' <reply@reply.github.com> Cc: 'Chitoku Yato' <cyato@nvidia.com> Subject: jetbot camera update problem Hi John, During running the live mode for collision avoidance, sometimes the camera only return one picture, so that the update function only be called once. When this problem happens, if the first picture is free, the car will go forward forever and if the first picture is blocked, the car will spinning forever. Since it happens 50% chance, I have to reboot all the time. Is there a way to make the camera more stable? Best Regards Hannah From: John [mailto:notifications@github.com] Sent: Wednesday, May 08, 2019 12:36 AM To: NVIDIA-AI-IOT/jetbot <jetbot@noreply.github.com<mailto:jetbot@noreply.github.com>> Cc: Zhang, Hannah (VWoA-ERL) <Hannah.Zhang@vw.com<mailto:Hannah.Zhang@vw.com>>; Mention <mention@noreply.github.com<mailto:mention@noreply.github.com>> Subject: Re: [NVIDIA-AI-IOT/jetbot] how to run a jetbot right after bootup? (#55) @HannahZhangVW<https://github.com/HannahZhangVW> Thanks for sharing! Very impressive! The road following code will be released soon, but we don’t have an exact date. If you’re eager to get started you can check out the notebooks in the road_following_ai branch. It’s under development, we still need clean the samples and add documentation. Please let me know if you have any questions or feedback. Best, John — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub<#55 (comment)>, or mute the thread<https://github.com/notifications/unsubscribe-auth/AL6PUEPF3DSOM65FS7M6MU3PUJ7FVANCNFSM4HJJH5UQ>.

[dvillevald]

Hi Hannah,

I tried a similar approach (a classifier with 4 classes - left, right, free, blocked) to teach JetBot to navigate through the maze of traffic cones and it worked pretty well. I also added FPV videos with superimposed classification and robot actions - it helped me a lot to debug. In case you are interested, the links are below (the code is on Github):

Youtube: https://youtu.be/YwxSwXEjd4c
Github: https://github.com/dvillevald/Finding-path-in-maze-of-traffic-cones
Hackster: https://www.hackster.io/dvillevald/transfer-learning-with-nvidia-jetbot-fun-with-cones-adf531

Hope it helps.
Good luck!
Dmitri