Joseph Masson's Notebook

Virtual Machine Resources

CPS Virtual Machine

CPS VM Resources

Table of Contents

• Spring 2024
• Fall 2024
• Spring 2025
• Virtual Machine - Spr2025
• Appendix

Spring-2024

Week 1 (1/14-1/20)

1/19/2024: To get up to speed with the ROS-HELICS sub-team, I worked on installing and running the ROS2 Docker Image at https://hub.docker.com/r/tiryoh/ros2-desktop-vnc. Since I am using a M2 Pro chip MacBook on ARM architecture, I tried to install a specific tag with these commands.

docker pull tiryoh/ros2-desktop-vnc:humble-20240114T0416

docker run -p 6080:80 --security-opt seccomp=unconfined --shm-size=512m tiryoh/ros2-desktop-vnc:humble-20240114T0416

Inside the Ubuntu server, I went to the ROS 2 Humble installation guide for Ubuntu (Debian packages) found here: https://docs.ros.org/en/humble/Installation/Ubuntu-Install-Debians.html

At the end of the demo, I was able to get the following talker/listener pair to run inside the Ubuntu image.

I'm hopeful that this is correct and will be able to verify when we start working in ROS2 for our project.

Week 2 (1/21 - 1/27)

1/23/2024: The team met Tuesday to prepare for our kickoff presentation. I was assigned to talk about our proposed solution of using line-inspection robots to collect data and make repairs to parts of power grids. I found these two resources from the sub-team helpful in understanding the project at this phase and used them to develop slides and collect graphics.

Fall 2023 Midterm Research Paper: https://drive.google.com/file/d/1Xx9eKL_qrykQch-r0lwdZmC3rKyrWP0P/view?usp=sharing

Fall 2023 Final Presentation: https://docs.google.com/presentation/d/1I_N87Nr7LH8UvbbR0zB2Erq3luHvZWur0h_-rJcflmg/edit?usp=sharing

1/25/2024: On Thursday, I presented the slide talking about our solution and the team discussed the steps we would take this semester to progress.

Week 3 (1/28 - 2/3)

1/30/2024: The team met Tuesday to discuss our plans for becoming trained in ROS2. We learned that our original planned online course is behind a paywall, so we looked and talked about alternative plans. We came up with several crash course videos to follow, as well as official ROS documentation in Humble to follow. I will be working on the ROS documentation first, then follow crash courses next week.

2/1/2024: Today was a work session in class, so I got started with the CLI tools documentation for Humble at this link: https://docs.ros.org/en/humble/Tutorials/Beginner-CLI-Tools.html

It looks like the Ubuntu image keeps the ROS environment active by saving the shells state - the first two commands to setup the environment did not work, but printing | grep -i ROS gives the correct results.

I read through additional documentation for choosing a ROS_DOMAIN_ID and settled on 0 for simplicity. I followed documentation to install TurtleSim and opened it successfully in Ubuntu.

Finally, I was able to install rqt to add GUI services and move and draw with one of the turtles, as well as spawn a second one in.

Week 4 (2/4 - 2/10)

2/6/2024 The team elected to send status messages by Discord and did not meet as normal due to a member's emergency. I took time today to continue through ROS2 documentation. I completed documentation on nodes, topics, and services, from the above Humble documentation set. Notably, I was able to find the data publishing rate, which may be useful in terms of simulations and time steps.

2/7/2024 I was able to continue documentation and covered ROS2 parameters.

2/8/2024 Today we presented during standup and I was able to get through the ROS2 actions tutorial with turtlebot, as well as run rqt_console to view warnings and messages from turtlebot's operations.

Week 5 (2/11-17)

2/13/2024 The group met shortly today virtually to discuss our midterm research paper and progress on tutorials. I finished the CLI tools tutorial with the sections on launching nodes to see how different files get launched with ROS2. I think it would be worthwhile to pursue the additional material on Launch Files, specifically the different ways these can be created:

https://docs.ros.org/en/humble/How-To-Guides/Launch-file-different-formats.html

Finally, I completed the CLI tools tutorials with ros2 bag and learned how to record and track data done with our TurtleBots. This means that I have finished part one of the team's documentation, and need to put in some work to catch up after I complete the busy week ahead.

2/16/2024 I was unable to come to the Thursday meeting time due to my exams in other classes. The next day, I took some time to catch up on the next part of the ROS2 documentation, Client Libraries. I started by creating building a workspace with colcon.

My Ubuntu system had trouble running colcon build --symlink-install, and I'm interested if others had this issue. I checked Docker when VNC froze and saw it using 1000/100% container usage, and the VNC would freeze intermittently as it tried to build. The entire build took 4 minutes and package examples_rclccpp_async_client failed while 9 others were aborted. As such, my tests and .bash setup did not work without that package.

Next, I created a package and learned file structures, including how to edit package information and descriptions. Notably, building this package took much less time and Ubuntu was seamless in processing colcon build.

I completed my work today with the C++ publishers and clients tutorial and successfully got a talker/listener functioning. I also skimmed the parallel Python tutorials.

Week 6 (2/18-24)

2/18/2024 The goal for today was to finish as much of the documentation as possible so I could focus on the research paper due this week. I was successful in building custom message files and creating the AddressBook interface. Finally, I reviewed the C++ code examples to see how to include parameters into our classes, which will undoubtedly be necessary as we create packages. I skimmed the final module about ros2doctor, and I'm going to link the suggested in-depth README.md in case I need to reference this in the future.

https://github.com/ros2/ros2cli/tree/humble/ros2doctor

I'm now up-to-date on the ROS2 documentation for now, and will be working on our research paper before we split into our subteams for the rest of the semester.

2/20/2024 The sub team met in person today to discuss details on our midterm research paper. I collected a variety of resources that I found pertaining to the robotic arm that we want to simulate in ROS2/Gazebo.

OpenManipulator-X - Arguably the high-end model, and I think pretty expensive. Also pretty bulky and probably hard to put on a TurtleBot, but this is what comes up first.

Robotic Arm Environment - This is an alternative but it doesn't look like a physical model. This does work with Gazebo and ROS2, but it runs on Foxy. I'm not super aware of how deprecated different ROS2 versions are, or how parallel they can be implemented, but that is an issue that stands out. This repo includes tutorials that might be a good starting point for our subsubteam.

Trossen Robotics Robot Arm Kit A middle-end solution in terms of price, but something that would be easy to integrate with a TurtleBot. I want to research its ROS2 stack a little more.

2/22/2024 During stand-up today, myself and other members of the sub team shared our issues with the colcon build command used to create packages with ROS2. We recreated the error and did some research into the error message. We learned that the issue was related to use of memory and swap storage by Docker to run the ROS2 virtual environment. We followed the following guide: https://stackoverflow.com/questions/44533319/how-to-assign-more-memory-to-docker-container

I upped my Docker's memory by a few gigabytes and increased the swap space. The next time we ran the command, Docker ran smoothly. The exception took a few seconds and completed successfully.

Week 7 (2/25-3/2)

2/27/2024 The team met today and confirmed components of our IPR presentation this week. I'll be building off the TurtleBot research we included in our paper, and looking into the arm we set out to include from the Kick-Off.

2/29/2024 We presented today and got really good feedback, especially concerning the arm I was looking into. I agree that it'll be better to have a very basic and responsive bot in the simulation first, and then we can build up some advancements for it. Now that we are working in sub-sub-teams, I need to do some work to learn my around Gazebo and ROS2's specific TurtleBot commands. I have a busy week in terms of exams, so next week I can start working on getting Gazebo in the VM and getting acquainted with the VIP's GitHub for collaborating on ROS2 code.

Week 8 (3/3-9)

3/5/2024 Back to doing technical work. I wanted to accomplish the goal of getting Gazebo running on the Ubuntu VM so that the team could have some documentation as well. It looked to me like the classic Gazebo (which we have screenshots for in our presentations) is unsupported, and there is a newer version optimized for Ubuntu development.

Quick Note - I tried installing the classic Gazebo, and its been delisted from Ubuntu repos and can't be installed per the original documentation.

I used this quick start in my Docker image: https://gazebosim.org/docs/harmonic/install_ubuntu

I had no issues, just ran the code in my Ubuntu terminal. I moved on to the succeeding Getting Started guide (https://gazebosim.org/docs/all/getstarted). With that, I was able to get their basic simulator running.

I wanted to get a TurtleBot model into a base simulation. I started by executing gz sim in my Terminal. This pulls up a simulation quick start menu. I chose the "Empty" preset, in the bottom right corner. Pressing the x in the top right to close the window also opens an empty sim. You can also use gz sim empty.sdf.

I'm noting the file path for gz files, which is where assets are stored: /usr/share/gz/(sim). It's also available in the right sidebar.

The Getting Started guide provides a model collection website for SDF assets to add to your simulation. They have a TurtleBot one (https://app.gazebosim.org/OpenRobotics/fuel/models/TurtleBot). This guide was really helpful (https://gazebosim.org/docs/harmonic/fuel_insert), and showed me how to find these resources in Gazebo and add them directly. I got some errors trying to add it in.

This is an error with the model, and probably means we'll need to download and modify its code to add some joints to satisfy the message.

I did notice my Mac heating up a bit running the simulation, here's my Docker data.

It plummeted after shutting down Gazebo.

Week 9 (3/10-16)

3/14/2024 Met with entire team and talked about my Gazebo work in standup. Looked around at some prebuilt sim rooms, but nothing really stood out. Going to keep looking through .sdf file structure next week.

Week 10 (3/24-30)

3/26/2024 Met with the team earlier today to discuss our progress. Started reviewing the documentation from Sabina about our HELICS/ROS integration. I also found some documentation on TurtleBot3 that might be useful for troubleshooting across different ROS versions: https://www.google.com/search?client=safari&rls=en&q=turtlebot+3+pc&ie=UTF-8&oe=UTF-8

Week 11 (3/31-4/6)

4/1/2024 I continued reviewing the TurtleBot3 docs I had found and started working on finding a good .sdf for a Turtlebot, although I think that if we are moving forward as quickly as we are with HELICS integration, the priority of the sub team might need to shift to get ROS2/Gazebo integrated.

I met with the team today to focus on the HELICS integration. Sabina helped me debug some stuff from her guide on GitHub (see her notebook) and I was able to get everything done right, but I am still not getting the right outputs. I think my HELICS is not rolled back like hers might be in the notebook.

4/2/2024

During the team meeting, Sabina clarified a fix I was missing to rebuild the packages and I was able to get the HELICS integration running on my VM!

Week 12 (4/7-13)

4/7/2024

I am going to get started on linking a ROS2 simulation to Gazebo. It looks like I need to avoid any Gazebo classic documentation, like I've mentioned above. The new Gazebo documentation is very barebones, so here is what I have to work with: https://gazebosim.org/docs/harmonic/ros_installation, and then move onto https://gazebosim.org/docs/harmonic/ros2_integration hopefully.

Luckily, Gazebo recommends using ROS2 Humble with Gazebo Fortress, so we picked a good ROS2 release. I started working on learning gz-bridge, which is how ROS2 and Gazebo interoperate. There is a keypress tutorial that I haven't been able to get working, so I will follow up on the video for it soon.

This also looks really viable - it is in the new Gazebo (you can tell because its bright and pretty and not gray and depressing): https://docs.ros.org/en/humble/Tutorials/Advanced/Simulators/Gazebo/Gazebo.html

Week 13 (4/14-4/20)

4/15/2024

I was finishing up some Gazebo tutorials today, and I am seriously hitting roadblocks on performance. Gazebo through Docker/Ubuntu struggles to load and render files. I've found some SDFs that could relate to an electrical grid physical sim but I can't get it to open on my instance. Gazebo is also clunky at all times and manipulating small elements in an empty simulation is difficult. It will take a lot of practice (and maybe a performance fix) to make building a robot model or electrical grid possible.

We previously raised Docker's usage limits for the colcon tutorials, so I'm not sure how much more power I can give to the container. I will note this in our presentation this week and we can look at solutions next semester.

4/16/2024 Met with the team to complete our presentation slideshow and prepare for tomorrow. I worked on the slides about our physical environment progress.

4/18/2024 Presented today in the meeting. Promising progress this semester but still issues with the Docker that need to be addressed. Shifting focus to the paper.

Week 14 (4/21-27)

4/23/2024 Wrote my subsection on our Gazebo work into our paper and left spaces for Suraj to discuss our setbacks. I found this on Gazebo's site, which explains the version history really well. I incorporated some of this to my sections.

From: https://gazebosim.org/about

4/24/2024 Team meeting today to discuss the paper. Probably going to find some images to update the old ones of Gazebo Classic in our appendices.

Fall-2024

Week 1 (8/22)

8/22/2024 Syllabus week - this was the introductory meeting, so I didn't complete too much work.

Week 2 (8/29)

8/28/2024 During this meeting, we decided on an outside meeting time and talked specifics about our frameworks for the semester.

9/2/2024 I went back and dug up our Final and Kick-Off presentations from the Google Drive and shared them with the sub team for ease. We should make a decision soon about what template to reuse/if we need a new one and then start working on it soon. Since our old Kick-Off uses old Gazebo screenshots, I will probably boot up my VM instance sometime this week and get some newer screenshots, which will give me a chance to get an estimate on any performance changes from the summer (probably none).

Week 3 (9/5)

9/4/2024 Finishing up my slides, needed to make some changes to make it broader/future-focused. I booted up GZ on the VM and got a few things of interest. I found a pretty interesting power grid SDF, as well as some utility poles I could link up to make more interesting.

First, found some values for altering magnetic fields, which is our big push on the whole sub team this semester. I'll need to evaluate how I can change/detect those.

Second, performance looked way more stable.

Big spike when booting up, big spike when I inserted the power plant to an empty sim world, and then flatlines while I drag around. I can't really track down a changelog, so I also need to verify 1) I have an up-to-date release and 2) Make a guide to install/update with ROS2. I'll probably work with a returner to build that for our new members.

9/5/2024 Today we are presenting our kick-off presentation. I reworked the existing Gazebo/physical environment slides that Suraj and I made last semester and then updated some goals and text, as well as including the new photos.

Obviously, we'll hear feedback after our presentation and then I'll make some tweaks to my goals for the semester.

9/8/2024 We got great feedback after our presentation, I'm sure more will come when the instructors upload the written feedback. Before the team meets tomorrow (if we aren't interrupted by career fair), I'm checking to make sure Gazebo is up-to-date and installed correctly on my machine.

Just running the install command sudo apt-get install ros-humble-ros-gz seems to show that I am up-to-date, although there must be a couple months of open-source contributions I'm missing based on the GitHub pull requests for gz-sim. The documentation points to an OSR dependency as well, but I'm not sure if that's what I need.

Important note: Gazebo Ionic (Gazebo v9.0.0) is expected to release this month!

I'm going to leave this here because I know this was discussed earlier this semester: https://github.com/gazebosim/gz-sim/pull/2081. This is the Python support available in gz-transport.

Week 4 (9/12)

9/9/2024 Good meeting in standup today. We have a few tasks on Physical Modeling. First, I'm looking for existing power grid simulations.

This is a video of a program that does a similar proof-of-concept with flying inspections. https://www.youtube.com/watch?v=a7_lW9RJ-og They have some nice power lines, but nothing too fancy.

This paper also deals with modeling substations (w/ LIDAR) and https://www.researchgate.net/figure/Gazebo-simulation-substation-model-are-representative-equipment-for-evaluating-the_fig8_366076729 is a good representation of what we want.

This is a substantial thread that talks about ROS magnetic sensors from 2014: https://github.com/gazebosim/gazebo-classic/issues/1294. This reply (https://github.com/gazebosim/gazebo-classic/issues/1294#issuecomment-616051697) points to using hector_gazebo_plugins (http://wiki.ros.org/hector_gazebo_plugins#GazeboRosMagnetic) and its magnetic topic for a bot to sense something, hopefully in Gazebo.

Good news! I am finally able to open the electrical substation file from last semester after fixing my setup - I'm going to include some file tree structure and photos for that!

I needed to create a folder in my home directory called my-local-models. In that folder, add materials and meshes folders. Include the substation file Gazebo Fuel.

Run gz sim electrical_substation_v3.sdf and give it a minute for the runtime warnings to clear.

9/10/2024 I've been asked to look into ROS2 physical sims alternatives: it seems like the only one worth noting is called WeBots, and it's a lot less than Gazebo from what I can tell. I'm willing to be because Gazebo is more open-source and has more contributions, that's why I can find so many different resources and models to play with. WeBots doesn't seem to have any examples close to our problem space.

9/12/2024 I did some research after the workday today to figure out how I can apply a magnetic field to a specific piece of the model, like our simulation will require. I've altered the substation's SDF to add a base magnetic field to the world, but changing an individual wire or part to have a field will be more complex. I'll need to try to build a custom model plugin that takes in the model and applies a physics force somehow. The other alternative seems to be faking magnetic field with <force>, since our purposes don't require the magnets to actually push and pull, we just need to show detection.

Week 5 (9/19)

9/16/2024 We had our sub team meeting today. I presented my substation file and above research on ROS2 physical simulations. It looks like we will be continuing with Gazebo - I'm not complaining. A common idea is building a unique substation for the team using the models provided in the substation file, so my goal is to integrate Git/our GitHub into the Virtual Machine, along with get my sub-sub team partner on the Docker image as well.

I created a repo in our organization (https://github.com/cps-vip/gazebo-physical-env) and pushed the SDF I've been doing exploration on to it. I moved the my_local_models file tree inside the cloned repo to maintain my working flow, and got the repo integrated to the VM's VSCode. Pretty easy to authenticate and push. I'm going to work on sharing the VM, installation guides, and repo to my teammates tomorrow!

9/17/2024 I built a quick setup dictionary for my team members. It is a very wordy guide, so I am putting it at the bottom of this notebook in an Appendix section. I'm sharing it via Discord to the ROS/HELICS team as well.

9/19/2024 I've started making a Gazebo plugin. I've built a C++ file that should add a force to models and a CMakeLists.txt that allows it to compile on my local machine. I'm having issues getting it to run on the VM, which I blame on Git issues inside the VSCodium app. I am going to try again later, maybe by recloning/PRing the code in the repo. The VM might also be missing cmake for some reason.

Here's a link to the code inside my Magnetic branch, I did my best to comment the plugin itself! https://github.com/cps-vip/gazebo-physical-env/blob/magnetic/my_local_models/MagneticField.cpp

ROS Jazzy Release notes: https://docs.ros.org/en/rolling/Releases/Release-Jazzy-Jalisco.html#changes-to-how-ros-2-and-gazebo-integrate

Week 6 (9/26)

9/24/2024 Had to miss this week's standup for a conflicting meeting. I believe we are just coordinating within sub-sub-teams, so hopefully my teammate is getting everything installed and is maybe starting ROS2 tutorials. We can touch base during meetings Thursday.

I spent some time today looking at example GZ/ROS plugins in this repository (https://github.com/ros-simulation/gazebo_ros_pkgs/tree/noetic-devel/gazebo_plugins/include/gazebo_plugins) to make sure my Magnetic Field plugin was doing everything right. I'm interested in why they use a .h file type, and I had some confusion in what to use before I decided on .cpp for C++. I might look into this issue more, but my next step is to spend time in the VM compiling everything.

9/26/2024 Today is a virtual meeting for the VIP due to the hurricane. I continued looking at my MagneticField and made a PR for what I've done so far. It should be functional so I'll spend time testing it after the paper draft is done, as well as get started building a proprietary SDF file.

Here's the PR with changes: Magnetic Field PR

9/27/2024 I'm spending time today working on our Paper draft. My contributions will be to the Physical Environment section, of course. Since our research is much more iterative and is a little more established compared to some other subsubteams, I'll work off of the old paper drafts and add the progress and goals we've developed this Fall.

Week 7 (10/3)

9/30/2024 Completed work on my section of the paper today. Discussed the new progress made - opening the substation SDF, creating a new GitHub repo, and building Gazebo plugins from scratch for sensing. I've included older details like the Docker issues and the switch to Ignition. Waiting for it to get reviewed by the team and I can see what else to write for the conclusion.

10/2/2024 Completed the paper today. Did some revisions and reviewed the later sections about the team's future steps as whole. I think the goal for the midterm presentation next week is to have a quick demo of a personalized SDF and be able to show the plugins I've been building. I'll plan on working on the SDF tomorrow.

10/4/2024 Working on a few tasks for the midterm presentation. I'm sending the slide deck to the sub team later today, and I started looking at individual models in the V3 Substation SDF to start building my own SDF. I copied a few wires and array pieces over and messed around with them.

I didn't really realize how strict those wires are, you can see they are designed to fit in the V3 configuration entirely. Solutions I can think of are using thin cylinders to model as wires or just cutting extra stuff out of V3 to use a trimmed version that is less performance-intensive.

Week 8 (10/10)

10/7/2024 Today was the standup meeting - not a lot of people came and there wasn't that much to discuss. We just confirmed the direction our midterm presentation would follow from our paper draft, and I set up the slideshow and shared it with everyone.

10/8/2024 Small workday today - I updated the Physical Simulation slides to better reflect the progress made this semester. Less bullet points, more technical findings!

10/10/2024 Today was the midterm presentation.

Week 9 (10/17)

10/17/2024 Limited work this week thanks to Fall Break. We met virtually for class today and discussed our mission trajectories. This was the biggest feedback we got from the midterm presentation and it will help us create a Proof-of-concept and tie all of our work together. I'm attaching the rough outline from our Standup notes below.

This means I am probably pausing my existing work on forces/SDFs to support this task for the team. No problem!

The physical sub team will be focusing on creating the physical simulation (TurtleBot and electrical part/cube) and probably handling the movement of the robot with ros_gz_bridge.

10/20/2024 I've started using the guide in the below link to connect a ROS2 publisher to Gazebo. The first step is to get ROS keystrokes to be shown in Gazebo, but we will need to mock that command for the command-control to publish is my guess. Sabina probably already has that figured out. https://gazebosim.org/docs/latest/ros2_integration/

Week 10 (10/24)

10/21/2024 During the Standup meeting today, we onboarded into a subteam Jira for task and story management to help us focus on our mission and understand where we each are at in progress. I also continued working on teleops for ROS2 and hope to have the sim working after class Thursday.

10/24/2024 Moved my Jira task to In Progress, I've started testing publishers for Ros-GZ now. Nothing concrete but I am hoping to finish by the next standup on Monday.

https://planformteach.atlassian.net/jira/core/projects/PM/board?selectedIssue=PM-3 (maybe visible?)

Here's the subtasks for this activity for me and Shrayes to complete.

• Use ros-gz-bridge to teleoperate a robot in Gazebo (Done 11/14)
  • Confirm ros-gz-bridge will connect on the VM (Done 11/6)
• Connect the key publisher to the command-control structure
• Add a fault object and publish an SDF to the team

10/27/2024 No big updates, Shrayes is replacing his laptop and redoing all of the installations, so I'm going to wait until Monday to touch base and finish the teleops.

10/29/2024 I'm struggling a little with the tutorial - I can set up the three terminals and Gazebo but no data comes through the window, and the video explanation is out of date. I've found another guide: https://gazebosim.org/docs/latest/moving_robot/ that is closer when you use the ackerman_steering SDF from gz sim menu. Bridge might not be the best option and I might need to switch to teleop package: https://answers.ros.org/question/315020/how-to-get-turtlebot-to-move-in-gazebo/

Week 11 (10/31)

10/28/2024 Had a brief standup today to check in on progress.

10/31/2024 I was able to crack it today, and get data from keypresses made in gazebo to publish to the /keyboard/keypress topic. I'm not sure what fixed it - I ran the sim using ros_gz_sim command: ros2 launch ros_gz_sim gz_sim.launch.py gz_args:=empty.sdf. I also pressed different keys than the arrows that normally work, it looks like w-a-s-d might be the teleoperation keys here.

Week 12 (11/07)

11/4/2024 Starting to look into inputting a robot and connecting it to teleoperation in Gazebo. The tutorial I've been using has some instructions, but I also want to look into using Fuel to pick a robot. Reported in to our stand up meeting.

11/6/2024 Kept experimenting with adding a vehicle into Gazebo - that's the easy part with the Fuel capabilities of Harmonic. Connecting it is the difficult part, and may involve manipulating the SDF to include the bridge. I've been incredibly busy and focused on other things, so planning on resetting and putting a lot of time into cracking this soon to make sure I'm ready for our mission.

11/7/2024 Met today in class, shared out my progress. I helped my teammate initialize the bridge based off of my notes. Working on adding the robot to keystrokes.

Week 13 (11/14)

11/10/2024 I'm finishing the Gazebo integration for command and control. Now that I have the correct ros-gz-bridge setup, it looks like I need to start the dedicated SDF and write a plugin for movement. Using this tutorial SDF file - https://github.com/gazebosim/docs/blob/master/ionic/tutorials/moving_robot/moving_robot.sdf I have the code for this tutorial: https://gazebosim.org/docs/latest/moving_robot/

This gives us keyboard and topic subscription for a robot in Gazebo, so hopefully the CnC can publish the topics necessary. I'll test this later this week.

11/11/2024 Brief standup meeting today. Helped Natalie debug her Gazebo issues and shared my progress with the topics. Planning to have a result for lecture on Thursday, still need to finish testing on the VM.

11/13/2024 Finishing testing moving the robot today. Using moving_robot.sdf, and running gz sim moving_robot.sdf in the directory, I get the basic robot into Gazebo.

In another terminal, running gz topic -e -t /keyboard/keypress gets a listener to keyboard presses within Gazebo (also make sure to open Key Publisher in Gazebo UI). Below, you can see the data outputted by pressing the arrow keys.

Now, after starting the simulation (press Play!), we can alter the velocity of the robot and move it omnidirectionally on the x and z axis!

A video of the simulation should be visible here: https://drive.google.com/file/d/1YpJUVqCwUDatwyrVkCry_w8lNM7nrZCU/view?usp=sharing

With this, we can move the robot with keystrokes. What I think is most relevant is Gazebo interprets a keystroke and publishes out an integer - so if our CnC publishes out the same integers, we should be able to move the robot?

Here is the current movement:

Left ➞ 16777234 ➞ linear: {x: 0.0}, angular: {z: 0.5} Up ➞ 16777235 ➞ linear: {x: 0.5}, angular: {z: 0.0} Right ➞ 16777236 ➞ linear: {x: 0.0}, angular: {z: -0.5} Down ➞ 16777237 ➞ linear: {x: -0.5}, angular: {z: 0.0}

11/14/2024 Shared out my findings with the team today. Sabina is going to work on building a topic that publishes the up integer variable to the robot, I hope we can just reconfigure its plugin to receive from this topic and it will publish to the Gazebo sim.

Week 14 (11/21)

11/18/2024 The sub team met on Discord for standup. I created a new set of slides for our final presentation this week and shared them with the group. I'll be working on my slides tomorrow.

11/19/2024 Not much to report. Completed my slides for review tonight.

11/21/2024 We presented today. Future presentations will definitely need some revision and be less bloated. Think we shared our technical progress really well though! Need to work on the paper and see if we will get everything connected before the end of the semester.

Week 15 (11/28 - Thanksgiving)

11/25/2024 No standup for break week. I posted my code and tutorial to my physical environment repo: https://github.com/cps-vip/gazebo-physical-env/commit/860bc05a293c9fc1b79ec7666eeb5096e015f02f

Week 16 (12/04)

Spending this week working on finalizing the paper. As always, Shrayes and I will complete the section about physical environments. We can write a big update about our research to keybindings and add it in. We should also edit our section to be more concise and flow better since we switched focus since the midterm paper was submitted.

After it's submitted, we're done for this semester and we'll be really close to a huge breakthrough in January!

Spring-2025

Week 1 (1/9)

Third semester with the VIP. I have a class conflict with my networking lecture this semester, so I will be missing most of the in-person meetings at the lecture time. I do think I can get out of enough to make it to the presentations.

Week 2 (1/16)

1/16/2025 When the team met in class, I filled out the meeting time survey. I also got pinged that a new Gazebo version has been released, which I would like to get a look through. I've had trouble organizing and understanding all of the Gazebo releases (because the community is insane and releases a new version every year somehow), and I want to get a better understanding of how it works for future team members.

1/17/2025 We had a virtual meeting this morning. We discussed our sub teams and any shifts they would be making, as well as got a feel for where the new members want to work. We also laid out a couple of tasks for our Kickoff presentation next week. I'll talk with Shrayes to get our next steps and work on trimming the presentation from last year's final. I'll also mention our new task of understanding Gazebo lifecycle and documentation changes.

1/19/2025 Some brief notes from my skimming of the Ionic release, which I am finally catching up on. I added some work to the kickoff presentation - combining the Gazebo slides for clarity and adding some rough next steps we could decide to work on this semester.

Ionic Release Notes: Investigate new gazebo releases for longevity Mentioned a new NAV2 demo to share to Natalie New GZ documentation? https://community.gazebosim.org/t/gazebo-ionic-release/3088 Highlight - improvements to ros_gz

Week 3 (1/23)

1/21/2025 Spent time today updating the Kick-off slides. Worked on updating team information and condensing down extra information that wasn't necessary. I want to trim the starting slides too - we've used the same 8 for a couple of presentations and people get what we are going for :)

1/22/2025 Finalized the presentation tonight before Safiy could record his parts. Feel good about the overall flow compared to previous presentations.

1/23/2025 Presentation day! Think we did a great job. Got some feedback for future presentations, and the instructors are excited to see the demos we will be executing this semester.

1/24/2025 Subteam meeting over Discord this morning. Below is the screenshot of our tasks for now.

I'll be handling finishing off Mission 1's Gazebo integration with Safiy and Shrayes will handle the onboarding for now. Once we have a new Jira, we can digitize this as well. We also want to start switching over to our standard containers, so I think I'll take time next week to go ahead and start with that so I can test the NAV2 demo for Natalie.

Virtual-Machine-Spr2025

Week 4 (1/30)

1/27/2025 My first goal is to get the new VM working on my machine from Sabina.

I located the VM container in the Packages tab on the cps-vip GitHub. I used these instructions in my terminal to login/link Docker to my GitHub: https://docs.github.com/en/packages/working-with-a-github-packages-registry/working-with-the-container-registry. After setting up the PAT and executing the login command, I was able to run the docker pull command. I needed to remove some stuff from Docker for storage but got it to run fully after three tries.

I ran the image and bound it to port 6080, like the old VMs. I got it to boot in browser and it had ROS2 installed. I should look at updating it with Gazebo installations! I'll talk to them about that later.

1/28/2025 Followed my own documentation to get Gazebo and ROS-GZ bridge running on this new VM. Difficulty getting sudo to respond on the VM - permissions are not set right. I tried to fix with chown root:root /usr/bin/sudo && chmod 4755 /usr/bin/sudo in the Docker app - Exec tab in the folder. This just gets more problems and I've probably modified the sudo file the wrong way and will need to reinstall. I need to ask Safiy and Sabina to make sure this is part of the Dockerfile as well, or else we can't install Gazebo right on the VM.

2/2/2025 I tried to fix the issue with this link https://github.com/nextcloud/docker/issues/1037 from Sabina. Unfortunately, no dice. I think I will need to ask her for the Dockerfile, and then I can add a RUN command to get Gazebo for a new VM too.

2/4/2025 I used the Dedockify tool from Sabina to get the Dockerfile for the packages. I've pushed it and an attempt to simplify it to my physical environment repo (https://github.com/cps-vip/gazebo-physical-env/commit/56047a4713e1e2c5b056ab73b7144c531250abfb). I think there are more files I need to build it because I get an error without a file in the directory. I really need Safiy to respond for this or my messages about Mission 1 to continue I think.

Week 5 (2/6)

2/6/2025 My class conflict was cancelled today so I was able to go to the in-person workday. Shared my updates with the instructor and helped a new student with their VM. Talked with Natalie a bit about NAV2 demos and her bugs. Still waiting for a response from Safiy.

2/7/2025 We had our standup meeting this morning online. Kind of brief since not a lot of progress has been made. I discussed my VM issues with Safiy and afterwards he was able to upload the directory he built from to our GitHub: https://github.com/cps-vip/helics-dockerfiles

Here's my task list for the next week/week and a half:

• Use Safiy's files to build the Docker container on my own Complete 2/10
• Change how the container assigns account to create a root user that can use sudo Complete 2/10
• Include a Gazebo Fortress installation to the container Complete 2/13
• Publish new container to GHCR (will need Safiy's notes on this) Complete 2/14

Milestone 2 (Updated Milestone Tasks in Later Entry):

• Iterate on container and add NAV2 installations
• Republish to GHCR

Going to start taking a closer look Monday!

Week 6 (2/13)

2/10/2025 I cloned Safiy's repo and ran docker build on my own system. Unsurprisingly, it takes a long time to download all the different files (Ubuntu, HELICS) and build them. Safiy's Dockerfile is notably different from what DeDockify spat out for Sabina's, either her Dockerfile was really different or that tool did not do well with our containers. I also needed to a little work to clean my Docker storage - it was clogged and stopped the build from working. Total build time was 15 minutes.

When I tried to run the image through the GUI, it did not automatically let me select a port. I solved this with docker run -p 6080:80 container-name and it popped up in the container screen. This version also has sudo working correctly! I'll start reading documentation to ship a container with Gazebo.

2/11/2025 Success! I added the binary installation code at the end of the Dockerfile. It compiles and builds on my device and works on the port. You can test with gz sim empty.sdf.

Here is my PR with code changes: https://github.com/cps-vip/helics-dockerfiles/pull/1. I'll continue soon by adding it to the container registry soon.

2/13/2025 We met today in lecture and had two guest speakers!

2/14/2025 Brief standup meeting this morning since a couple of people were absent. We talked about our updates and discussed paper logistics. I shared my progress on the Silicon VM. Safiy doesn't have any notes for pushing the VM to GHCR, but I think it's simple. I saw the documentation on GitHub when I was pulling it initially. Since I've authenticated in Docker to the cps-vip organization, I hopefully can push outright.

I was right, it's just a new build and push command. I've run it for a first try and I'm waiting for it to push this 10gb container.

Update: success! I accidentally pushed to my personal git, but I changed the repo to the organization.

Week 7 (2/20)

2/16/2025 I added a new goal for this container work for the team - I'll try and get a working NAV2 demo on it too. I know we've been struggling with getting that to run on these containers, so if we can solve it, we should have the entire ROS sub team tech stack on one machine.

2/17/2025 We started working on our midterm paper this week. I updated the sections about the containers with my data and progress on that front.

2/19/2025 To help the team, I went and condensed/removed unneeded parts of the paper. Like our presentations, we have a lot of dead weight from semesters past. I clean up a lot of Section 2 (past research/case studies/etc) and our current progress areas.

2/21/2025 Subteam Meeting today. We discussed progress and started assigning tasks for our presentation this week. Shared successes in the Docker and getting ready to transition to NAV2 installations. My plan is very similar to how I got Gazebo on board - figure out how to get a stable installation within the container - then alter the image's Dockerfile. We also discussed the topic setup for fault detection work.

2/22/2025 To help the team understand my work, I reorganized the physical environment repo. I was just dumping files and some had bugs that needed to be patched. Here's the commit: https://github.com/cps-vip/gazebo-physical-env/commit/545b80034d7767529a7d237ac4b3d8cf054236c4 Going to look at the presentation next time.

Week 8 (2/27)

2/23/2025 Created a slide for my physical environment updates in our slideshow.

2/25/2025 Did a review of the slide deck to make sure we didn't have any extra information or errors. I will circle back before we submit to resolve all of these issues!

2/27/2025 We presented first today. I helped Suraj debug some issues now that he's recovering from his injury. The presentation went well, we got good feedback on our slide order (Mission 1 slide needs to be moved). Will also gave notes on some of the new stuff we are pursuing, like SROS and NAV2 sensors.

2/28/2025 Had to miss subteam meeting this morning. No real updates since we've been working on report/presentation for past two weeks. Plan is to resume with NAV2 documentation next week.

Week 9 (3/6)

Milestone 2:

• Iterate on container and add NAV2 installations Complete 4/2
  • Update container to Jazzy Complete 3/31
  • Add NAV2 installation to container Complete 4/2
• Republish to GHCR Complete 4/2
• Write installation guide in Notebook with commands and resources (don't forget GHCR auth) Complete 4/3 - see package page/README
• Open issue for HELICS command failure in building image Complete 4/2
• Resolve issue with HELICS command failure Complete 4/7

3/4/2025 I started with Natalie's notebook to find the correct NAV2 information. Here is the documentation for installation. https://docs.nav2.org/getting_started/index.html#installation

The commands I got to work were:

sudo apt install ros-humble-navigation2
sudo apt install ros-humble-nav2-bringup
Sudo apt install ros-humble-turtlebot3*

The last command should be sudo apt install ros-humble-turtlebot3-gazebo, but I received a "package not found" error in terminal when trying to run it. I did some research (https://robotics.stackexchange.com/questions/103620/cannot-install-ros-humble-turtlebot3-gazebo-package, https://roboticsbackend.com/ros2-nav2-tutorial/) and came up with the new command. After running these, there are still missing pieces? I'm going to see if they are necessary by testing the demo next time.

3/5/2025 I used a new command to resolve the issues from above. The new installs work fine after adding this new package. See: https://www.reddit.com/r/ROS/comments/16wc472/unable_to_install_gazeboros_and_its_plugins_on/. However, I have a new error where it cannot find a gazebo_ros package. I don't think Natalie got this error.

sudo apt-get install ros-humble-ros-gz
sudo apt install ros-humble-navigation2
sudo apt install ros-humble-nav2-bringup
Sudo apt install ros-humble-turtlebot3*

3/7/2025 Sub team meeting was cancelled for low attendance today. Spent more time diagnosing my error on GitHub issues and StackOverflow. No luck. My plan is to reset my VM and start fresh with the new commands I have and go from there!

3/9/2025 Reset VM and rebuilt to 6080:80.

Week 10 (3/13)

3/11/2025 - Resetting yielded the same error, I asked Natalie if she had seen it at all.

3/12/2025 Natalie got back to me overnight, but she's never seen the error. I'm still doing research, but this lead seems promising:

I think the issue is the combination of ROS2 Humble and Gazebo Harmonic is not easily compatible and there is a package issue. I can't understand why, since Natalie got it to run on Harmonic. I downgraded Gazebo to Fortress (https://github.com/josephmasson26/helics-dockerfiles/commit/daf33410c2074f1aa1532b2b8569aa0c7055ed0e). When I ran the commands, the error persisted. I'm going to revert back to Harmonic now then.

3/13/2025 Still zero idea what I'm missing. Natalie and I set up Gazebo the same way so I have no clue why random commands work differently. I'm going to look and see if I can clear up the compatibility issues after Spring Break. This is difficult to do on one computer (maybe I can upgrade the VM with real copy/paste functionality :| )

3/14/2025 - Meeting cancelled for break.

Week 11 - Spring Break

3/17/2025 Tried to do some more poking around but I needed to rebuild the container again and of course Docker just created some random error to stop my progress. It won't run apt-get in the Dockerfile, which I'm pretty sure is because the cache is full or whatever. I've had to prune the Docker resources before, so I'll put a list of commands I use to reclaim space here and in the Dockerfile.

# docker image prune
# docker builder prune
# docker container prune

To check resource usage: docker system df

The work I did today revolves around trying to install from source and using colcon to build NAV2. I ended up in a rabbit hole of needing to colcon ros2 and gazebo just to find these missing packages, which of course don't exist there anyway. I'm starting to think the issue is going to be with our ROS2 installation. I'm going to try later to upgrade the Docker image to Jazzy - this version matches Harmonic best and will hopefully play nicer.

3/18/2025 Updated to Jazzy by testing a new Docker build. Switched over the ROS install, the base tiryoh Docker image, and had to change a setting on the HELICS building with an extra flag. The Jazzy base of course disables pip because none of this can be simple. I need to figure out how to VENV the pip commands.

Week 12 (3/27)

3/24/2025 Natalie and I talked more about the bugs I've been having. She mentioned Nguyen has gotten NAV2 to work. I think the issue might be that I'm using a Mac? Based on my understanding of VMs that shouldn't be an issue, but I know I used a different base image of the ROS2 machine earlier in the VIP. Something I might try is having Natalie run the new VM for me and see if she can get it to work.

3/25/2025 Nguyen's notebook isn't up-to-date yet. Going to wait for that documentation for another two days and then I'll DM for his notes.

3/28/2025 Was out of town for the weekend, so I was unable to attend the sub team meeting. I shared my progress with the team on Discord.

3/30/2025 Nguyen's notebook is still not up-to-date so I just sent him a few questions on his system/version/notes about installing NAV2. I'll work with his responses for a better direction this next week.

3/30/2025 Nguyen confirmed with me that they were able to follow the basic setup instructions from NAV2 documentation on a Jazzy machine. After conferring with Natalie, I'm going to go back to trying to upgrade the VM to Jazzy, then I can add NAV2 on top of it. Just because of the compatibility errors, I think this is the next best step.

Week 13 (4/3)

3/31/2025 Using all of my skills, I was able to update the Docker VM to ROS2 Jazzy. The base Ubuntu image changes between Tiryoh's Humble and Jazzy VMs, so there were a couple of compatibility issues to change around the board. Updating to Jazzy was easy - all of the commands are the same after you change 'humble' to 'jazzy'.

The challenge was making the C/HELICS stuff carry over. Ubuntu 22.04 disallows using RUN pip, so you have to spin up a virtual environment to install packages with that. Even with that change, one command was not cooperating. I've commented it out for now, since it doesn't relate to my overall goals, but I'll need to open an issue to fix it later.

Here's the commit with my changes: https://github.com/cps-vip/helics-dockerfiles/commit/bed1ec1040291c19e1f121f9be1cdfca434d7804

Here's a better commit with more comments and organization: https://github.com/cps-vip/helics-dockerfiles/commit/4e518f88ab5f590a9196a9b4a847474213ea14db

4/1/2025 Today's target was to successfully install NAV2 on the Jazzy machine. It worked like a charm. Compared to the compatibility issues I had in Humble, Jazzy went exactly as it should have.

I followed the instructions here: https://docs.nav2.org/getting_started/index.html By using the commands for Jazzy (see below), I was able to get everything to install correctly. I got the demo to come up with Gazebo Harmonic and RViz:

Here are the commands I ran:

sudo apt install ros-jazzy-navigation2
sudo apt install ros-jazzy-nav2-bringup
sudo apt install ros-jazzy-nav2-minimal-tb* #Doesn't do anything
source /opt/ros/jazzy/setup.bash
export GAZEBO_MODEL_PATH=$GAZEBO_MODEL_PATH:/opt/ros/jazzy/share/turtlebot3_gazebo/models #Possibly not needed

To run demo: ros2 launch nav2_bringup tb3_simulation_launch.py headless:=False

Next step: adding commands to Dockerfile!

4/2/2025 Today I added the necessary installation and sourcing commands to the Dockerfile.

Here's the commit with the additions: https://github.com/cps-vip/helics-dockerfiles/commit/d624760d5c4b25d3edc3a4e3b4999433c4e36d76

Here's the PR with the Jazzy/NAV2 changes: https://github.com/cps-vip/helics-dockerfiles/pull/3

Since there is an existing Humble-Harmonic container available on GHCR here: https://github.com/orgs/cps-vip/packages/container/package/helics-ros2-gz, I feel okay overwriting the repo with the up-to-date version.

The package is now available in the Packages page of the organization! (https://github.com/orgs/cps-vip/packages/container/package/ros2-jazzy-gz-harmonic-nav2)

Pull with docker pull ghcr.io/cps-vip/ros2-jazzy-gz-harmonic-nav2:latest Run with docker run -p 6080:80 ros2-jazzy-gz-harmonic-nav2

4/3/2025 On a roll. I started investigating the HELICS installation errors. I think it has to do with needing to have C++ 20, which means we need to be on HELICS 3.6.0. I used this page (https://github.com/GMLC-TDC/HELICS/releases) to figure it out. I tested by bumping the Dockerfile to use HELICS 3.6.1 (newest release), and it worked perfectly. I messaged Sabina to make sure upgrading it was okay, if it works, I will repush to GHCR.

Here is the issue with the errors: https://github.com/cps-vip/helics-dockerfiles/issues/4 Here is the PR resolving it: https://github.com/cps-vip/helics-dockerfiles/pull/6

I also went ahead and updated the README for the repo with installation instructions. When it is back on GHCR I will see if I can put that on the container page as well.

Repushing to GHCR. I can see the next step will be to test ROS2-HELICS-GLD integration.

Went in to lecture today - shared my progress. Worked more on getting the HELICS-GLD demo to function, just missing an import and had to move stuff out of the read-only directory.

4/4/2025 Quick update - shared progress at sub team meeting on Discord. Linked the repository we've been adjusting the VM to the package page, so documentation is now visible. The repo README.MD syncs to the package page here: https://github.com/cps-vip/helics-dockerfiles/pkgs/container/cps-vip-vm. Going to open a new branch to look at the HELICS-GLD syncing.

Week 14 (4/10)

4/7/2025 I isolated the last package missing in the Dockerfile and successfully ran the GLD-HELICS-ROS2 integration test from Sabina: https://girlsinquantum.notion.site/ROS2-HELICS-GridLAB-D-Integration-Running-the-simple_gld_example-2ff07d8bae104e6b9989f0d3d21a6f57. I will submit a PR and push the updated VM.

4/9/2025 Shared slides for Final presentation with everyone. Started working on mine. Planning on removing some extra recap slides from physical environment (since Safiy is connecting ROS-GZ-BRIDGE this semester).

4/10/2025 Created a recap slide from the Midterm presentation and a slide discussing the Jazzy update with some research.

4/11/2025 Added some more information to the Jazzy slide about what I had to resolve with the Ubuntu image update, screenshot below.

Week 15 (4/17)

4/15/2025 I finished editing my slides and coordinated submission with the other members of the sub team.

4/17/2025 Today was the presentation for the VIP in class. I talked about the updates made to the VM. We got good feedback and will be in a great place to start connecting all of the pieces next semester.

4/18/2025 Missed the sub team meeting this morning (which possibly never took place). Hopefully the paper document is released soon and I can update my portion!

Week 16 (4/24)

4/21/2025 Resent the relevant paper file to the sub team. Did a quick sweep of the virtual machine section, I think I will rewrite the entire thing to make it more clear the different updates made to our Dockerfile.

4/22/2025 I spent time today rewriting my paper section. It flows a lot better and is more clear for new readers.

4/24/2025 Reviewed and revised my section, then the entire paper before submission. Looks good! I cleaned up my notebook a little bit to make sure that my formatting was complete. I made sure my checklists were all marked off and I will add a section to the table of contents where I started maintaining the virtual machine for easy access. I added the important links to access the VM to the top and I will mark the appendix with deprecation notes.

Appendix

CPS Setup Guide

Deprecated April 2025. Utilize the new VM at the top of the notebook and skip steps 1-3. Complete step 4 and then move on to the tutorials

1. Install a ROS2 Desktop VM via Docker. It’s available at https://hub.docker.com/r/tiryoh/ros2-desktop-vnc. You should browse the tags for “humble” and grab the most recent one. Install, run the image, and open the port to access the VM’s interface in your web browser. Everything else for this tutorial should be done in the VM using the Terminator or VSCodium apps, with Firefox as a web browser.
2. Install ROS2 Humble following this tutorial: https://docs.ros.org/en/humble/Installation/Ubuntu-Install-Debs.html. I can’t remember if it is already installed or not, but you should run all the steps to verify. If you get the talker/listener at the end of the guide to work, that’s perfect.
3. Installing Gazebo: First, run the two terminal blocks in this page (https://gazebosim.org/docs/harmonic/install_ubuntu/). Then, you can follow the steps in this page (https://gazebosim.org/docs/all/getstarted/), excluding Step 1.
4. I’ve set up a GH Repo for the sub team at https://github.com/cps-vip/gazebo-physical-env. Setting up Git in the VM can be a pain - I used the VSCodium (VM’s VSCode bootleg) GUI and authenticated within the VM. Once you clone, inside the repo’s my_local_models folder, you can run gz sim electrical_substation_v3.sdf to open the file we are working with. It’ll take a second.
5. If your VM struggles or there is very noticeable lag with Gazebo, try bumping up your resources in Docker.

Other Resources:

• Last semester, we used https://docs.ros.org/en/humble/Tutorials.html and did all of the Beginner: CLI Tools and Client Libraries guides to learn ROS. I think they are worth skimming to understand.
• You can also refer to my section in our research paper (should be in Discord) or my notebook starting at Week 8 from Spring 2024 Semester to understand our progress/research so far.