Repository navigation
Sam Ling's Notebook
Week 1 (1/09 - 1/16) Introduction to the VIP, deciding which subteam to join.
Week 2 (1/16 - 1/23) Joined ROS/Helics subteam, assigned to work on Navigation/Nodes and Sensors subteam. Reviewed slides/paper from last semester to get caught up.
Week 3 (1/23 - 1/30) Setting up tech stack. USE JOSEPHS NOTEBOOK!!
Got Gazebo running
Moved from humble --> jazzy
Week 4 (1/30 - 2/6) Beginning to do mission research on how device models may malfunction in the real world. Reading over relevant articles/research.
Week 5 (2/6 - 2/13)
Continued to research use cases, specifically LIDAR, optical sensors, and heat sensors. Reviewed the paper midterm draft, made some slight modifications to the NAV/Sensors section.
https://eprijournal.com/robots-take-on-tough-tasks-in-transmission-and-distribution/
https://newsroom.axis.com/en-us/blog/electrical-substations-thermal-imaging
These artices emphasized the importance of heat sensors, so this semms like a good first step.
Week 6 (2/13 - 2/20)
We decided to begin finding out how to implement the sensors in simulation. We will start with heat sensors and see what progress is made from there.
During class we reviewed the project structure to make sure everyone had a good understanding of what is going on, the meeting was good for understanding how each piece of the VIP will fit together in the long run.
Figuring out where to start from here, the overall goal of the VIP is finally starting to make sense now.
TODO: Find out how to get a heat sensor in Gazebo
Week 7 (2/20 - 2/27)
Researching how to actually implement the heat sensor in Gazebo. Found this tutorial:
.
It requires that we already have a model so I will begin implementing the robot in the team's github repo.
USE JOSEPH'S APPENDIX/NOTEBOOK for setup help.
TODO: Get the heat sensor working in an environment
Week 8 (2/27 - 3/6)
Got github setup in VSCodium.
.
Made sure ROS is correctly installed in the VM, I can now begin implementing the heat sensor in Gazebo. Gazebo tutorial: https://gazebosim.org/api/sensors/6/thermalcameraigngazebo.html
Natlie and Nguyen started working more on the navigation side of things, so Rick and I will be doing different things from them from now on.
TODO: Get a working heat sensor in environment
Week 9 (3/6 - 3/13) We already have an environment sdf with a robot in it, so the plan now is to get a working heat sensor on the robot here.
Got the electrical substation running on the VM.
Performance is questionable, will have to figure out a way to fix this. Continuing to follow the Gazebo tutorial on adding a heat sensor to this model.
TODO: Add a heat sensor to this environment (Rick will be doing this while I try to get the actual camera working).
Week 10 (3/13 - 3/20) SPRING BREAK: not much progress made
Week 11 (3/20 - 3/27)
Got the thermal camera model working, currently trying to figure out how to set up publisher/subscriber to actually extract meaningful information from the camera. As of now, the whiter an object is, the more heat it has.
To run the camera: gz sim thermal_camera.sdf
Attempting to make a listener, running into this error.
TODO: Get data from the heat sensor
Week 12 (3/27 - 4/3)
Got thermal_camera subscriber working...
The data has no useful meaning right now, but am working on finding a way to make use of it.
To get the subscriber working: gz topic -e -n 1 -t /thermal_camera
TODO: Find a way to make sense of heat sensor data
Week 13 (4/3 - 4/10)
Attempting to publish gazebo topic data to ros2, will read sensor data from rviz.
To open RVIZ2: ros2 run rviz2 rviz2
Natalie and Nguyen are doing the same thing for a LIDAR camera, so will ask them how they did it.
ODO: Get heat sensor data into RVIZ2
Week 14 (4/10 - 4/17)
Final week of VIP, last week was very busy with other classes so didn't make too much progress on getting the thermal camera into RVIZ2. Prepared for the final presentation mostly this week. I'll try getting the camera into RVIZ2 if I am able to find some time in between everything going on before the semester ends.
Week 15 (4/17 - 4/24)
Mainly worked on adding to the final paper. Helped with a short section about the heat sensors that Rick and I have been working on this semester.
TODO: Figure out next steps, review kickoff presentation from last semester.
Week 2: Hopped on a call with subteam to figure stuff out. Worked on what needs to be cut/included for this years kickoff presentation.
Decided that we will put sensors on hold and try to get a mission working. Also decided that I will help out a bit with the physical robot subteam. Given this link to look at: https://girlsinquantum.notion.site/ROS2-HELICS-GridLAB-D-Integration-Running-the-simple_gld_example-2ff07d8bae104e6b9989f0d3d21a6f57
TODO: Reach out to Sabina to ask about physical robot tasks. Review previous work on fault detection (Safiy's notebook).
Week 3: Reviewed previous work on fault detection, not too much else was done, projects from other classes are picking up...
TODO: Ask Will about physical robot implementation?
Refamiliarize myself with ROS2 and progress made from last semester. Read over some material given out during team meeting to help with this.
Team Meeting: Mainly discussed progress and what to do moving forward.
Logged into VM and got reacquainted with Linux workflow.
Reran everything just to make sure it is still working.
Found that everything is still working as expected.
Team Meeting: Still considering if I should Sabina's physical robot team, shall see how busy I am moving forward.
Researching how to get my Gazebo topic exposed to Rviz2. Based on what I've found, it seems like next steps are to use the Gazebo -> ROS bridge pack in order to get it into Ros2, which can then be read into Rviz2.
- Install bridge package: sudo apt-get install ros-jazzy-ros-gz-bridge
...
TODO: Continue with next steps of getting data into Rviz2
Team Meeting: Was not able to attend because I was sick, decided to just stick with thermal camera implementation moving forward.
Slow week for the whole team with many tests and projects due before fall break, mainly worked on what to do for the midterm presentation. Overall we did not get too much done over the break.
Continue getting sensor data into Rviz.
2.1) First open up the thermal camera: gz sim thermal_camera.sdf
2.2) and listener: gz topic -e -n 1 -t /thermal_camera in a new terminal.
-
ros2 run ros_gz_bridge parameter_bridge
"/world/my_world/model/robot/link/thermal_camera_link/sensor/thermal_camera/image@sensor_msgs/msg/Image@gz.msgs.Image" in a new terminal -
ros2 run ros_gz_bridge parameter_bridge
"/thermal_camera_8bit/image@sensor_msgs/msg/Image@gz.msgs.Image"
"/thermal_camera_8bit/camera_info@sensor_msgs/msg/CameraInfo@gz.msgs.CameraInfo" in a new terminal -
Run the simulation from original terminal.
Ran into an issue where it seems like Ros2 is recognizing the thermal camera topic, but data is not being passed in. Need to figure this out next week.
Team Meeting: Discussed progress and started bringing up ideas of what to add to final presentation
Figured out above issue, in same terminal where listener is (step 2.2) run: ros2 topic echo /thermal_camera_8bit/image
Seems like array of pixels has been turned into a decimal value.
The above command is used for image information only, the camera metadata must also be known for correct 3D visualization, so run
ros2 topic echo /thermal_camera_8bit/camera_info for camera info.
Showing camera metadata.
Team Meeting: Team is making steady progress, we are going in a good direction.
In same terminal, run: ros2 run rviz2 rviz2
In Rviz2 go to Displays -> Add (bottom left corner) -> By Topic -> Scroll down till you see the Image topic and add.
It seems like the image is not being correctl displayed in the 3D world (right sde of the screenshot). Must figure out what went wrong from here.
Got very sick this week so was not able to get much done at all...
Did a little bit of research and it seems like issue can be fixed if I publish the correct TF frame for Rviz2 to see
TODO: Look into how I can use TF frames to communicate between these 2 environments.
To publish TF frame run in a new terminal: ros2 run tf2_ros static_transform_publisher 0 0 0 0 0 0 world thermal_camera_8bit/link/thermal_camera_8bit
In Rviz2 under global options, go to drop down menu of fixed frame and select the thermal_camera_8bit
A cone of the camera view should appear
Almost there but it seems like the actual image being viewed by the thermal camera is not being sent to Rviz2, just the camera "viewing angle"
Team Meeting: Making plan for final presentation
Did a bit more research on TF frames/the issue aboce, but was not able to debug issue in time to put it on final presentation. Ideally I can get this issue figured out early next semester. Overall, seems like a decent stopping point with good progress made on getting the heat sensor working. I think we are close to having a working heat sensor!
Apart from debugging, spent some time working on final presentation.
Team Meeting: Discussing presentation
Thanksgiving break this week so not much progress made on debugging issue. Mainly just spent time changing my section on the final paper.
Intro to VIP spent time mainly talking with old teammates, went over usual introduction stuff for new people
Began looking into device security subteam after initial presentations. Seems pretty interesting and more in line with what I would like to learn. Spent the week researching stuff related to PUFs to add to kick off presentation.
Gave kickoff presentation, continued to define scope for PUF subteam, mainly waiting for hardware to begin testing on
https://data.embeddedcomputing.com/uploads/articles/whitepapers/14491.pdf Read an article that Hayden provided. Paper mainly provided a quick overview of PUFs and why they are needed.
https://ieeexplore.ieee.org/document/8824887 Read another article provided by Hayden. Paper mainly goes the the more technical aspects of PUFs and how to actually implement. Will definitely considers things from this paper once we get our hands on hardware.
Talked to Will and set up some goals for when hardware is delivered next week. Goals: Determine if the startup values are truly random, make sure they aren't zeroed, figure out how to configure code to do this
Mainly worked on the midterm presentation with the group.
Team began learning Rust with https://rustlings.rust-lang.org/
Next tasks are to dump SRAM contents for further analysis.
This week a script was made to dump SRAM contents.
- We collected ~20 logs separated by board number
Raspberry PIs were handed out. Began following Hayden's documentation on getting it setup but we ended up deciding to use the STM32's instead. Began doing research on how we can analyze the current dumps.
- Planning on making a script that counts the ratio of 0s and 1s to make sure that there is no bias.
Made a script that analyzes bit bias, results seem like the boards are very well balanced.
https://github.com/Sling38/sam
Spring break, not much done
Group is deciding on which ECC method to use. Getting most of our ideas from this paper https://www.researchgate.net/publication/375121303_A_Comprehensive_Comparison_of_Fuzzy_Extractor_Schemes_Employing_Different_Error_Correction_Codes Overall a slower week coming out of spring break for everyone.
Other members are looking into BCH for Error Correction. I wanted to look more into spatial analysis in order to confirm that the randomness is indeed spread out evenly across the blocks.
Picture above shows that the blocks are all roughly .5 uniformity and have high average stabilities.
https://github.com/Sling38/sam
Spent rest of the week with presentation and final report write up.