Authors: Sharwin Patil, Asa Rogers, Tony Shilati, and Grayson Snyder
Toastbot is a ros2 package that relies on the moveitapi package to command a Franka Emika Panda robot pick up a piece of bread, toast it, and place it on a plate. The package uses April tags to track objects in the physical environment of the robot and to track when object states change.
This repository depends on the moveit API developed by the same authors. To maintain this project and the API separately, it is a submodule in this repo and must be updated to include it.
When first cloning the repo:
# Ensure you're cloning in your ws/ directory
git clone --recurse-submodules git@github.com:snydergi/ToastBot.gitTo update the submodules when there are changes made to their repositories run:
git submodule init # If you're missing submodule
git submodule updateTo run the package and make toast, do the following commands:
colcon build
source install/setup.bash
ros2 launch toast make_toast.launch.py
ros2 service call /initiateToasting std_srvs/srv/EmptyA video of the toast_bot in action:
ToastBot.Demo.webm
A screen-recording of the rviz configuration:
Screencast.from.2024-12-11.22-17-38.2.webm
The toast_bot node is the main node which contains all the services necessary to accomplish the goal outlined in the overview.
The transform_auditor is used to constantly update and publish the poses of the scenes April Tags on topics used by subscribers in toast_bot.
This launch file launches all necessary nodes and rviz2 to make the robot ready to make toast.
The package uses RealSense cameras for our computer vision, utilizing the ROS April Tag package. With transforms made for each tag, the toast_bot contains services necessary to perform each action in the process of making toast, as well as a service that combines each intermediate step. A multi-thread executor is used in the toast_bot node such that once the toast completes and the toaster raises, the executor executes the remaining functions autonomously to allow for only one service to be called for the whole process.
ros2 service call /initiateToasting std_srvs/srv/Empty- Opens the gripper
- Go to Loaf Tray
- Close Gripper
- Lift Bread out of Loaf Tray
- Go to Home Position
- Move Bread directly over Toaster Slot 0
- Move Bread into slot
- Open the gripper
- Go to Home Position
- Waits for lever pose to exist
- Move above Lever
- Press Lever
- Move up above lever
- Go to Home Position
ros2 service call /buildScene std_srvs/srv/EmptyCreates the scene with the objects in simulation so planned paths account for them.
ros2 service call /breadToToaster std_srvs/srv/EmptyBread from Loaf Tray to Toaster Slot 0
ros2 service call /actuateLever std_srvs/srv/EmptyPress the Lever on the Toaster until it clicks.
ros2 service call /gohome std_srvs/srv/EmptySends the Franka to the Home Position
ros2 service call /toastToPlate std_srvs/srv/EmptyGrabs from toaster slot 0 and plates it
ros2 service call /openGripper std_srvs/srv/EmptyOpens the gripper
ros2 service call /closeGripper std_srvs/srv/EmptyCloses the gripper
Special thanks to Zhengyang Kris Weng (https://github.com/wengmister) for providing the CAD for the April Tag holder that was affixed to the base of the robot. The files can be found here: CAD LINK
Each part should have a location for an april tag to be attached to it so we can locate it with the camera. Since we know the geometry of all the fixtures and parts we can hard-code those transformations and dynamically locate their respective tags.
Thick, long prongs that resemble chopsticks so we can grab the bread from the holder and the toaster with quite some Z offset. The end-effector doesn't need an april tag.\
A holder with several slots for bread to be placed into. This is for pre-toasted bread and we can start with just 2 slots filled but we should have several slots to pick from.
Dimensions:
- Slot Length: 120 mm
- Slot Width: 20 mm
- Slot Spacing (center to center): 33.3 mm
- Overall Length: 200 mm
- Overall Width: 140 mm
- Height: 85 mm
- Number of Slots: 4
April Tag Size: 50 mm
Two pieces that enclose around the actual knife handle to make it a rectangular prism. \
An elevated stand (toaster box on its side is the perfect height) for the toaster. It will sit on the table close to the arm and off to the left (when facing the robot's forward direction) \
A platform or slab to attach to the toaster lever to make it easier to manipulate with the end-effector. Doesn't need an April tag but might be worth trying to fit a small one on there.
Lever Dimensions:
- Length: 35 mm
- Width: 20.4
- Height: 11.5 mm
April Tag Size: 70 mm
A fixture to hold the plate and keep it on the table. This should have an april tag to locate it.
Toast Dimensions:
- Length: 133 mm
- Width: 110 mm
- Height: 20 mm
April Tag Size:
-
Toaster tag 0
- total size: 20 mm - 12.5 mm
- tag size : 20 mm - 12.5 mm
-
Lever tag 1
- total size:: 30 mm
- tag size : 15 mm
-
Plate tag 3
- total size: 30
- tag size : 24
-
Bread Holder tag 4
- total size: 85 mm
- tag size : 85 mm
Stretch Goal
-
Lever Dial tag 5
- total size: 35 mm
- tag size : 10 mm
-
Knife Handle tag 2
- total size:: 30 mm
- tag size : 30 mm
Length: 241.3 mm
Width: 152.4 mm
Height: 180.975 mm
-
Apriltags
- install by running the following:
sudo apt install ros-jazzy-apriltag-ros
- install by running the following:
-
To launch
colcon clean workspace -y && colcon build && source install/setup.bash && ros2 launch realsense camera.launch.py
The toast_bot team's stretch goal was to be able to butter the toast after toasting, as well as actuate the dial to a preferred 'toastiness.' Additionally, further improvements could be made to allow for multiple pieces of toast to be made in one program execution.