A self-directed robotics R&D project: a mobile particle-forensics robot — a UR5e-class cobot on a linear rail, carrying an inspection head with UV illumination and multi-modal cameras. It reaches into enclosed precision equipment, images fine particle distributions, and uses a vision + LLM-agent pipeline to help localise where contamination is coming from.
Two perception jobs run at once:
- Inspection eyes — high-magnification UV/2D imaging → particle detection, size distribution, spatial mapping onto a reference frame.
- Guiding eyes — a wrist-mounted depth camera → voxel map → collision-aware motion planning for safe entry into tight spaces.
The LLM agent is a collaborator, not an autopilot: a human describes the symptom, the agent proposes an inspection sequence, executes it behind confirm-gates, presents image evidence, and iterates.
The simulation work is the live part of this repo — everything runs on a laptop, no hardware required.
| Doc | Phase | What it covers |
|---|---|---|
00_Simulation_Roadmap.md |
— | Master roadmap: phases, sim-engine tradeoffs, what the sim can and cannot prove |
01_Phase1_UR5e_URScript.md |
1 ✅ | UR5e moving via real URScript through URSim's actual PolyScope controller, plus MoveIt2 planning and Gazebo visualisation |
02_Phase2_Linear_Rail.md |
2 ✅ | The rail as a true 7th DOF in one kinematic chain — one IK solution, one trajectory, one execution |
02b_OvalRail_Custom_Kinematics_Plugin.md |
2 | Why a custom closed-loop kinematics plugin was evaluated and rejected |
02c_Planning_Stack_Concepts.md |
2 | How the MoveIt planning stack actually fits together |
02_Setup_Sequences_and_Troubleshooting.md |
— | Bring-up sequences and the failure modes hit along the way |
Code:
moveit_to_urscript.py— exports a MoveIt-planned trajectory as URScript a real UR controller could run directlyphase2_tools/— deterministic coordinate-based motion scripts:move_to_xyz.py,move_arm_only.py,move_sequenced.py,reach_envelope_publisher.pystadium.scad/stadium_rail.stl— the rail geometry
A standalone reference on the robot communication stack: UR's eight native control and integration interfaces, and what ROS 2 actually buys you over a raw TCP socket.
| Phase | Goal | Status |
|---|---|---|
| 1 | UR5e in sim, driven by real URScript and by MoveIt2 | ✅ Done |
| 2 | Linear rail as a 7th DOF, single coordinated trajectory | ✅ Done |
| 3 | Inspection-tool mockup in scene as a collision object; scripted inspection sweep | 📋 Planned |
| 4 | Perception — UV/2D particle detection + depth → occupancy feeding the MoveIt collision scene | 📋 Planned |
| 5 | Agent layer — natural-language symptom → bounded inspection sequence, human confirm-gates, structured evidence report | 📋 Planned |
| 6 | Integration, cheap physical surrogate, demo video and write-ups | 📋 Planned |
ROS 2 Jazzy · MoveIt 2 · Gazebo (Isaac Sim targeted for vision-in-the-loop) · URSim / PolyScope · URScript · Python · WSL2 on Windows 11
- Learn robotics for real — motion planning, kinematics, coordinate frames, EOAT design, sim-to-real workflow.
- Build genuine perception pipelines — 2D + 3D detection, segmentation, depth → occupancy. Not slideware.
- Practice agentic AI on hardware — an LLM agent wired to a bounded, safe action executor with human-in-the-loop gates.
- Produce a portfolio-grade artifact — clean repo, reproducible build, real demos, honest write-ups of what worked and what didn't.
Several docs reference a CDR (Critical Design Review) and its slides. That design review is the project's architecture record; the deck itself is kept privately and is not published in this repository. The references are retained because the reasoning they point at is genuine design history — treat them as pointers to an internal record, not a broken link.