Skip to content

Conversation

@leshy
Copy link
Contributor

@leshy leshy commented Oct 8, 2025

depends on #666

resolves dept from the g1 video rush, significantly improves the detection system

  • person detection message and sketch (WIP)
  • reorganized and rewrote 2d detectors into detectors/ dir, implemented detic, yolo person, yolo
  • much better detection results across the board

additional utils

  • timestamp alignment that takes into account early and late messages in relation to a primary stream (using timestamped buffer and new weaklist type)
def align_timestamped(
    primary_observable: Observable[PRIMARY],
    *secondary_observables: Observable[SECONDARY],
    buffer_size: float = 1.0,  # seconds
    match_tolerance: float = 0.1)  # seconds
  • implemented dimos.utils.decorators simple_mcache function for caching methods (functools.cache had issues with considering self an argument - should switch all functools.cache calls to above)

small test fixes

  • rewrote alex frontier explorer test to not be too slow
  • renamed TestB1ConnectionModule to MockB1ConnectionModule - since pytest think it's a test and emits a warning

@leshy leshy changed the title wip Detection second pass Detection second pass Oct 9, 2025
paul-nechifor
paul-nechifor previously approved these changes Oct 10, 2025
Co-authored-by: Paul Nechifor <paul@nechifor.net>
@leshy leshy merged commit 66440ab into dev Oct 10, 2025
12 checks passed
spomichter added a commit that referenced this pull request Oct 12, 2025
* g1 switched over to webcam module

* camera system cleanup, calibration loading

* zed calibration file

* removed comments / unused imports from zed

* integration wip

* universal camera module

* fixed flakey test_reactive test

* removed obsolete test

* print cleanup

* topic change for bridge, small camera module fixes

* g1 local changes

* ros global map

* height filter config for module3d

* splitting types, object db work

* circular imports solved

* foxglove sceneupdate

* pointcloud bounding box intersection, detection3d projection refactor

* checkpoint

* detection work snapshot

* testing refactor

* good replay example

* bugfixes, improvements, g1 compatibilty

* working on universal recorder

* recorder cli

* onboard g1 changes and recording

* corrected timestamp alignment

* temporary nav integration

* color hash type, timestamp alignment fix

* new timestamp alignment

* timed replay refactor

* correct detected image broadcast from module2d

* better dict repr

* g1 replay system

* g1 filters

* weaklist

* raycast bugfix

* small bugfixes

* agent integration to unitree_go2

* TOFIX double pub goal message for reliability

* fix

* cam fix

* added joy message type to dimos

* added set autonomy mode

* added joy to ros bridge

* fixed

* CI code cleanup

* Fully working G1 ros navigation to origin

* g1 agents2 spatial navigation

* CI code cleanup

* comment out camera image

* fix

* commit

* adapt

* image

* crop image

* switch back to old camera

* sharpness window generalized to quality_barrier

* tests consolidation, preparing for merge

* tests fix

* qwen localization

* nav to object in view

* forgot init

* bugfix

* bugfix

* fix for timestamp on g1

* killing time stuff

* quick fixes

* onboard unitree changes

* CI code cleanup

* moduledb hack

* moduledb hack

* current g1

* quaternion fix

* Re enabled detections, removed go to origin on startup

* Fully working save location and navigate to saved location

* Fully working G1 spatial memory, detections, location saving on agents2

* Fully working G1 webrtc skills integrated as SkillContainer module for agents2

* foxglove vis for 3d localization

* cleaning up detection2d

* added detic

* detic

* pose detector sketch

* restructure

* pose -> person

* person detector sketch

* yolo pose test

* separated detection3d and detection3dpc

* object3d new test

* lcm replay test

* thread cleanup

* added seek for example

* deactivate detic

* detection3d bugfix

* fixing tests 1

* fixing tests

* wavefront explorer smaller costmap for faster testing

* person detector merged with follower spec

* fixing tests, timestamp alignment threading

* people annotations test fix

* removed recorder, moved qewen from dev

* fixing tests

* tests passing now

* indeterministic test fix

* removing temp file

* bugfix

* Update dimos/perception/detection2d/detectors/detic.py

Co-authored-by: Paul Nechifor <paul@nechifor.net>

---------

Co-authored-by: alexlin2 <alex.lin416@outlook.com>
Co-authored-by: alexlin2 <44330195+alexlin2@users.noreply.github.com>
Co-authored-by: Paul Nechifor <paul@nechifor.net>
Co-authored-by: paul-nechifor <1262969+paul-nechifor@users.noreply.github.com>
Co-authored-by: Stash Pomichter <pomichterstash@gmail.com>
spomichter added a commit that referenced this pull request Oct 28, 2025
Release v0.0.5


## What's Changed
* Unitree WebRTC implementation on rebased dev by @leshy in #277
* Update ros_observable_topic timeout to 100s by @leshy in #273
* Updated README, more clear on API key requirements and updated go2_ros2_sdk remote by @spomichter in #272
* Release v0.0.4 Patch: readme changes by @spomichter in #292
* Readme patch v0.0.4 by @spomichter in #293
* Development container & CI by @leshy in #278
* env/devcontainer ruff formatting/typing by @leshy in #294
* Global reformat 100 line length  by @spomichter in #300
* Global code reformat with ruff by @leshy in #295
* Position/Vector type cleanup & tests by @leshy in #297
* Linelength100 by @leshy in #301
* Auto-delivery of binary data files for testing, rewrite of dev script by @leshy in #298
* pre-commit hooks in dev container & CI, automatic LFS upload by @leshy in #303
* Removed all submodules - Testing by @spomichter in #306
* Fixed v0.0.4 Unitree ROS runfile broken by WebRTC development, Vector.py fixes by @spomichter in #307
* test/mapper by @leshy in #305
* Reduced CI cleanup frequency to PRs only into dev/main by @spomichter in #312
* DimOS Manipulation Framework, ObjectDetectionStream Changes by @spomichter in #308
* Added auto-license header to pre-commit by @spomichter in #336
* Move thread fix for alex planner by @leshy in #334
* base typing cleanup, sensor reply tests+docs by @leshy in #309
* devcontainer docs by @leshy in #338
* ci docs by @leshy in #339
* Add Cerebras Agent by @joshuajerin in #310
* Repo cleanup by @leshy in #340
* noros builds by @leshy in #341
* Update testing_stream_reply.md by @leshy in #342
* ONNX conversions for YOLOv11 and FastSAM by @mdaiter in #350
* Test cicd fake ros change by @spomichter in #361
* Reverted cleanup workflow frequency to on any PUSH due to CICD docker workflow issues by @spomichter in #360
* Trigger docker ros rerun by @spomichter in #363
* Ros CI change detection by @leshy in #364
* trigger full rebuild by @leshy in #365
* Add CLIP ONNX conversion and support, with passing vision and text tests by @mdaiter in #353
* CI fix 3 by @leshy in #367
* ONNX Support for YOLO, SAM2 + Unit tests for CLIP, YOLO, SAM2 by @spomichter in #345
* LFS moved to utils from testing by @leshy in #368
* Contact graspnet integration on pytorch and pyproject build processes setup with cuda/manipulation tags by @spomichter in #370
* data/* deletions by @leshy in #369
* Ci pre-commit and docker builds run in parallel by @leshy in #372
* Ci shared docker cache by @leshy in #371
* Unitree WebRTC integrated with full functionality, remove all ROS dependency, refactored entire robot base class and connection interface, added explore skill by @alexlin2 in #279
* Unitree WebRTC only implementation, Exploration skills [Staging --> Dev] by @spomichter in #379
* Dask lcm multiprocess by @leshy in #377
* DimOS Packaging & Build Improvements for CPU-only, CUDA, Manipulation installations by @spomichter in #394
* Multitree go2 by @leshy in #381
* better LCM system checks, fixes bin/lfs_push by @leshy in #382
* UnitreeSpeak skill over webrtc, Voice Interface added on localhost, Voice interface on mobile device on network by @spomichter in #400
* FIX: multiprocess by @leshy in #402
* Lcmspy cli by @leshy in #404
* changed position type name to pose by @alexlin2 in #358
* WIP: foxglove bridge stub by @leshy in #411
* Create running_without_devcontainer.md by @leshy in #405
* new LCM class format support by @leshy in #417
* Fixed PoseStamped ros_msgs error in dimos-lcm by @spomichter in #457
* Fixes move stream issue, Odom receive issue by @leshy in #456
* Small stream/type fixes for unitree by @leshy in #460
* Local planner, Global Planner, Explore, SpatialMemory working via LCM/Dask Multiprocess by @spomichter in #467
* Added working runfile to Unitreego2Light class by @spomichter in #474
* Point Cloud Filtering and Segmentation, Full 6DOF Object pose estimation, Grasp generation, ZED driver support, Hosted grasp integration by @spomichter in #458
* Stream fixes, Twist, Pose, Quaternion updates by @leshy in #471
* Added self-hosted runner to full CICD by @spomichter in #484
* Full Unitree (Local planner, Explore, SpatialMemory) FakeRTC/WebRTC LCM modules working in self-hosted devcontainer  by @spomichter in #487
* Porting types/ LCM msgs/ new LCM types, Transform visualization by @leshy in #477
* Tracking streams lcm dask refactor by @spomichter in #488
* Pytransforms by @leshy in #491
* Fix python and dev docker builds for CICD by @spomichter in #489
* Remove PIL Image Usage by @alexlin2 in #490
* Added missing __init__.py's to transforms  by @spomichter in #493
* Added tofix pytest tag back to addopts by @spomichter in #494
* Added module docs by @spomichter in #495
* SpatialMemory converted to Dask module, input LCM odom and video streams by @spomichter in #481
* Run modules tests only on 16gb runner by @spomichter in #499
* Trigger CI only on PR or push to main/dev by @spomichter in #500
* Added more aggressive cleanup workflows by @spomichter in #501
* Visual Servoing for Pick and Place Demo by @alexlin2 in #476
* Testing run-tests container pull fix and removed modules tests by @spomichter in #505
* Fix permissions in pre-build-cleanup by @spomichter in #508
* Moved pre-build cleanup to build template by @spomichter in #509
* dimos lcm update to main branch latest commit by @leshy in #498
* RPC Kwargs by @leshy in #503
* Transform system, stream convinience features, type checking by @leshy in #504
* Dimoslcm bump by @leshy in #510
* Testing UV builds in docker by @spomichter in #513
* OccupancyGrid, Path types by @leshy in #511
* subscribing to transports/streams from main loop by @leshy in #524
* Alex Lin's version of ROS Nav2 by @alexlin2 in #514
* Agent refactor conversation history by @spomichter in #541
* Exposed optional memory_limit param in dimos core by @spomichter in #540
* Agent refactor by @spomichter in #535
* Validating transforms with ros examples by @leshy in #538
* rpc timeout by @leshy in #542
* MuJoCo Simulation by @paul-nechifor in #539
* Revert "MuJoCo Simulation" by @spomichter in #548
* perception refactor to be on parity with old architecture by @alexlin2 in #534
* Skill coordinator by @leshy in #536
* WIP Mujoco simulation by @paul-nechifor in #549
* Fix event loop leak by @paul-nechifor in #547
* Correct way to build package directly in non-editable mode, no manife… by @spomichter in #551
* Office environment mujoco by @paul-nechifor in #554
* Less bandwidth usage on LCM, bug fixed with navigation by @alexlin2 in #559
* disabled old agent tests by @leshy in #563
* Camera Module Refactor, added image rectification by @alexlin2 in #566
* long rpc timeout by @leshy in #569
* Twist message for all move command, added keyboard teleop for easy robot control in sim by @alexlin2 in #570
* numerical sort for sensor replay by @leshy in #564
* 2d detection module by @leshy in #567
* Stream timestamp alignment by @leshy in #557
* Sharpness for Images by @leshy in #560
* Jetson humanoid integration by @spomichter in #590
* 2d detection module + Agent2 - yolo demo by @leshy in #582
* jetson.md cleanup by @spomichter in #602
* Unitree b1 integration with continuous cmd_vel Twist interface, joystick control for testing, C++ UDP server for onboard B1 by @spomichter in #601
* Joystick integrated g1 humanoid by @spomichter in #603
* Unitree b1 manipulation pose integration by @spomichter in #604
* use SHM in Foxglove by @paul-nechifor in #607
* CPU isolated shared mem by @mdaiter in #589
* silence unnecessary unitree go 2 tricks by @paul-nechifor in #615
* Pshm to lcm by @paul-nechifor in #616
* Unitree agents2 skill integration paul by @paul-nechifor in #617
* Unitree go2 runfile integration tool call issues by @spomichter in #605
* gstreamer camera by @paul-nechifor in #613
* zed local node by @leshy in #623
* ROS Bridge for Unitree G1 and B1 Navigation, Working G1 navigation by @spomichter in #610
* B1 ros navigation rebase by @spomichter in #626
* Added build directory to gitignore by @yashas-salankimatt in #628
* 2D detection module + Pointcloud localization by @leshy in #583
* Camera calibration loading by @leshy in #629
* Agent2 nav skills by @paul-nechifor in #630
* WIP shared mem again by @paul-nechifor in #650
* Fix leaks by @paul-nechifor in #649
* Fix SHM leak by @paul-nechifor in #652
* Suppress echos with counter by @paul-nechifor in #653
* Removing websocket vis causing crazy lag by @spomichter in #656
* Suppress with UUID by @paul-nechifor in #655
* Modules navigate object bbox by @spomichter in #654
* Ros bridge test fix by @alexlin2 in #660
* video g1 spatial mem + detection - tomerge by @leshy in #651
* Update README.md by @spomichter in #664
* Image upgrades! Impls for CUDA + numpy, along with an abstraction and full backwards compatibility by @mdaiter in #612
* Revert "Image upgrades! Impls for CUDA + numpy, along with an abstraction and full backwards compatibility" by @leshy in #665
* Detection second pass by @leshy in #662
* CudaImage by @spomichter in #671
* Add start/stop to all modules and other resources by @paul-nechifor in #675
* forgotten context managers by @paul-nechifor in #676
* CUDAImage, NumpyImage, Image implementations with robust backend tests for image operations by @spomichter in #680
* CudaImage by @spomichter in #677
* alibaba env var fix by @leshy in #673
* Rename FakeRTC --> ReplayRTC by @spomichter in #681
* Fix websocketvis performance rebase by @spomichter in #682
* Alexl ros nav intergration by @alexlin2 in #632
* detection pipeline rewrite, embedding, vl model standardization, reid system by @leshy in #674
* cli tooling theme by @leshy in #687
* Fix spatial memory bug in g1  by @spomichter in #689
* Add autoconnect back2 by @paul-nechifor in #684
* Add ability to remap module connections name. by @paul-nechifor in #698
* Add transport which encodes images as JPEG to improve performance. by @paul-nechifor in #693
* New Ruff autofixes by @paul-nechifor in #694

## New Contributors
* @joshuajerin made their first contribution in #310
* @mdaiter made their first contribution in #350
* @yashas-salankimatt made their first contribution in #628

**Full Changelog**: https://github.com/dimensionalOS/dimos/commits/v0.0.5
spomichter added a commit that referenced this pull request Jan 8, 2026
* g1 switched over to webcam module

* camera system cleanup, calibration loading

* zed calibration file

* removed comments / unused imports from zed

* integration wip

* universal camera module

* fixed flakey test_reactive test

* removed obsolete test

* print cleanup

* topic change for bridge, small camera module fixes

* g1 local changes

* ros global map

* height filter config for module3d

* splitting types, object db work

* circular imports solved

* foxglove sceneupdate

* pointcloud bounding box intersection, detection3d projection refactor

* checkpoint

* detection work snapshot

* testing refactor

* good replay example

* bugfixes, improvements, g1 compatibilty

* working on universal recorder

* recorder cli

* onboard g1 changes and recording

* corrected timestamp alignment

* temporary nav integration

* color hash type, timestamp alignment fix

* new timestamp alignment

* timed replay refactor

* correct detected image broadcast from module2d

* better dict repr

* g1 replay system

* g1 filters

* weaklist

* raycast bugfix

* small bugfixes

* agent integration to unitree_go2

* TOFIX double pub goal message for reliability

* fix

* cam fix

* added joy message type to dimos

* added set autonomy mode

* added joy to ros bridge

* fixed

* CI code cleanup

* Fully working G1 ros navigation to origin

* g1 agents2 spatial navigation

* CI code cleanup

* comment out camera image

* fix

* commit

* adapt

* image

* crop image

* switch back to old camera

* sharpness window generalized to quality_barrier

* tests consolidation, preparing for merge

* tests fix

* qwen localization

* nav to object in view

* forgot init

* bugfix

* bugfix

* fix for timestamp on g1

* killing time stuff

* quick fixes

* onboard unitree changes

* CI code cleanup

* moduledb hack

* moduledb hack

* current g1

* quaternion fix

* Re enabled detections, removed go to origin on startup

* Fully working save location and navigate to saved location

* Fully working G1 spatial memory, detections, location saving on agents2

* Fully working G1 webrtc skills integrated as SkillContainer module for agents2

* foxglove vis for 3d localization

* cleaning up detection2d

* added detic

* detic

* pose detector sketch

* restructure

* pose -> person

* person detector sketch

* yolo pose test

* separated detection3d and detection3dpc

* object3d new test

* lcm replay test

* thread cleanup

* added seek for example

* deactivate detic

* detection3d bugfix

* fixing tests 1

* fixing tests

* wavefront explorer smaller costmap for faster testing

* person detector merged with follower spec

* fixing tests, timestamp alignment threading

* people annotations test fix

* removed recorder, moved qewen from dev

* fixing tests

* tests passing now

* indeterministic test fix

* removing temp file

* bugfix

* Update dimos/perception/detection2d/detectors/detic.py

Co-authored-by: Paul Nechifor <paul@nechifor.net>

---------

Co-authored-by: alexlin2 <alex.lin416@outlook.com>
Co-authored-by: alexlin2 <44330195+alexlin2@users.noreply.github.com>
Co-authored-by: Paul Nechifor <paul@nechifor.net>
Co-authored-by: paul-nechifor <1262969+paul-nechifor@users.noreply.github.com>
Co-authored-by: Stash Pomichter <pomichterstash@gmail.com>
Former-commit-id: 66440ab
paul-nechifor added a commit that referenced this pull request Jan 8, 2026
* g1 switched over to webcam module

* camera system cleanup, calibration loading

* zed calibration file

* removed comments / unused imports from zed

* integration wip

* universal camera module

* fixed flakey test_reactive test

* removed obsolete test

* print cleanup

* topic change for bridge, small camera module fixes

* g1 local changes

* ros global map

* height filter config for module3d

* splitting types, object db work

* circular imports solved

* foxglove sceneupdate

* pointcloud bounding box intersection, detection3d projection refactor

* checkpoint

* detection work snapshot

* testing refactor

* good replay example

* bugfixes, improvements, g1 compatibilty

* working on universal recorder

* recorder cli

* onboard g1 changes and recording

* corrected timestamp alignment

* temporary nav integration

* color hash type, timestamp alignment fix

* new timestamp alignment

* timed replay refactor

* correct detected image broadcast from module2d

* better dict repr

* g1 replay system

* g1 filters

* weaklist

* raycast bugfix

* small bugfixes

* agent integration to unitree_go2

* TOFIX double pub goal message for reliability

* fix

* cam fix

* added joy message type to dimos

* added set autonomy mode

* added joy to ros bridge

* fixed

* CI code cleanup

* Fully working G1 ros navigation to origin

* g1 agents2 spatial navigation

* CI code cleanup

* comment out camera image

* fix

* commit

* adapt

* image

* crop image

* switch back to old camera

* sharpness window generalized to quality_barrier

* tests consolidation, preparing for merge

* tests fix

* qwen localization

* nav to object in view

* forgot init

* bugfix

* bugfix

* fix for timestamp on g1

* killing time stuff

* quick fixes

* onboard unitree changes

* CI code cleanup

* moduledb hack

* moduledb hack

* current g1

* quaternion fix

* Re enabled detections, removed go to origin on startup

* Fully working save location and navigate to saved location

* Fully working G1 spatial memory, detections, location saving on agents2

* Fully working G1 webrtc skills integrated as SkillContainer module for agents2

* foxglove vis for 3d localization

* cleaning up detection2d

* added detic

* detic

* pose detector sketch

* restructure

* pose -> person

* person detector sketch

* yolo pose test

* separated detection3d and detection3dpc

* object3d new test

* lcm replay test

* thread cleanup

* added seek for example

* deactivate detic

* detection3d bugfix

* fixing tests 1

* fixing tests

* wavefront explorer smaller costmap for faster testing

* person detector merged with follower spec

* fixing tests, timestamp alignment threading

* people annotations test fix

* removed recorder, moved qewen from dev

* fixing tests

* tests passing now

* indeterministic test fix

* removing temp file

* bugfix

* Update dimos/perception/detection2d/detectors/detic.py

Co-authored-by: Paul Nechifor <paul@nechifor.net>

---------

Co-authored-by: alexlin2 <alex.lin416@outlook.com>
Co-authored-by: alexlin2 <44330195+alexlin2@users.noreply.github.com>
Co-authored-by: Paul Nechifor <paul@nechifor.net>
Co-authored-by: paul-nechifor <1262969+paul-nechifor@users.noreply.github.com>
Co-authored-by: Stash Pomichter <pomichterstash@gmail.com>
Former-commit-id: 607c3c8 [formerly 66440ab]
Former-commit-id: 4a54ad1
jeff-hykin pushed a commit that referenced this pull request Jan 9, 2026
* g1 switched over to webcam module

* camera system cleanup, calibration loading

* zed calibration file

* removed comments / unused imports from zed

* integration wip

* universal camera module

* fixed flakey test_reactive test

* removed obsolete test

* print cleanup

* topic change for bridge, small camera module fixes

* g1 local changes

* ros global map

* height filter config for module3d

* splitting types, object db work

* circular imports solved

* foxglove sceneupdate

* pointcloud bounding box intersection, detection3d projection refactor

* checkpoint

* detection work snapshot

* testing refactor

* good replay example

* bugfixes, improvements, g1 compatibilty

* working on universal recorder

* recorder cli

* onboard g1 changes and recording

* corrected timestamp alignment

* temporary nav integration

* color hash type, timestamp alignment fix

* new timestamp alignment

* timed replay refactor

* correct detected image broadcast from module2d

* better dict repr

* g1 replay system

* g1 filters

* weaklist

* raycast bugfix

* small bugfixes

* agent integration to unitree_go2

* TOFIX double pub goal message for reliability

* fix

* cam fix

* added joy message type to dimos

* added set autonomy mode

* added joy to ros bridge

* fixed

* CI code cleanup

* Fully working G1 ros navigation to origin

* g1 agents2 spatial navigation

* CI code cleanup

* comment out camera image

* fix

* commit

* adapt

* image

* crop image

* switch back to old camera

* sharpness window generalized to quality_barrier

* tests consolidation, preparing for merge

* tests fix

* qwen localization

* nav to object in view

* forgot init

* bugfix

* bugfix

* fix for timestamp on g1

* killing time stuff

* quick fixes

* onboard unitree changes

* CI code cleanup

* moduledb hack

* moduledb hack

* current g1

* quaternion fix

* Re enabled detections, removed go to origin on startup

* Fully working save location and navigate to saved location

* Fully working G1 spatial memory, detections, location saving on agents2

* Fully working G1 webrtc skills integrated as SkillContainer module for agents2

* foxglove vis for 3d localization

* cleaning up detection2d

* added detic

* detic

* pose detector sketch

* restructure

* pose -> person

* person detector sketch

* yolo pose test

* separated detection3d and detection3dpc

* object3d new test

* lcm replay test

* thread cleanup

* added seek for example

* deactivate detic

* detection3d bugfix

* fixing tests 1

* fixing tests

* wavefront explorer smaller costmap for faster testing

* person detector merged with follower spec

* fixing tests, timestamp alignment threading

* people annotations test fix

* removed recorder, moved qewen from dev

* fixing tests

* tests passing now

* indeterministic test fix

* removing temp file

* bugfix

* Update dimos/perception/detection2d/detectors/detic.py

Co-authored-by: Paul Nechifor <paul@nechifor.net>

---------

Co-authored-by: alexlin2 <alex.lin416@outlook.com>
Co-authored-by: alexlin2 <44330195+alexlin2@users.noreply.github.com>
Co-authored-by: Paul Nechifor <paul@nechifor.net>
Co-authored-by: paul-nechifor <1262969+paul-nechifor@users.noreply.github.com>
Co-authored-by: Stash Pomichter <pomichterstash@gmail.com>
Former-commit-id: aff4902 [formerly 66440ab]
Former-commit-id: 4a54ad1
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants