Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add Offloading VIO Plugins to ILLIXR #331

Merged
merged 21 commits into from
May 5, 2022
Merged

Add Offloading VIO Plugins to ILLIXR #331

merged 21 commits into from
May 5, 2022

Conversation

JeffreyZh4ng
Copy link
Contributor

Adding offloading VIO plugins for ILLIXR with two configs to make running those plugins easy.

The following is copy pasted directly from the README.md within the offload_vio directory
In order to run the offloading plugins you need the following extra dependecies

sudo add-apt-repository ppa:ecal/ecal-latest
sudo apt-get update
sudo apt-get install ecal
sudo apt-get install libprotobuf-dev protobuf-compiler

The most simple example is running the offloaded setup on one machine with the server in one terminal
and the device running in a different terminal. Start ILLIXR using the runner with configs/offload-server.yaml
for one terminal and configs/offload-server.yaml on the other terminal. The default configuration within the
config files will run Kimera-VIO on the server and will feed it with the EuRoC dataset from the device

To run more complicated experiment setups where the device and server are not on the same machine,
you will need to setup each machine to be able to send/recieve UDP multicasts. A detailed guide
can be found here

charmoniumQ
charmoniumQ reviewed Apr 8, 2022
View reviewed changes
Copy link
Member

@charmoniumQ charmoniumQ left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice work

configs/native.yaml Outdated Show resolved Hide resolved
offload_vio/README.md Outdated Show resolved Hide resolved
offload_vio/device_tx/plugin.cpp Show resolved Hide resolved
offload_vio/device_rx/plugin.cpp Outdated Show resolved Hide resolved
offload_vio/server_rx/plugin.cpp Outdated Show resolved Hide resolved
offload_vio/server_rx/plugin.cpp Show resolved Hide resolved
offload_vio/server_rx/plugin.cpp Show resolved Hide resolved
offload_vio/server_tx/plugin.cpp Show resolved Hide resolved
@JeffreyZh4ng
Copy link
Contributor Author

Ignore the latest commit. Its hacky ugly code meant to try to solve an odd bug

@JeffreyZh4ng JeffreyZh4ng force-pushed the feat-offloading-release branch 3 times, most recently from 1a562a2 to c968ca3 Compare May 3, 2022 09:50
mhuzai
mhuzai reviewed May 4, 2022
View reviewed changes
Copy link
Member

@mhuzai mhuzai left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wasn't able to test the code today. I will do that tomorrow.

configs/offload-server.yaml Outdated Show resolved Hide resolved
docs/offload_vio.md Outdated Show resolved Hide resolved
docs/offload_vio.md Show resolved Hide resolved
offload_vio/device_rx/Makefile Show resolved Hide resolved
offload_vio/device_rx/vio_output.proto Outdated Show resolved Hide resolved
offload_vio/device_tx/vio_input.proto Show resolved Hide resolved
offload_vio/device_tx/plugin.cpp Show resolved Hide resolved
offload_vio/server_rx/plugin.cpp Show resolved Hide resolved
offload_vio/server_rx/plugin.cpp Outdated Show resolved Hide resolved
offload_vio/server_tx/plugin.cpp Outdated Show resolved Hide resolved
mhuzai
mhuzai reviewed May 5, 2022
View reviewed changes
docs/offload_vio.md Outdated Show resolved Hide resolved
mhuzai
mhuzai approved these changes May 5, 2022
View reviewed changes
Copy link
Member

@mhuzai mhuzai left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Running this plugin felt like magic. :D

@charmoniumQ charmoniumQ self-requested a review May 5, 2022 07:02
@mhuzai mhuzai merged commit 62433d6 into master May 5, 2022
mhuzai added a commit that referenced this pull request May 9, 2022
@charmoniumQ @e3m3
@qinjunj @JeffreyZh4ng @mhuzai @hungdche @yingj4
Standardize time in ILLIXR (#250)
5a16aef 
* Correct configs

* Fix sqlite

* Exorcise glfw3

* Rename classes

* Exorcise glfw3 (electric boogaloo)

* Respond to reviewer comments

* Respond to reviewer comments

* Remove dataset_time

* Added '_m_dataset_first_time' in 'ground_truth_slam/plugin.cpp' (#58)

* disable debugview

* add condition

* remove -Werror in OPT-FLAGS

* copy common/debug.hpp from the scheduling branch

* add declaration of NANO_SEC

* use local monado

* remove NANO_SEC

* add absolute_ns but has error with +

* convert _m_start and relative to ns before adding them

* change name to passthrough_integrator and zeor out matrice

* comment conditions

* removed dummy_integrator, changed to passthrough_integrator

* tests

* Started time_point

* create our own time_point

* use issue-58-standardize-clock branch in monado_integration

* change back to size_t for imu_data loop

* fix naming and spacing

* remove repeated synchronization

* enable debugview

* add back glfw3 for debugview

* Added documentation for using Switchboard and Phonebook externally (#320)

* Added documentation for using Switchboard and Phonebook externally

* fix typo and spacing

* Fix typos

Co-authored-by: Qinjun Jiang <90299267+qinjunj@users.noreply.github.com>

* added new line at EOF

* use offload_duration to be less confusing

* remove dummy

* change back to size_t

* update comments

* synchronize after thread setup

* synchronize before p_thread_setup

* Fix and add documentation (#330)

* Add documentation for updating tags and documentation

* Set up using master instead of v2-latest

* Update doxygen project name

* Update Monado's main branch (they renamed 'master' to 'main')

* Fix typo in mkdocs.yml

* Update README with IISWC paper info

* Update bibtex

Co-authored-by: Sam Grayson <sam@samgrayson.me>

* Fix bibtex

Co-authored-by: Sam Grayson <sam@samgrayson.me>

* reponses to comments

* remove trailing space

* Add IMU and Cam Recorder (#327)

* Added record_imu_cam plugin

* Fixing fstream precision

* Adding data_record option for native.yaml

* Adding record_imu_cam plugin option

* Remove data_record.

* Fix minor issues

* Fix minor issues

* Used boost::filesystem instead of system

* Add lboost_filesystem flag

* Add the description for record_imu_cam

* Fix record_imu_cam path

* Fix boost filesystem operator

* Remove extra /

* Fix minor issues

* compute data paths using list initialization

* Fix minor issues and clean up extra variables

Co-authored-by: yingj4 <yingj4@illinois.edu>

* remove pose_lookup

* able to compile

* restore

* Add stoplight to synchronize thread starts/stops

* Add missing file (the perils of -u instead of -A)

* Respond to reviewer comments

* Add documentation and comments

* Respond to reviewer comments

* Respond to reviewer comments

* Fix compile bugs

* Add stoplight to synchronize thread starts/stops

* Add relative clock to common and runtime

* Use relative clock in offline_imu_cam

* Use relative clock in gldemo (exorcise GLFW)

* Use relative clock in pose_*

* Use relative clock in timewarp

* wip

* Use relative clock in integrators

* Use relative clock in offload_data

* Use relative clock in realsense and ZED

* Fix stuff

* wip

* Fix stuff

* wip

* Fix integrators

* wip (ov + dummy int works)

* wip (ov + rk4 works but its shakey)

* wip (ov + gtsam works)

* Correct configs

* Fix sqlite

* Exorcise glfw3

* Rename classes

* Exorcise glfw3 (electric boogaloo)

* Respond to reviewer comments

* Respond to reviewer comments

* Remove dataset_time

* Added '_m_dataset_first_time' in 'ground_truth_slam/plugin.cpp' (#58)

* disable debugview

* add condition

* remove -Werror in OPT-FLAGS

* copy common/debug.hpp from the scheduling branch

* add declaration of NANO_SEC

* use local monado

* remove NANO_SEC

* add absolute_ns but has error with +

* convert _m_start and relative to ns before adding them

* change name to passthrough_integrator and zeor out matrice

* comment conditions

* removed dummy_integrator, changed to passthrough_integrator

* tests

* Started time_point

* create our own time_point

* use issue-58-standardize-clock branch in monado_integration

* change back to size_t for imu_data loop

* fix naming and spacing

* remove repeated synchronization

* enable debugview

* add back glfw3 for debugview

* added new line at EOF

* use offload_duration to be less confusing

* remove dummy

* change back to size_t

* update comments

* synchronize after thread setup

* synchronize before p_thread_setup

* reponses to comments

* remove trailing space

* remove pose_lookup

* able to compile

* restore

* remove dummy integrator

* correct order of declaring

* suggested change to replace dataset_time

* Add Offloading VIO Plugins to ILLIXR (#331)

* Semi working with reader using callbacks

* Semi working with reader using sequential reads

* Semi working offloading. VIO doesnt work ATM

* test commit

* Add offloading server and device code

* Fixing common and adding extra prints for debugging

* Add prints and offloading config

* Server fix for zed pt1

* Offloading device cleanup

* Add offload client config

* Renamed files and merge master

* Cleaned up code ready for PR

* Fix readme

* Revert hotfix and address comments

* Revert rt_slam config

* Minor fix and verify all combinations of ZED/dataset with OV/Kimera work

* Address comments, add plugins to clean.yaml

* Add nproc back and update docs

* Typo

* Fix RealSense plugin (#337)

* Fix counter initialization and tidy up some code

* Properly initialize camera iteration count

* Add stoplight to synchronize thread starts/stops

* Add missing file (the perils of -u instead of -A)

* Respond to reviewer comments

* Add documentation and comments

* Respond to reviewer comments

* Respond to reviewer comments

* Fix compile bugs

* Add stoplight to synchronize thread starts/stops

* Add relative clock to common and runtime

* Use relative clock in offline_imu_cam

* Use relative clock in gldemo (exorcise GLFW)

* Use relative clock in pose_*

* Use relative clock in timewarp

* wip

* Use relative clock in integrators

* Use relative clock in offload_data

* Use relative clock in realsense and ZED

* Fix stuff

* wip

* Fix stuff

* wip

* Fix integrators

* wip (ov + dummy int works)

* wip (ov + rk4 works but its shakey)

* wip (ov + gtsam works)

* Correct configs

* Fix sqlite

* Exorcise glfw3

* Rename classes

* Exorcise glfw3 (electric boogaloo)

* Respond to reviewer comments

* Respond to reviewer comments

* Remove dataset_time

* Added '_m_dataset_first_time' in 'ground_truth_slam/plugin.cpp' (#58)

* disable debugview

* add condition

* remove -Werror in OPT-FLAGS

* copy common/debug.hpp from the scheduling branch

* add declaration of NANO_SEC

* use local monado

* remove NANO_SEC

* add absolute_ns but has error with +

* convert _m_start and relative to ns before adding them

* change name to passthrough_integrator and zeor out matrice

* comment conditions

* removed dummy_integrator, changed to passthrough_integrator

* tests

* Started time_point

* create our own time_point

* use issue-58-standardize-clock branch in monado_integration

* change back to size_t for imu_data loop

* fix naming and spacing

* remove repeated synchronization

* enable debugview

* add back glfw3 for debugview

* added new line at EOF

* use offload_duration to be less confusing

* remove dummy

* change back to size_t

* update comments

* synchronize after thread setup

* synchronize before p_thread_setup

* reponses to comments

* remove trailing space

* remove pose_lookup

* able to compile

* restore

* Add stoplight to synchronize thread starts/stops

* Add missing file (the perils of -u instead of -A)

* Respond to reviewer comments

* Add documentation and comments

* Respond to reviewer comments

* Respond to reviewer comments

* Fix compile bugs

* Add stoplight to synchronize thread starts/stops

* Add relative clock to common and runtime

* Use relative clock in offline_imu_cam

* Use relative clock in gldemo (exorcise GLFW)

* Use relative clock in pose_*

* Use relative clock in timewarp

* Use relative clock in realsense and ZED

* Fix stuff

* wip

* Fix stuff

* wip

* Fix integrators

* wip (ov + dummy int works)

* wip (ov + gtsam works)

* Wip (works with gtsam)

* Correct configs

* Respond to reviewer comments

* Respond to reviewer comments

* Remove dataset_time

* disable debugview

* add declaration of NANO_SEC

* remove NANO_SEC

* removed dummy_integrator, changed to passthrough_integrator

* tests

* Started time_point

* create our own time_point

* change back to size_t for imu_data loop

* enable debugview

* add back glfw3 for debugview

* remove dummy

* change back to size_t

* synchronize after thread setup

* synchronize before p_thread_setup

* reponses to comments

* remove trailing space

* remove pose_lookup

* able to compile

* restore

* suggested change to replace dataset_time

* respond to comments

* use relative clock in offload_vio

* respond to comments

* update distribution checking and warning msg for realsense

* Fix device_rx compilation

* Fix offload server config

* Resolve pending review comments

* update empty constructor of pose_type

* Update OV tag

* Update Kimera, OV, and Monado tags to reflect the usage of RelativeClock

* Update tags in offload-server

Co-authored-by: Giordano Salvador <73959795+e3m3@users.noreply.github.com>
Co-authored-by: Qinjun Jiang <qinjunj2@illinois.edu>
Co-authored-by: Jeffrey Zhang <23205149+JeffreyZh4ng@users.noreply.github.com>
Co-authored-by: Qinjun Jiang <90299267+qinjunj@users.noreply.github.com>
Co-authored-by: Muhammad Huzaifa <mhuzai@users.noreply.github.com>
Co-authored-by: Henry Che <90811347+hungdche@users.noreply.github.com>
Co-authored-by: yingj4 <yingj4@illinois.edu>
Co-authored-by: Muhammad Huzaifa <huzaifa2@illinois.edu>
shg8 pushed a commit to shg8/ILLIXR that referenced this pull request May 18, 2022
@JeffreyZh4ng @shg8
Add Offloading VIO Plugins to ILLIXR (ILLIXR#331)
b553d32 
* Semi working with reader using callbacks

* Semi working with reader using sequential reads

* Semi working offloading. VIO doesnt work ATM

* test commit

* Add offloading server and device code

* Fixing common and adding extra prints for debugging

* Add prints and offloading config

* Server fix for zed pt1

* Offloading device cleanup

* Add offload client config

* Renamed files and merge master

* Cleaned up code ready for PR

* Fix readme

* Revert hotfix and address comments

* Revert rt_slam config

* Minor fix and verify all combinations of ZED/dataset with OV/Kimera work

* Address comments, add plugins to clean.yaml

* Add nproc back and update docs

* Typo
shg8 pushed a commit to shg8/ILLIXR that referenced this pull request May 18, 2022
@charmoniumQ @e3m3
@qinjunj @JeffreyZh4ng @mhuzai @hungdche @yingj4 @shg8
Standardize time in ILLIXR (ILLIXR#250)
98d64de 
* Correct configs

* Fix sqlite

* Exorcise glfw3

* Rename classes

* Exorcise glfw3 (electric boogaloo)

* Respond to reviewer comments

* Respond to reviewer comments

* Remove dataset_time

* Added '_m_dataset_first_time' in 'ground_truth_slam/plugin.cpp' (ILLIXR#58)

* disable debugview

* add condition

* remove -Werror in OPT-FLAGS

* copy common/debug.hpp from the scheduling branch

* add declaration of NANO_SEC

* use local monado

* remove NANO_SEC

* add absolute_ns but has error with +

* convert _m_start and relative to ns before adding them

* change name to passthrough_integrator and zeor out matrice

* comment conditions

* removed dummy_integrator, changed to passthrough_integrator

* tests

* Started time_point

* create our own time_point

* use issue-58-standardize-clock branch in monado_integration

* change back to size_t for imu_data loop

* fix naming and spacing

* remove repeated synchronization

* enable debugview

* add back glfw3 for debugview

* Added documentation for using Switchboard and Phonebook externally (ILLIXR#320)

* Added documentation for using Switchboard and Phonebook externally

* fix typo and spacing

* Fix typos

Co-authored-by: Qinjun Jiang <90299267+qinjunj@users.noreply.github.com>

* added new line at EOF

* use offload_duration to be less confusing

* remove dummy

* change back to size_t

* update comments

* synchronize after thread setup

* synchronize before p_thread_setup

* Fix and add documentation (ILLIXR#330)

* Add documentation for updating tags and documentation

* Set up using master instead of v2-latest

* Update doxygen project name

* Update Monado's main branch (they renamed 'master' to 'main')

* Fix typo in mkdocs.yml

* Update README with IISWC paper info

* Update bibtex

Co-authored-by: Sam Grayson <sam@samgrayson.me>

* Fix bibtex

Co-authored-by: Sam Grayson <sam@samgrayson.me>

* reponses to comments

* remove trailing space

* Add IMU and Cam Recorder (ILLIXR#327)

* Added record_imu_cam plugin

* Fixing fstream precision

* Adding data_record option for native.yaml

* Adding record_imu_cam plugin option

* Remove data_record.

* Fix minor issues

* Fix minor issues

* Used boost::filesystem instead of system

* Add lboost_filesystem flag

* Add the description for record_imu_cam

* Fix record_imu_cam path

* Fix boost filesystem operator

* Remove extra /

* Fix minor issues

* compute data paths using list initialization

* Fix minor issues and clean up extra variables

Co-authored-by: yingj4 <yingj4@illinois.edu>

* remove pose_lookup

* able to compile

* restore

* Add stoplight to synchronize thread starts/stops

* Add missing file (the perils of -u instead of -A)

* Respond to reviewer comments

* Add documentation and comments

* Respond to reviewer comments

* Respond to reviewer comments

* Fix compile bugs

* Add stoplight to synchronize thread starts/stops

* Add relative clock to common and runtime

* Use relative clock in offline_imu_cam

* Use relative clock in gldemo (exorcise GLFW)

* Use relative clock in pose_*

* Use relative clock in timewarp

* wip

* Use relative clock in integrators

* Use relative clock in offload_data

* Use relative clock in realsense and ZED

* Fix stuff

* wip

* Fix stuff

* wip

* Fix integrators

* wip (ov + dummy int works)

* wip (ov + rk4 works but its shakey)

* wip (ov + gtsam works)

* Correct configs

* Fix sqlite

* Exorcise glfw3

* Rename classes

* Exorcise glfw3 (electric boogaloo)

* Respond to reviewer comments

* Respond to reviewer comments

* Remove dataset_time

* Added '_m_dataset_first_time' in 'ground_truth_slam/plugin.cpp' (ILLIXR#58)

* disable debugview

* add condition

* remove -Werror in OPT-FLAGS

* copy common/debug.hpp from the scheduling branch

* add declaration of NANO_SEC

* use local monado

* remove NANO_SEC

* add absolute_ns but has error with +

* convert _m_start and relative to ns before adding them

* change name to passthrough_integrator and zeor out matrice

* comment conditions

* removed dummy_integrator, changed to passthrough_integrator

* tests

* Started time_point

* create our own time_point

* use issue-58-standardize-clock branch in monado_integration

* change back to size_t for imu_data loop

* fix naming and spacing

* remove repeated synchronization

* enable debugview

* add back glfw3 for debugview

* added new line at EOF

* use offload_duration to be less confusing

* remove dummy

* change back to size_t

* update comments

* synchronize after thread setup

* synchronize before p_thread_setup

* reponses to comments

* remove trailing space

* remove pose_lookup

* able to compile

* restore

* remove dummy integrator

* correct order of declaring

* suggested change to replace dataset_time

* Add Offloading VIO Plugins to ILLIXR (ILLIXR#331)

* Semi working with reader using callbacks

* Semi working with reader using sequential reads

* Semi working offloading. VIO doesnt work ATM

* test commit

* Add offloading server and device code

* Fixing common and adding extra prints for debugging

* Add prints and offloading config

* Server fix for zed pt1

* Offloading device cleanup

* Add offload client config

* Renamed files and merge master

* Cleaned up code ready for PR

* Fix readme

* Revert hotfix and address comments

* Revert rt_slam config

* Minor fix and verify all combinations of ZED/dataset with OV/Kimera work

* Address comments, add plugins to clean.yaml

* Add nproc back and update docs

* Typo

* Fix RealSense plugin (ILLIXR#337)

* Fix counter initialization and tidy up some code

* Properly initialize camera iteration count

* Add stoplight to synchronize thread starts/stops

* Add missing file (the perils of -u instead of -A)

* Respond to reviewer comments

* Add documentation and comments

* Respond to reviewer comments

* Respond to reviewer comments

* Fix compile bugs

* Add stoplight to synchronize thread starts/stops

* Add relative clock to common and runtime

* Use relative clock in offline_imu_cam

* Use relative clock in gldemo (exorcise GLFW)

* Use relative clock in pose_*

* Use relative clock in timewarp

* wip

* Use relative clock in integrators

* Use relative clock in offload_data

* Use relative clock in realsense and ZED

* Fix stuff

* wip

* Fix stuff

* wip

* Fix integrators

* wip (ov + dummy int works)

* wip (ov + rk4 works but its shakey)

* wip (ov + gtsam works)

* Correct configs

* Fix sqlite

* Exorcise glfw3

* Rename classes

* Exorcise glfw3 (electric boogaloo)

* Respond to reviewer comments

* Respond to reviewer comments

* Remove dataset_time

* Added '_m_dataset_first_time' in 'ground_truth_slam/plugin.cpp' (ILLIXR#58)

* disable debugview

* add condition

* remove -Werror in OPT-FLAGS

* copy common/debug.hpp from the scheduling branch

* add declaration of NANO_SEC

* use local monado

* remove NANO_SEC

* add absolute_ns but has error with +

* convert _m_start and relative to ns before adding them

* change name to passthrough_integrator and zeor out matrice

* comment conditions

* removed dummy_integrator, changed to passthrough_integrator

* tests

* Started time_point

* create our own time_point

* use issue-58-standardize-clock branch in monado_integration

* change back to size_t for imu_data loop

* fix naming and spacing

* remove repeated synchronization

* enable debugview

* add back glfw3 for debugview

* added new line at EOF

* use offload_duration to be less confusing

* remove dummy

* change back to size_t

* update comments

* synchronize after thread setup

* synchronize before p_thread_setup

* reponses to comments

* remove trailing space

* remove pose_lookup

* able to compile

* restore

* Add stoplight to synchronize thread starts/stops

* Add missing file (the perils of -u instead of -A)

* Respond to reviewer comments

* Add documentation and comments

* Respond to reviewer comments

* Respond to reviewer comments

* Fix compile bugs

* Add stoplight to synchronize thread starts/stops

* Add relative clock to common and runtime

* Use relative clock in offline_imu_cam

* Use relative clock in gldemo (exorcise GLFW)

* Use relative clock in pose_*

* Use relative clock in timewarp

* Use relative clock in realsense and ZED

* Fix stuff

* wip

* Fix stuff

* wip

* Fix integrators

* wip (ov + dummy int works)

* wip (ov + gtsam works)

* Wip (works with gtsam)

* Correct configs

* Respond to reviewer comments

* Respond to reviewer comments

* Remove dataset_time

* disable debugview

* add declaration of NANO_SEC

* remove NANO_SEC

* removed dummy_integrator, changed to passthrough_integrator

* tests

* Started time_point

* create our own time_point

* change back to size_t for imu_data loop

* enable debugview

* add back glfw3 for debugview

* remove dummy

* change back to size_t

* synchronize after thread setup

* synchronize before p_thread_setup

* reponses to comments

* remove trailing space

* remove pose_lookup

* able to compile

* restore

* suggested change to replace dataset_time

* respond to comments

* use relative clock in offload_vio

* respond to comments

* update distribution checking and warning msg for realsense

* Fix device_rx compilation

* Fix offload server config

* Resolve pending review comments

* update empty constructor of pose_type

* Update OV tag

* Update Kimera, OV, and Monado tags to reflect the usage of RelativeClock

* Update tags in offload-server

Co-authored-by: Giordano Salvador <73959795+e3m3@users.noreply.github.com>
Co-authored-by: Qinjun Jiang <qinjunj2@illinois.edu>
Co-authored-by: Jeffrey Zhang <23205149+JeffreyZh4ng@users.noreply.github.com>
Co-authored-by: Qinjun Jiang <90299267+qinjunj@users.noreply.github.com>
Co-authored-by: Muhammad Huzaifa <mhuzai@users.noreply.github.com>
Co-authored-by: Henry Che <90811347+hungdche@users.noreply.github.com>
Co-authored-by: yingj4 <yingj4@illinois.edu>
Co-authored-by: Muhammad Huzaifa <huzaifa2@illinois.edu>
shg8 pushed a commit to shg8/ILLIXR that referenced this pull request May 18, 2022
@JeffreyZh4ng @shg8
Add Offloading VIO Plugins to ILLIXR (ILLIXR#331)
3fa8ea9 
* Semi working with reader using callbacks

* Semi working with reader using sequential reads

* Semi working offloading. VIO doesnt work ATM

* test commit

* Add offloading server and device code

* Fixing common and adding extra prints for debugging

* Add prints and offloading config

* Server fix for zed pt1

* Offloading device cleanup

* Add offload client config

* Renamed files and merge master

* Cleaned up code ready for PR

* Fix readme

* Revert hotfix and address comments

* Revert rt_slam config

* Minor fix and verify all combinations of ZED/dataset with OV/Kimera work

* Address comments, add plugins to clean.yaml

* Add nproc back and update docs

* Typo
shg8 pushed a commit to shg8/ILLIXR that referenced this pull request May 18, 2022
@charmoniumQ @e3m3
@qinjunj @JeffreyZh4ng @mhuzai @hungdche @yingj4 @shg8
Standardize time in ILLIXR (ILLIXR#250)
eb314f1 
* Correct configs

* Fix sqlite

* Exorcise glfw3

* Rename classes

* Exorcise glfw3 (electric boogaloo)

* Respond to reviewer comments

* Respond to reviewer comments

* Remove dataset_time

* Added '_m_dataset_first_time' in 'ground_truth_slam/plugin.cpp' (ILLIXR#58)

* disable debugview

* add condition

* remove -Werror in OPT-FLAGS

* copy common/debug.hpp from the scheduling branch

* add declaration of NANO_SEC

* use local monado

* remove NANO_SEC

* add absolute_ns but has error with +

* convert _m_start and relative to ns before adding them

* change name to passthrough_integrator and zeor out matrice

* comment conditions

* removed dummy_integrator, changed to passthrough_integrator

* tests

* Started time_point

* create our own time_point

* use issue-58-standardize-clock branch in monado_integration

* change back to size_t for imu_data loop

* fix naming and spacing

* remove repeated synchronization

* enable debugview

* add back glfw3 for debugview

* Added documentation for using Switchboard and Phonebook externally (ILLIXR#320)

* Added documentation for using Switchboard and Phonebook externally

* fix typo and spacing

* Fix typos

Co-authored-by: Qinjun Jiang <90299267+qinjunj@users.noreply.github.com>

* added new line at EOF

* use offload_duration to be less confusing

* remove dummy

* change back to size_t

* update comments

* synchronize after thread setup

* synchronize before p_thread_setup

* Fix and add documentation (ILLIXR#330)

* Add documentation for updating tags and documentation

* Set up using master instead of v2-latest

* Update doxygen project name

* Update Monado's main branch (they renamed 'master' to 'main')

* Fix typo in mkdocs.yml

* Update README with IISWC paper info

* Update bibtex

Co-authored-by: Sam Grayson <sam@samgrayson.me>

* Fix bibtex

Co-authored-by: Sam Grayson <sam@samgrayson.me>

* reponses to comments

* remove trailing space

* Add IMU and Cam Recorder (ILLIXR#327)

* Added record_imu_cam plugin

* Fixing fstream precision

* Adding data_record option for native.yaml

* Adding record_imu_cam plugin option

* Remove data_record.

* Fix minor issues

* Fix minor issues

* Used boost::filesystem instead of system

* Add lboost_filesystem flag

* Add the description for record_imu_cam

* Fix record_imu_cam path

* Fix boost filesystem operator

* Remove extra /

* Fix minor issues

* compute data paths using list initialization

* Fix minor issues and clean up extra variables

Co-authored-by: yingj4 <yingj4@illinois.edu>

* remove pose_lookup

* able to compile

* restore

* Add stoplight to synchronize thread starts/stops

* Add missing file (the perils of -u instead of -A)

* Respond to reviewer comments

* Add documentation and comments

* Respond to reviewer comments

* Respond to reviewer comments

* Fix compile bugs

* Add stoplight to synchronize thread starts/stops

* Add relative clock to common and runtime

* Use relative clock in offline_imu_cam

* Use relative clock in gldemo (exorcise GLFW)

* Use relative clock in pose_*

* Use relative clock in timewarp

* wip

* Use relative clock in integrators

* Use relative clock in offload_data

* Use relative clock in realsense and ZED

* Fix stuff

* wip

* Fix stuff

* wip

* Fix integrators

* wip (ov + dummy int works)

* wip (ov + rk4 works but its shakey)

* wip (ov + gtsam works)

* Correct configs

* Fix sqlite

* Exorcise glfw3

* Rename classes

* Exorcise glfw3 (electric boogaloo)

* Respond to reviewer comments

* Respond to reviewer comments

* Remove dataset_time

* Added '_m_dataset_first_time' in 'ground_truth_slam/plugin.cpp' (ILLIXR#58)

* disable debugview

* add condition

* remove -Werror in OPT-FLAGS

* copy common/debug.hpp from the scheduling branch

* add declaration of NANO_SEC

* use local monado

* remove NANO_SEC

* add absolute_ns but has error with +

* convert _m_start and relative to ns before adding them

* change name to passthrough_integrator and zeor out matrice

* comment conditions

* removed dummy_integrator, changed to passthrough_integrator

* tests

* Started time_point

* create our own time_point

* use issue-58-standardize-clock branch in monado_integration

* change back to size_t for imu_data loop

* fix naming and spacing

* remove repeated synchronization

* enable debugview

* add back glfw3 for debugview

* added new line at EOF

* use offload_duration to be less confusing

* remove dummy

* change back to size_t

* update comments

* synchronize after thread setup

* synchronize before p_thread_setup

* reponses to comments

* remove trailing space

* remove pose_lookup

* able to compile

* restore

* remove dummy integrator

* correct order of declaring

* suggested change to replace dataset_time

* Add Offloading VIO Plugins to ILLIXR (ILLIXR#331)

* Semi working with reader using callbacks

* Semi working with reader using sequential reads

* Semi working offloading. VIO doesnt work ATM

* test commit

* Add offloading server and device code

* Fixing common and adding extra prints for debugging

* Add prints and offloading config

* Server fix for zed pt1

* Offloading device cleanup

* Add offload client config

* Renamed files and merge master

* Cleaned up code ready for PR

* Fix readme

* Revert hotfix and address comments

* Revert rt_slam config

* Minor fix and verify all combinations of ZED/dataset with OV/Kimera work

* Address comments, add plugins to clean.yaml

* Add nproc back and update docs

* Typo

* Fix RealSense plugin (ILLIXR#337)

* Fix counter initialization and tidy up some code

* Properly initialize camera iteration count

* Add stoplight to synchronize thread starts/stops

* Add missing file (the perils of -u instead of -A)

* Respond to reviewer comments

* Add documentation and comments

* Respond to reviewer comments

* Respond to reviewer comments

* Fix compile bugs

* Add stoplight to synchronize thread starts/stops

* Add relative clock to common and runtime

* Use relative clock in offline_imu_cam

* Use relative clock in gldemo (exorcise GLFW)

* Use relative clock in pose_*

* Use relative clock in timewarp

* wip

* Use relative clock in integrators

* Use relative clock in offload_data

* Use relative clock in realsense and ZED

* Fix stuff

* wip

* Fix stuff

* wip

* Fix integrators

* wip (ov + dummy int works)

* wip (ov + rk4 works but its shakey)

* wip (ov + gtsam works)

* Correct configs

* Fix sqlite

* Exorcise glfw3

* Rename classes

* Exorcise glfw3 (electric boogaloo)

* Respond to reviewer comments

* Respond to reviewer comments

* Remove dataset_time

* Added '_m_dataset_first_time' in 'ground_truth_slam/plugin.cpp' (ILLIXR#58)

* disable debugview

* add condition

* remove -Werror in OPT-FLAGS

* copy common/debug.hpp from the scheduling branch

* add declaration of NANO_SEC

* use local monado

* remove NANO_SEC

* add absolute_ns but has error with +

* convert _m_start and relative to ns before adding them

* change name to passthrough_integrator and zeor out matrice

* comment conditions

* removed dummy_integrator, changed to passthrough_integrator

* tests

* Started time_point

* create our own time_point

* use issue-58-standardize-clock branch in monado_integration

* change back to size_t for imu_data loop

* fix naming and spacing

* remove repeated synchronization

* enable debugview

* add back glfw3 for debugview

* added new line at EOF

* use offload_duration to be less confusing

* remove dummy

* change back to size_t

* update comments

* synchronize after thread setup

* synchronize before p_thread_setup

* reponses to comments

* remove trailing space

* remove pose_lookup

* able to compile

* restore

* Add stoplight to synchronize thread starts/stops

* Add missing file (the perils of -u instead of -A)

* Respond to reviewer comments

* Add documentation and comments

* Respond to reviewer comments

* Respond to reviewer comments

* Fix compile bugs

* Add stoplight to synchronize thread starts/stops

* Add relative clock to common and runtime

* Use relative clock in offline_imu_cam

* Use relative clock in gldemo (exorcise GLFW)

* Use relative clock in pose_*

* Use relative clock in timewarp

* Use relative clock in realsense and ZED

* Fix stuff

* wip

* Fix stuff

* wip

* Fix integrators

* wip (ov + dummy int works)

* wip (ov + gtsam works)

* Wip (works with gtsam)

* Correct configs

* Respond to reviewer comments

* Respond to reviewer comments

* Remove dataset_time

* disable debugview

* add declaration of NANO_SEC

* remove NANO_SEC

* removed dummy_integrator, changed to passthrough_integrator

* tests

* Started time_point

* create our own time_point

* change back to size_t for imu_data loop

* enable debugview

* add back glfw3 for debugview

* remove dummy

* change back to size_t

* synchronize after thread setup

* synchronize before p_thread_setup

* reponses to comments

* remove trailing space

* remove pose_lookup

* able to compile

* restore

* suggested change to replace dataset_time

* respond to comments

* use relative clock in offload_vio

* respond to comments

* update distribution checking and warning msg for realsense

* Fix device_rx compilation

* Fix offload server config

* Resolve pending review comments

* update empty constructor of pose_type

* Update OV tag

* Update Kimera, OV, and Monado tags to reflect the usage of RelativeClock

* Update tags in offload-server

Co-authored-by: Giordano Salvador <73959795+e3m3@users.noreply.github.com>
Co-authored-by: Qinjun Jiang <qinjunj2@illinois.edu>
Co-authored-by: Jeffrey Zhang <23205149+JeffreyZh4ng@users.noreply.github.com>
Co-authored-by: Qinjun Jiang <90299267+qinjunj@users.noreply.github.com>
Co-authored-by: Muhammad Huzaifa <mhuzai@users.noreply.github.com>
Co-authored-by: Henry Che <90811347+hungdche@users.noreply.github.com>
Co-authored-by: yingj4 <yingj4@illinois.edu>
Co-authored-by: Muhammad Huzaifa <huzaifa2@illinois.edu>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@charmoniumQ charmoniumQ charmoniumQ approved these changes

@mhuzai mhuzai mhuzai approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
3 participants
@JeffreyZh4ng @charmoniumQ @mhuzai