Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use ref-counting in Switchboard (patch memory leak) #32

Merged
merged 38 commits into from
Apr 20, 2021

Conversation

charmoniumQ
Copy link
Member

@charmoniumQ charmoniumQ commented Sep 15, 2020

  1. Reclaim memory of events which are passed through Switchboard.In a producer-consumer relationship with multiple consumers, which is responsible for deleting memory? I added reference counting (on every Switchboard event) to decide that. This involves changing the types around a bit, but the interface is largely the same. See the end of common/tests/test_switchboard.hpp for verification).
  2. Run synchronous callbacks in parallel.
  3. Use blocking instead of spinning.
  4. Don't require events to inherit from switchboard::event.

@charmoniumQ charmoniumQ created this issue from a note in ILLIXR improvements (In progress) Jul 26, 2020
@charmoniumQ charmoniumQ self-assigned this Jul 26, 2020
@charmoniumQ charmoniumQ added this to the 2.1 milestone Jul 26, 2020
@charmoniumQ charmoniumQ changed the title Patch memory leak in Switchboard Use ref-counting in Switchboard (patch memory leak) Jul 26, 2020
@charmoniumQ charmoniumQ added the bug Something isn't working label Jul 26, 2020
@charmoniumQ charmoniumQ moved this from In progress to Review in progress in ILLIXR improvements Aug 16, 2020
@charmoniumQ charmoniumQ moved this from Review in progress to In progress in ILLIXR improvements Aug 16, 2020
@charmoniumQ charmoniumQ moved this from In progress to High priority in ILLIXR improvements Aug 19, 2020
@mhuzai mhuzai modified the milestones: 2.1, 2.2 Sep 6, 2020
@charmoniumQ charmoniumQ added the WIP Work in progress label Sep 15, 2020
@charmoniumQ charmoniumQ marked this pull request as draft September 15, 2020 05:30
@charmoniumQ charmoniumQ added WIP Work in progress and removed WIP Work in progress labels Sep 15, 2020
@charmoniumQ charmoniumQ modified the milestones: 2.2, 2.3 Sep 15, 2020
@Zee2 Zee2 moved this from High priority to In progress in ILLIXR improvements Sep 19, 2020
@charmoniumQ charmoniumQ marked this pull request as ready for review October 2, 2020 00:29
Copy link
Member

@mhuzai mhuzai left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Couldn't go through the whole thing in detail, but left a few comments. One high level feedback that I have is that this PR should update all of the switchboard and plugin documentation (writing a new plugin, etc.).

Thread safety looks good (only skimmed through it). However, I do miss the "proof of thread safety" comment blocks that you used to have.
Didn't get a chance to review memory safety though, so that's something other reviewers should focus on.

One high level question about the thread structure: as many threads as synchronous topics, right?

ground_truth_slam/plugin.cpp Outdated Show resolved Hide resolved
pose_prediction/plugin.cpp Outdated Show resolved Hide resolved
pose_prediction/plugin.cpp Outdated Show resolved Hide resolved
gldemo/plugin.cpp Outdated Show resolved Hide resolved
pose_prediction/plugin.cpp Outdated Show resolved Hide resolved
common/switchboard.hpp Outdated Show resolved Hide resolved
timewarp_gl/plugin.cpp Outdated Show resolved Hide resolved
common/switchboard.hpp Outdated Show resolved Hide resolved
common/switchboard.hpp Outdated Show resolved Hide resolved
common/switchboard.hpp Outdated Show resolved Hide resolved
common/managed_thread.hpp Show resolved Hide resolved
common/record_logger.hpp Show resolved Hide resolved
common/switchboard.hpp Outdated Show resolved Hide resolved
common/tests/test_switchboard.cpp Show resolved Hide resolved
pose_prediction/plugin.cpp Show resolved Hide resolved
pose_prediction/plugin.cpp Outdated Show resolved Hide resolved
pose_prediction/plugin.cpp Outdated Show resolved Hide resolved
pose_lookup/plugin.cpp Outdated Show resolved Hide resolved
e3m3
e3m3 previously approved these changes Dec 4, 2020
Copy link
Contributor

@e3m3 e3m3 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There were some old unresolved conversations from a previous review, but I think you are addressing these later.

configs/native.yaml Outdated Show resolved Hide resolved
gtsam_integrator/common Outdated Show resolved Hide resolved
e3m3 added a commit that referenced this pull request Jan 15, 2021
ILLIXR improvements automation moved this from In progress to Review in progress Jan 15, 2021
e3m3 added a commit that referenced this pull request Jan 15, 2021
e3m3
e3m3 previously approved these changes Feb 1, 2021
e3m3 added a commit that referenced this pull request Feb 5, 2021
e3m3 and others added 17 commits April 15, 2021 21:37
- New 'plugin_id_t' type definition
- Fixed Switchboard logging
- Added pre-sleep feature for debugging with gdb
- Updated inheritance in 'common/data_format.hpp'
- Updated signatures in 'common/plugin.hpp'
- Moved 'configs/ci.yaml' plugin order to match 'configs/native.yaml'
- New preintegration handler in 'gtsam_integrator/plugin.cpp'
- Reformatted 'gtsam_integrator/plugin.cpp'
- Changed assertion to RAC in 'runtime/sqlite_record_logger.hpp'
- Updated branch and symlinks in 'scripts/install_gtsam.sh'
- Removed '-fsanitize' from 'common/common.mk'
- Fixed nullptr check in 'offload_data/plugin.cpp'
- Added 'maybe_unused' to constexpr in 'runtime/main.cpp'
- Fixed compilation for 'zed/src/main.cpp'

Co-authored by Samuel Grayson <sam@samgrayson.me>
- Added comments to 'zed/CMakeLists.txxt'
- Changed most topic `put` calls to use in-place calls to `allocate`
- Removed 'get' alias for 'get_ro' in 'common/switchboard.hpp'
- Updated 'put' example in 'common/switchboard.hpp'
- Fixed version tags in 'configs/*'
- Added back 'groud_truth_slam' to 'configs/native.yaml'
- Added/moved 'fast_pose_sample_time' in 'gldemo/plugin.cpp'
- Added back '_m_first_time' to 'gldemo.plugin.cpp'
- Added in-line construction for PimObject in
'gtsam_integrator/plugin.cpp'
- Added comment/todo for each integrator's 'last_imu_offset'
- Removed '_m_imu_integrator' from 'offline_imu_cam/plugin.cpp'
- Fixed '_m_ground_truth_offset' usage in 'pose_lookup/plugin.cpp'
- Removed member names from allocations in 'pose_prediction/plugin.cpp'
- Removed '_m_imu_integrator' from 'realsense/plugin.cpp'
- Updated calls to 'put' in 'realsense/plugin.cpp'
- Added pre-sleep usage comment in 'runtime/main.cpp'
- Added back '_p_should_skip' in 'timewarp_gl/plugin.cpp'
- Added missing 'kimera_path' to 'configs/headless.yaml'
- Updated versions/integration for 'configs/monada.yaml'
- Updated 'time_type' usage in 'gldemo/plugin.cpp'
- Fixed calls to 'abort' in 'pose_lookup'
- Removed named arguments in 'pose_lookup/plugin.cpp'
- Removed unused 'frame' in 'timewarp_gl/plugin.cpp'
Copy link
Member

@mhuzai mhuzai left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Almost there.

rk4_integrator/plugin.cpp Outdated Show resolved Hide resolved
timewarp_gl/plugin.cpp Outdated Show resolved Hide resolved
@@ -17,7 +19,7 @@ SET(EXECUTABLE_OUTPUT_PATH ".")

find_package(ZED 3 REQUIRED)
find_package(OpenCV REQUIRED)
find_package(CUDA ${ZED_CUDA_VERSION} EXACT REQUIRED)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We're in a gray area here. I'm willing to let this be if CUDA 11.2 has been properly tested with ZED 3.4.2 (which complains during install if you don't have the exact version).

common/switchboard.hpp Outdated Show resolved Hide resolved
* Thread-safe
* - Caveat:
*
* This (circular) queue based solution may race if >= N writers attempt
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Doesn't have to be N writers. Just N writes.

- Typo: 'destructible' in 'common/switchboard.hpp'
- Updated description of 'event' in 'common/switchboard.hpp'
- Removed unused '_m_imu_cam' in 'rk4_integrator/plugin.cpp'
- Removed unneeded assertion in 'timewarp_gl/plugin.cpp'
@e3m3 e3m3 merged commit 5277855 into master Apr 20, 2021
ILLIXR improvements automation moved this from PRs in Review to Done Apr 20, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working WIP Work in progress
Projects
Development

Successfully merging this pull request may close these issues.

None yet

3 participants