Filter Bags#37
Merged
Merged
Conversation
Contributor
There was a problem hiding this comment.
Pull Request Overview
Adds a destructive CLI command to strip non-IMU/LiDAR data from ROS1 and ROS2 bags, reducing storage size by ~50%.
- Implements
filter_ros1andfilter_ros2helper functions to rewrite bags with only the specified topics. - Adds a
filtercommand indataset_manager.py, wiring up topic-based filtering for downloaded datasets. - Bumps
rosbagsdependency to>=0.10.10inpyproject.toml.
Reviewed Changes
Copilot reviewed 2 out of 3 changed files in this pull request and generated 3 comments.
| File | Description |
|---|---|
| python/evalio/cli/dataset_manager.py | New filter command, plus filter_ros1 and filter_ros2 functions for topic-based bag rewriting. |
| pyproject.toml | Updated rosbags dependency to >=0.10.10 for required bugfixes. |
Comments suppressed due to low confidence (2)
python/evalio/cli/dataset_manager.py:191
- [nitpick] Naming this function
filtershadows the built-infilter; consider renaming to avoid confusion (e.g.,filter_bags).
def filter(
python/evalio/cli/dataset_manager.py:190
- No tests have been added for the new
filtercommand; consider adding unit or integration tests to verify destructive behavior and error paths.
@app.command(no_args_is_help=True)
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR adds a (destructive) command to remove all data from downloaded rosbags that aren't the required IMU or LiDAR data. This greatly reduces storage size (by 50% for newer college)!
I'm on the fence about the command name. I considered
filter,clean, orprune, but felt that filter best represents what is happening here.