-
Notifications
You must be signed in to change notification settings - Fork 252
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Adds play until functionality -- follow up to #962 #1005
Adds play until functionality -- follow up to #962 #1005
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@agalbachicar How customer will know what number for timestamp to use for command line playback_until
option?
If it's going to be expressed in seconds and floating point representation.
Sorry I haven't seen unit tests yet.
I concur with you. That's not the best way to determine it because of floating point to integer conversions, but I think it is better than receiving a timestamp in nanoseconds. I am open to suggestions if you have any. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I've left a couple questions that I was hoping you'd be able to clarify...
if (play_options_.playback_duration >= rclcpp::Duration(0, 0) || | ||
play_options_.playback_until >= rcutils_time_point_value_t{0}) | ||
{ | ||
play_until_time = std::max( |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If someone set both playback_duration
and playback_until
, you could run into a scenario that the starting_time
+ playback_duration
could be a later timestamp than the playback_until
value - which means, according to the std::max
here, that the playback_until
value would be effectively ignored in that instance, but NOT ignored in other instances.
Is this desired behavior? If so, we may want to document that behavior in the --help
from the CLI.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Before answering, I would like to double check what an instance means for you. Can you elaborate a bit more on that? I will regardless expand the help documentation to make sure that everyone understands if we end up having the playback_until
flag as well.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
An instance is a different usage of the program. So, one instance would be where the program is run with playback_duration
being greater than playback_until
, a different instance would be where playback_until
is greater than playback_duration
.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is this desired behavior?
Yes, this is the desired behavior.
An instance is a different usage of the program. So, one instance would be where the program is run with playback_duration being greater than playback_until, a different instance would be where playback_until is greater than playback_duration.
I don't think it should be a problem from the player point of view (aside from the duplicated topics). Even if both requests land to the same instance, the first one should prevail over the second one.
6d55bc8 adds the docstring. Let me know what you think.
I wouldn't expose this API to customer via CLI at all. |
Thanks @jhdcs , I replied to your comments, PTAL. @MichaelOrlov I haven't ignored yours, I am gathering data from the person that is actually handling all the set of features we are trying to add to rosbag but is currently using an adapted fork. That should help to the discussion more than my own experience. |
Just to clarify. For Although for My takeaway is that if there are no easy understandable way to get value for the command line option we shouldn't expose it. cc: @emersonknapp |
@MichaelOrlov and @agalbachicar I want to provide some use context for this API. We had requested this functionality as part of an experiment pipeline we run at TRI. We have a large multi-machine system that has baggers running on more than one host. The baggers also do not start at the same time because of distributed system launching. When analyzing the data, we either have to run a bag_merger process (which is fine for small bags but can be frustrating or impossible for large bags) or we need to stream channels from bags we care about up to a point into a new bag (or a node that's doing analysis). Playing until a specific timestamp avoids the need to know when the individual bag started and computing duration for each bag, so we can just run a bag player for each either from the host machines or after pulling the bags into a common location. We can't reliably say that something interesting happened after 32 seconds, because the camera data is in a different bag than telemetry. We can however analyze the camera data and find something interesting, then request the telemetry back to play back that data by timestamp. The expectation is that we are getting the initial timestamps from looking through one of our ros bags from the experiments |
I understand your use case. And it mostly dictated by the sub-optimal system design and lack of the I am about to design and implement 'Start/Stop' API for rosbag2 in nearest future. |
Interesting. I am exciting about remote start/stop as a capability as its another gap we want to fill. However, I am not sure it totally replaces the use of play until. Play until remains more flexible if we add or remove data collection devices for testing purposes without extensive integration into the main pipeline start / stop. I think both features are still useful. My specific case for that is when we may or may not want to log data from a source not part of the standard system bringup. We have a set of processes that run across multiple machines, some of which is the bagging. If we want to test out a new device without modifying the distributed configuration, we may want the ability to start and stop one-off recorders at will. We imagine a use case where the same data collection may be used by multiple sub-teams who only want specific data from certain sub-systems. I still want the ability to let them run their own data collector during an experiment and provide them with the overall system bag for them to dice as they please. |
@andrewbest-tri For the cases when need to extract some certain subset of topics from bag in another bag you can use I think we can add play until command line interface for a while. |
Would it work to have two arguments (seconds and nanoseconds) as two integer input args?
We would compose them and create a timestamp. The logic would be that we would only consider the postive parts, and set the flags to be by default negative. That way, you can opt to pass only the seconds, only the nanoseconds, or both. Thoughts? |
See e5d83fe . It implements my proposal in #1005 (comment) . I can either revert it or amend it in future commits if it is not satisfactory. I'm looking forward to hearing your thoughts. |
@agalbachicar |
I don't have a strong preference and implemented the mutually exclusive parameters as you suggested. Here is what a user will see in the future when passing them both: $ ros2 bag play --playback-until-sec 123.456 --playback-until-nsec 123456000000 a_bag_file
usage: ros2 bag play [-h] [-s {my_test_plugin,sqlite3,my_read_only_test_plugin}] [--read-ahead-queue-size READ_AHEAD_QUEUE_SIZE] [-r RATE] [--topics TOPICS [TOPICS ...]]
[--qos-profile-overrides-path QOS_PROFILE_OVERRIDES_PATH] [-l] [--remap REMAP [REMAP ...]] [--storage-config-file STORAGE_CONFIG_FILE] [--clock [CLOCK]] [-d DELAY]
[--playback-duration PLAYBACK_DURATION] [--playback-until-sec PLAYBACK_UNTIL_SEC | --playback-until-nsec PLAYBACK_UNTIL_NSEC] [--disable-keyboard-controls] [-p]
[--start-offset START_OFFSET] [--wait-for-all-acked TIMEOUT] [--disable-loan-message]
bag_file
ros2 bag play: error: argument --playback-until-nsec: not allowed with argument --playback-until-sec And when asking for help: $ ros2 bag play -h
usage: ros2 bag play [-h] [-s {my_test_plugin,my_read_only_test_plugin,sqlite3}] [--read-ahead-queue-size READ_AHEAD_QUEUE_SIZE] [-r RATE] [--topics TOPICS [TOPICS ...]]
[--qos-profile-overrides-path QOS_PROFILE_OVERRIDES_PATH] [-l] [--remap REMAP [REMAP ...]] [--storage-config-file STORAGE_CONFIG_FILE] [--clock [CLOCK]] [-d DELAY]
[--playback-duration PLAYBACK_DURATION] [--playback-until-sec PLAYBACK_UNTIL_SEC | --playback-until-nsec PLAYBACK_UNTIL_NSEC] [--disable-keyboard-controls] [-p]
[--start-offset START_OFFSET] [--wait-for-all-acked TIMEOUT] [--disable-loan-message]
bag_file
Play back ROS data from a bag
positional arguments:
bag_file bag file to replay
optional arguments:
-h, --help show this help message and exit
-s {my_test_plugin,my_read_only_test_plugin,sqlite3}, --storage {my_test_plugin,my_read_only_test_plugin,sqlite3}
Storage implementation of bag. By default tries to determine from metadata.
--read-ahead-queue-size READ_AHEAD_QUEUE_SIZE
size of message queue rosbag tries to hold in memory to help deterministic playback. Larger size will result in larger memory needs but might prevent delay of message playback.
-r RATE, --rate RATE rate at which to play back messages. Valid range > 0.0.
--topics TOPICS [TOPICS ...]
topics to replay, separated by space. If none specified, all topics will be replayed.
--qos-profile-overrides-path QOS_PROFILE_OVERRIDES_PATH
Path to a yaml file defining overrides of the QoS profile for specific topics.
-l, --loop enables loop playback when playing a bagfile: it starts back at the beginning on reaching the end and plays indefinitely.
--remap REMAP [REMAP ...], -m REMAP [REMAP ...]
list of topics to be remapped: in the form "old_topic1:=new_topic1 old_topic2:=new_topic2 etc."
--storage-config-file STORAGE_CONFIG_FILE
Path to a yaml file defining storage specific configurations. For the default storage plugin settings are specified through syntax:read: pragmas: ["<setting_name>" =
<setting_value>]Note that applicable settings are limited to read-only for ros2 bag play.For a list of sqlite3 settings, refer to sqlite3 documentation
--clock [CLOCK] Publish to /clock at a specific frequency in Hz, to act as a ROS Time Source. Value must be positive. Defaults to not publishing.
-d DELAY, --delay DELAY
Sleep duration before play (each loop), in seconds. Negative durations invalid.
--playback-duration PLAYBACK_DURATION
Playback duration, in seconds. Negative durations mark an infinite playback. Default is -1.0. When positive, the maximum of `playback-until` and the one that this attribute
yields will be used to determine which one stops playback execution.
--playback-until-sec PLAYBACK_UNTIL_SEC
Playback until timestamp, expressed in seconds since epoch. Mutually exclusive argument with '--playback-until-nsec'. Use this argument when floating point to integer conversion
error is not a problem for your application. Negative stamps disable this feature. Default is -1.0. When positive, the maximum of the effective time that `--playback-duration`
yields and this attribute will be used to determine which one stops playback execution.
--playback-until-nsec PLAYBACK_UNTIL_NSEC
Playback until timestamp, expressed in nanoseconds since epoch. Mutually exclusive argument with '--playback-until-sec'. Use this argument when floating point to integer
conversion error matters for your application. Negative stamps disable this feature. Default is -1. When positive, the maximum of the effective time that `--playback-duration`
yields and this attribute will be used to determine which one stops playback execution.
--disable-keyboard-controls
disables keyboard controls for playback
-p, --start-paused Start the playback player in a paused state.
--start-offset START_OFFSET
Start the playback player this many seconds into the bag file.
--wait-for-all-acked TIMEOUT
Wait until all published messages are acknowledged by all subscribers or until the timeout elapses in millisecond before play is terminated. Especially for the case of sending
message with big size in a short time. Negative timeout is invalid. 0 means wait forever until all published messages are acknowledged by all subscribers. Note that this option
is valid only if the publisher's QOS profile is RELIABLE.
--disable-loan-message
Disable to publish as loaned message. By default, if loaned message can be used, messages are published as loaned message. It can help to reduce the number of data copies, so
there is a greater benefit for sending big data. |
@MichaelOrlov thanks for your input so far. I think we are close to conclude how the CLI should work. Also, I hope that unit tests are aligned with your expectations. If there is anything else that I should change about them let me know. |
@agalbachicar Sorry for the delay on my side. |
@agalbachicar Could you please clean up your branch? |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@agalbachicar Overall looks good. I requested mostly minor changes or some nitpicks or improvements.
As regards to the tests. It would be nice to move common code as RosBag2PlayUntilTestFixture
and RosBag2PlayForTestFixture
in some common place and combined them to avoid code duplication. For instance it could be a common function
void InitPlayerWithPlaybackUntilAndDurationAndPlay(
int64_t playback_until_millis,
int64_t playback_duration_millis,
size_t expected_number_of_messages_on_topic1 = 3,
size_t expected_number_of_messages_on_topic2 = 3)
and specialized function
void InitPlayerWithPlaybackDurationAndPlay(
size_t expected_number_of_messages_on_topic1 = 3,
size_t expected_number_of_messages_on_topic2 = 3,
int64_t playback_duration_millis) {
return InitPlayerWithPlaybackUntilAndDurationAndPlay(
expected_number_of_messages_on_topic1,
expected_number_of_messages_on_topic2,
-1,
playback_duration_millis);
}
Also I would appreciate if you will address residual renaming of the play_for
to the playback_duration
.
Looking forward to see rebased version when #960 will be merged on master.
Signed-off-by: Agustin Alba Chicar <ag.albachicar@ekumenlabs.com>
Signed-off-by: Agustin Alba Chicar <ag.albachicar@ekumenlabs.com>
Signed-off-by: Agustin Alba Chicar <ag.albachicar@ekumenlabs.com>
Signed-off-by: Agustin Alba Chicar <ag.albachicar@ekumenlabs.com>
… in PlayOptions to clarify usage. Signed-off-by: Agustin Alba Chicar <ag.albachicar@ekumenlabs.com>
…ayback_until stamp Signed-off-by: Agustin Alba Chicar <ag.albachicar@ekumenlabs.com>
- Unifies the use of '-' instead of '_' to split words in arguments. - Uses mutually exclusive arguments: --playback-until-sec and --playback-until-nsec to pass the timestamp. Signed-off-by: Agustin Alba Chicar <ag.albachicar@ekumenlabs.com>
190c290
to
2ef415e
Compare
- Adds test cases to the player. - Renames play_until and similar occurrences to contain timestamp. - Renames play_for and similar occurrences to not contain for and include duration. Signed-off-by: Agustin Alba Chicar <ag.albachicar@ekumenlabs.com>
Would you be OK with doing this in a follow up pull request? |
2ef415e
to
c48d211
Compare
@MichaelOrlov let me know if you prefer to merge #1024 into this branch or you prefer to keep them in different PRs. |
Let's do it in a separate PR. I am afraid that it could last longer for code review and iterations. |
Signed-off-by: Agustin Alba Chicar <ag.albachicar@ekumenlabs.com>
I found that |
BTW, @MichaelOrlov all comments were addressed. PTAL when you have a moment. |
@agalbachicar Overall looks good. However I found one not addressed request. Would you address my comment in regards to the As regards to that |
The test body now evaluates a bag whose duration is exactly the same as given timestamp in PlayOptions. Signed-off-by: Agustin Alba Chicar <ag.albachicar@ekumenlabs.com>
Sorry, I missed it. It should be done in fa3654f @MichaelOrlov |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@agalbachicar Approving!
Thanks for you hard work.
@jhdcs anything to add? BTW, I don't see the merge button so I guess you have merge powers @MichaelOrlov and @jhdcs |
I will run CI today and if everything is good will merge it. |
Gist: https://gist.githubusercontent.com/MichaelOrlov/a4d1767e3221ad9709f5bfe56bc172e5/raw/6c1a4538436c74628c38b102e682e71193e17ee9/ros2.repos |
This pull request has been mentioned on ROS Discourse. There might be relevant details there: https://discourse.ros.org/t/ros-2-tsc-meeting-minutes-8-18-2022/27050/1 |
This PR offers the same functionality than #962 but is done directly on top of the latest version of #960 ( commit 59c52ef ). It has been done this way because to re-work the PR so it follows the style agreed on #960 a revert was required of the entire patch.
This PR requires #960 to be merged first (at which point it will probably need to be rebased).
This PR was co-authored with @gbiggs
Fore reviewers, you should start from df793c7 onwards.