-
Notifications
You must be signed in to change notification settings - Fork 69
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
cache lookup of importlib metadata in Node action #365
cache lookup of importlib metadata in Node action #365
Conversation
Signed-off-by: William Woodall <william@osrfoundation.org>
Signed-off-by: William Woodall <william@osrfoundation.org>
IIUC, this means that if one installs packages (that add extensions) while launch is still spinning up nodes, the nodes that start after the installation is done won't reflect the changes, right ? If yes, that seems like a weird use case to me, as when the installation completes is not predictable, and nor does launch guarantee the order in which nodes start. So yes a +1 to this change, looks good ! Would this cache also affect nested actions or launch files ? i.e. if a launch file includes another launch file will this cache be shared ? |
Right, the first time a node asks for the list of extensions it is cached, so if you install something new in parallel, it will be a race to see if it is included or not and with which nodes. I can't imagine a use case for that anyway, but I wanted to point it out. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks good with green CI !
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
it will prevent extensions from changing after launch has started
i believe this is not the user expectation or assumption. lgtm.
* cache lookup of importlib metadata in Node action Signed-off-by: William Woodall <william@osrfoundation.org> * style fixup Signed-off-by: William Woodall <william@osrfoundation.org> --------- Signed-off-by: William Woodall <william@osrfoundation.org>
The short version of this is that we use
importlib_metadata
to load extensions to theNode
action via Python'sentry_point
pattern. This lets new packages extendNode
without changinglaunch_ros
itself. The problem is that we load these extensions every time a node needs them and that can take as long as 1 second on some machines. If you have a launch file with hundreds of nodes, like I did, then you get pauses of 30 seconds to a minute before the launch file starts launching things.I used the profiler to find out that this
launch_ros.actions.Node.get_extensions()
was being called some 296 times in my launch file and each time it was taking ~0.4 seconds, leading to a delay of almost 30 seconds:After the change in this pr, the profiler shows what I experience on the command-line, which is much better behavior, taking almost no time to start launching things, since we only look this up once:
Some potential issues with this pr:
For the first item, I don't think the rest of the code is written to assume this is possible in the first place, and I'm not sure what the use case for this would look like. I suppose you could have launch running, and while running install new python packages that add extensions, then cause launch to start a new node after this, and in that case it would not see new extensions. However, I don't know when you'd do that in practice. And I think this change benefits the existing users at the cost of a potential use case that I do not believe folks are using today, but I'm open to hear otherwise.
For the second, I don't think we necessarily need to cache more, based on what I see in the second profiler.