Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor ListenerParser to be a plain old Python class #1490

Merged
merged 3 commits into from
Aug 9, 2021

Conversation

mdickinson
Copy link
Member

@mdickinson mdickinson commented Aug 6, 2021

This PR refactors the ListenerParser to turn it into a plain old Python class, rather than a HasTraits class.

This was mostly motivated by performance; this is only a small part of the performance story, but it is a part, and it's easy to fix. The overall goal is to speed up the creation of Traits listeners, since that was proving a bottleneck in some large Traits-using applications. This PR speeds up parsing of listener text strings like "foo.bar[]".

The behaviour of the old class also didn't make a lot of sense: in the __init__ method, there was a line self.text = text, prior to calling the superclass __init__. That self.text = text assignment triggered the _text_changed method, which then did all the parsing. That meant that all of the parsing had already taken place, and the listener trait set, before the super().__init__(**traits) call took place, making that superclass call entirely redundant. The trait-change dispatch to the _text_changed method was showing up as significant (not huge, but significant) in profiling.

This was motivated by #1489: we still have a lot of old Delegate-based code around (particularly in TraitsUI), and so speeding up that code should provide a quick win for downstream code.

@mdickinson
Copy link
Member Author

mdickinson commented Aug 6, 2021

Here are some informal timings on my machine, showing an approximate 80% speedup on a micro-benchmark.

First, on this branch (leaving out the steps that failed because I mistyped something):

Python 3.9.6 (default, Jul  3 2021, 08:35:53) 
Type 'copyright', 'credits' or 'license' for more information
IPython 7.26.0 -- An enhanced Interactive Python. Type '?' for help.

[ --- 8< ---- Mark's errors snipped ---- 8< ----]

In [3]: from traits.traits_listener import ListenerParser

In [4]: %timeit ListenerParser("shadow:handler").listener
5.31 µs ± 190 ns per loop (mean ± std. dev. of 7 runs, 100000 loops each)

And the same timing on the main branch:

(traits) mdickinson@mirzakhani traits % ipython
Python 3.9.6 (default, Jul  3 2021, 08:35:53) 
Type 'copyright', 'credits' or 'license' for more information
IPython 7.26.0 -- An enhanced Interactive Python. Type '?' for help.

In [1]: from traits.traits_listener import ListenerParser

In [2]: %timeit ListenerParser("shadow:handler").listener
9.82 µs ± 59.8 ns per loop (mean ± std. dev. of 7 runs, 100000 loops each)

@mdickinson
Copy link
Member Author

I realised I was using the wrong IPython for one of those timings (the global IPython instead of the one in the venv). Here are new timings, still showing more or less (actually, more rather than less) the same improvement:

(traits) mdickinson@mirzakhani traits % git checkout refactor/listener-parser-look-ma-no-traits
Switched to branch 'refactor/listener-parser-look-ma-no-traits'
Your branch is up to date with 'origin/refactor/listener-parser-look-ma-no-traits'.
(traits) mdickinson@mirzakhani traits % ipython
Python 3.9.6 (default, Jul  3 2021, 08:35:53) 
Type 'copyright', 'credits' or 'license' for more information
IPython 7.26.0 -- An enhanced Interactive Python. Type '?' for help.

In [1]: from traits.traits_listener import ListenerParser

In [2]: %timeit ListenerParser("shadow:handler").listener
4.94 µs ± 109 ns per loop (mean ± std. dev. of 7 runs, 100000 loops each)

In [3]: exit
(traits) mdickinson@mirzakhani traits % git checkout main
Switched to branch 'main'
Your branch is up to date with 'origin/main'.
(traits) mdickinson@mirzakhani traits % ipython          
Python 3.9.6 (default, Jul  3 2021, 08:35:53) 
Type 'copyright', 'credits' or 'license' for more information
IPython 7.26.0 -- An enhanced Interactive Python. Type '?' for help.

In [1]: from traits.traits_listener import ListenerParser

In [2]: %timeit ListenerParser("shadow:handler").listener
10.3 µs ± 199 ns per loop (mean ± std. dev. of 7 runs, 100000 loops each)

Copy link
Contributor

@rahulporuri rahulporuri left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mostly LGTM with one small question.

Also, it'd be useful to mention what performance problem this PR is addressing. I think it's the performance of text parsing in on_trait_change handlers but it'd be nice to document it in the PR. Also, I think the profiling work was driven by #1489 , right?

@@ -1009,9 +993,18 @@ def _get_name(self):

# -- object Method Overrides ----------------------------------------------

def __init__(self, text="", **traits):
def __init__(self, text):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

im not a 100% sure why we're changing the text default here

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mostly just for safety.

Previously, the text in the ListenerParser could be updated after initialization, triggering the _text_changed listener. With the refactored code, the only way the ListenerParser can be used correctly is to provide the text directly at initialization time. I made it non-optional so that we catch any code that tries to create a ListenerParser without providing the text to parse.

@mdickinson
Copy link
Member Author

it'd be useful to mention what performance problem this PR is addressing

I added a note to the description.

@mdickinson mdickinson merged commit 542e971 into main Aug 9, 2021
@mdickinson mdickinson deleted the refactor/listener-parser-look-ma-no-traits branch August 9, 2021 07:36
@mdickinson mdickinson added topic: on_trait_change performance Effort to speed up the on_trait_change machinery (without changing behaviour) component: core Issues related to the core library type: performance Issues related to speed or memory usage labels Aug 9, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
component: core Issues related to the core library topic: on_trait_change performance Effort to speed up the on_trait_change machinery (without changing behaviour) type: performance Issues related to speed or memory usage
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants