New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Micro optimizations 2 #1561

Merged
merged 11 commits into from Oct 28, 2016

Conversation

Projects
None yet
3 participants
@ssanderson
Member

ssanderson commented Oct 27, 2016

No description provided.

ssanderson added some commits Oct 26, 2016

PERF: Try cache on scalar asset lookups.
This provides a 15% speedup for an algo that calls `data.current` with
1000 every minute.
PERF: Refactor AdjustedArrayWindow.
Make `__next__` and `seek` share code instead of seek() calling
`__next__`.  This avoids having to make a large number of integer
comparisons and `asanyarray` calls when seeking more than one tick
forward.
PERF: Use vectorized assignment into dataframe.
This is a dramatic speedup (~25% in local benchmarks) for history calls
with a large number of assets and a short window length.

@ssanderson ssanderson force-pushed the micro-optimizations-2 branch from b4e8a0f to 9d10e28 Oct 27, 2016

@coveralls

This comment has been minimized.

coveralls commented Oct 27, 2016

Coverage Status

Coverage increased (+0.0009%) to 86.979% when pulling 9d10e28 on micro-optimizations-2 into e7ca080 on master.

@ssanderson ssanderson referenced this pull request Oct 27, 2016

Closed

Micro optimizations #1559

@coveralls

This comment has been minimized.

coveralls commented Oct 27, 2016

Coverage Status

Coverage increased (+0.0009%) to 86.979% when pulling 9d10e28 on micro-optimizations-2 into e7ca080 on master.

ssanderson added some commits Oct 27, 2016

PERF: Pull out loop-invariant code.
This shaves off 20 out of 160 seconds for an algorithm that makes a
large number of large universe, short window_length `history()` calls.
MAINT/PERF: Remove redundant method call.
`_get_minute_window_data` was just forwarding its input to a method with
the same signature.
PERF: Call concatenate directly instead of hstack.
Avoids a couple function calls in a hot path.
@coveralls

This comment has been minimized.

coveralls commented Oct 27, 2016

Coverage Status

Coverage remained the same at 86.978% when pulling 70cc602 on micro-optimizations-2 into e7ca080 on master.

if target_anchor == anchor:
return self.output
self._tick_forward(target_anchor - anchor)

This comment has been minimized.

@ehebert

ehebert Oct 27, 2016

Member

Should Exhausted be handled here? Or is the intent to have that raise for the consumer of seek to handle?

This comment has been minimized.

@ssanderson

ssanderson Oct 27, 2016

Member

The if target_anchor < anchor: check should preclude Exhausted ever being raised here.

This comment has been minimized.

@ssanderson

ssanderson Oct 27, 2016

Member

nvm, I misread the code. The check above is for making sure we don't seek backwards.

It's an error for the caller to try to seek past the end of the array. In the old code, that would fail with a StopIteration, which is actually probably bad because that can be caught and misinterpreted if seek is called inside a generator. I don't have a strong opinion on whether it's worthwhile to try to catch and re-raise Exhausted here with a more descriptive error. Given that this is in performance-critical code, I'm inclined to say that this is fine as-is.

@ssanderson ssanderson merged commit 285b5f7 into master Oct 28, 2016

2 checks passed

continuous-integration/appveyor/pr AppVeyor build succeeded
Details
continuous-integration/travis-ci/pr The Travis CI build passed
Details

@ssanderson ssanderson deleted the micro-optimizations-2 branch Oct 28, 2016

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment