ENH: HDF5-based FX Reader/Writer. #2597
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
added 4 commits
December 17, 2019 14:34
- Add test coverage for non-scalar lookups. - Performance optimizations for InMemoryFXRateReader and HDF5FXRateReader. We no longer construct a DataFrame on each call to get_rates, since doing so is surprisingly expensive and caused test_lookup_scalar to take more time than was reasonable.
Explicitly require that pipeline-compatible readers support 'default' as a key. I'm currently just using the string 'default' for this. We should probably move that value to a shared location, but it's not clear to me that value should live since it's defining an interface boundary between pipeline and fx data.
- Rather than using numpy's S3 and U3 types, which behave differently in py2 and py3, just expect object arrays of `str` everywhere. This makes it slightly more expensive for us to read the currency index from the file in py3, but that array is on the order of a hundred or so elements total, so that's not a major concern, and it simplifies the handling elsewhere.
quantophred
reviewed
Dec 31, 2019
added 3 commits
January 2, 2020 10:49
This makes it easier to distinguish different usages of 'default', and makes it easier in the future to change how we handle defaults.
Merged
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
HDF5-based implementation of
FXRateReader.HDF5FXRateReaderandHDF5FXRateWriterfor reading/writing fx rates stored in hdf5.zipline/data/fx.pyinto a directory. We now havezipline.data.fxas a module, with child modules implementing readers, pluszipline.data.fx.basedefining the interface they all implement.FXRateReader.InMemoryFXRateReader(previously only indirectly tested via the pipeline API tests) and the newHDF5FXRateReader.