New Backend, HDF5_01 - Optimizations for large fixed size arrayset schemas.

[rlizzo]

Motivation and Context

Why is this change required? What problem does it solve?:

Significant performance improvements for larger fixed size arrayset data.

Inline documentation explains rationale.

If it fixes an open issue, please link to the issue here:

Description

Describe your changes in detail:

• new HDF5_01 backend (to complement HDF5_00).
• introducing methods to more intelligently select backends for an arrayset schema based on it's access conventions and setup.
• updated tests
• updated docs

Types of changes

What types of changes does your code introduce? Put an x in all the boxes that apply:

• Documentation update
• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to change)

Is this PR ready for review, or a work in progress?

• Ready for review
• Work in progress

How Has This Been Tested?

Put an x in the boxes that apply:

• Current tests cover modifications made
• New tests have been added to the test suite
• Modifications were made to existing tests to support these changes
• Tests may be needed, but they are not included when the PR was proposed
• I don't know. Help!

Checklist:

• My code follows the code style of this project.
• My change requires a change to the documentation.
• I have updated the documentation accordingly.
• I have read the CONTRIBUTING document.
• I have signed (or will sign when prompted) the tensorwork CLA.
• I have added tests to cover my changes.
• All new and existing tests passed.

[rlizzo]

Hey @hhsecond this is a first draft PR here.. It's ready for a high level review, but will definitely need some work before thinking about merging it.

[codecov]

Codecov Report

Merging #160 into master will decrease coverage by 0.17%.
The diff coverage is 92.64%.

@@            Coverage Diff             @@
##           master     #160      +/-   ##
==========================================
- Coverage   95.31%   95.14%   -0.17%     
==========================================
  Files          64       65       +1     
  Lines       11548    11821     +273     
  Branches      977     1023      +46     
==========================================
+ Hits        11006    11246     +240     
- Misses        361      384      +23     
- Partials      181      191      +10

Impacted Files	Coverage Δ
tests/test_diff_staged_summary.py	100% <ø> (ø)	⬆️
src/hangar/remote/client.py	80.29% <ø> (-0.23%)	⬇️
tests/test_checkout.py	99.8% <100%> (ø)	⬆️
tests/test_arrayset_backends.py	100% <100%> (ø)	⬆️
tests/test_cli.py	99.57% <100%> (ø)	⬆️
src/hangar/arrayset.py	95.1% <100%> (ø)	⬆️
src/hangar/backends/hdf5_00.py	92.99% <100%> (+0.03%)	⬆️
tests/test_arrayset.py	100% <100%> (ø)	⬆️
tests/property_based/test_pbt_arrayset.py	100% <100%> (ø)	⬆️
tests/property_based/test_pbt_metadata.py	100% <100%> (ø)	⬆️
... and 14 more

[rlizzo]

@elistevens this will interest you. Once this is merged (hopefully in the next few days) I'll be pushing v0.4.0b1

[lgtm-com]

This pull request fixes 16 alerts when merging 9167eee into d105e56 - view on LGTM.com

fixed alerts:

• 8 for Unused local variable
• 5 for Unused import
• 1 for Unmatchable dollar in regular expression
• 1 for Module-level cyclic import
• 1 for Module is imported with 'import' and 'import from'