Improved dataset loading; fixes, safeties, diagnostics, and better feedback#613
Improved dataset loading; fixes, safeties, diagnostics, and better feedback#613
Conversation
|
Before you submit for review:
If you did not complete any of these, then please explain below. |
tlwillke
left a comment
There was a problem hiding this comment.
The fix works. The dataset loader is repeating the load 3 times, twice locally cached. This was not introduced by your PR, but can you please attempt a fix while you're at it?
Yes, I'll take a look. I think this is due to no clear distinction between acquiring (a handle to) and accessing (the data therein). I'll see if there is a way to improve it which isn't onerous. |
|
The fix works. Please disregard. It's actually the 3 required files per dataset. No fix required. |
…edback (#613) * enable better diagnostics for dataset loading * fix off-by-one error in extension truncation * enable java-native invocation of bench for debugging * qualify exec config to unconfuse IDE * avoid possible NPE from maven null argv invocation * make hdf5 loader more robust * add example config for glove-25-angular.yml for testing
This fixes a regression in hdf5 data loading, but it also adds minor improvements for testing diagnostics.
Each functional change is shown on a themed commit, for easier review.
Changes: