Change PdsFile to use shelve files for everything

[rfrenchseti]

Shelf files were a later addition to the PdsFile class. Re-factoring the code to use them for this purpose would not be a bad idea. Somebody would have to implement a glob.glob replacement that uses shelf files. Feel free!

Note that, if the match pattern crosses volume boundaries, it could actually be slower than a direct call to glob.glob. That's because each volume has its own shelf files.

On Dec 4, 2018, at 1:39 PM, Rob French rfrench@seti.org wrote:

It seems like PdsFile maintains these enormous "shelve" files (or pickle
files) that contain all sorts of useful information about every file in
pdsdata. But then when you call opus_products(), it uses glob() on the
real file system instead of looking into these files. Likewise when you
create a ViewSet for preview images, it has "must_exist=True", which
checks the actual filesystem. Same for the "exists" property.

Shouldn't we be using the shelve files for all of these cases?

.

[rfrenchseti]

Moved to pds-webtools