You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Idea: SQLite3 database or fast key-value database like berkeleydb named .guano.py.cache with index of (filename, filesize, timestamp, hash).
Hash should be a fast non-cryptographic function like crc32, md5, sha1, xxHash.
If we determine that the file hasn't changed, load metadata from cache.
Would this be significantly faster given that we'd need to do full file reads to compute hash? Is (filename, filesize, timestamp) sufficient without a hash?
The text was updated successfully, but these errors were encountered:
Idea: SQLite3 database or fast key-value database like berkeleydb named
.guano.py.cache
with index of(filename, filesize, timestamp, hash)
.Hash should be a fast non-cryptographic function like crc32, md5, sha1, xxHash.
If we determine that the file hasn't changed, load metadata from cache.
Would this be significantly faster given that we'd need to do full file reads to compute hash? Is
(filename, filesize, timestamp)
sufficient without a hash?The text was updated successfully, but these errors were encountered: