-
Notifications
You must be signed in to change notification settings - Fork 75
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
missing snappy support in linux wheels #68
Comments
hmmm. weird. plyvel has almost identical code between 0.9 and 1.x except for a) one extra option, for which the leveldb minimum version was bumped, and b) added byte-like input flexibility. see https://plyvel.readthedocs.io/en/latest/news.html#plyvel-1-0-0 diff is at (most of it are docs) |
see also #8 btw (rather old) |
Narrowing it down, I'm having a database that was created with plyvel 0.9, and getting an error when trying to read from it with plyvel 1.0.3:
We do store some bytes and binary content. |
Found the issue -- databases created on OSX cannot be opened in Linux with plyvel > 0.9. Steps to reproduce:
|
any chance you can share a minimal tarfile with a fresh db created on osx with one db.put(foo, bar) key pair in it? i don't have a functional osx dev machine currently. |
Interesting. If I prepare a very simple db like this it works on Linux too. Here is a more complex one that breaks plyvel >0.9 on Linux: leveldb-broken-in-linux.tar.gz (1.5mb, taken after syncing a few hundred NEO blocks). We are using it with neo-python. A really large database we're using for running the tests and which can't be loaded with plyvel 1.0.3 can be found here: https://s3.us-east-2.amazonaws.com/cityofzion/fixtures/fixtures_v4.tar.gz (1 GB) |
hmmm. this might have to do with apparently missing snappy support in the linux wheels. without local compilation (wheels):
with local compilation (no wheels, leveldb and snappy dev headers installed):
|
i just pushed 6fc14cb to enable building docker wheels with snappy support:
contrary to the 1.0.3 wheel which was not linked against snappy,
i have just pushed 1.0.4 to pypi: https://plyvel.readthedocs.io/en/latest/news.html#plyvel-1-0-4 please give it a try! |
Awesome, it works! |
the reason it may have worked for a very simple db with only one key/value pair is that leveldb has not yet compacted it into a not entirely sure, i am a bit rusty on the details since i recently dusted off this project after 3 years. ;) |
You are a great maintainer <3 |
💥 thanks for confirming! 💟 |
We're using plyvel v0.9 for neo-python and run the tests with downloaded fixtures (leveldb snapshots). Now I tried upgrading to plyvel v1.0.2, and there are now some errors that say
plyvel._plyvel.CorruptionError: b'Corruption: corrupted compressed block contents'
: https://travis-ci.org/CityOfZion/neo-python/builds/329522567The text was updated successfully, but these errors were encountered: