Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

nd-db index transferability #13

Open
2 tasks
luposlip opened this issue Jan 30, 2023 · 0 comments
Open
2 tasks

nd-db index transferability #13

luposlip opened this issue Jan 30, 2023 · 0 comments
Assignees

Comments

@luposlip
Copy link
Owner

Situation:
I'm using a 100GB large .ndjson file, for which it takes several minutes to generate (and persist) an index (.nddbmeta).

When moving the database to another computer, I copy the .nddbmeta together with the .ndjson database.

Problem is - the other machine (an Amazon Linux instance running in EC2) doesn't recognize the pre-generated .nddbmeta as the correct one (generated on a MBP m1), so it generates a new one.

TODO:

  • nd-db seems to not generate the same sha ID for the same database on the 2 machines, thus generating a new one
  • On Amazon Linux the generated index doesn't work properly (the parsing of the document seems to yield wrong start/length data!?)

NB: nd-db has previously been tested on different variants of Ubuntu, plus on Mac, without any issues.

@luposlip luposlip self-assigned this Jan 30, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant