-
Notifications
You must be signed in to change notification settings - Fork 4
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Joining and sorting ranks fails to load serialized double array holding page rank scores #5
Comments
The failure loading of a double array is reproducible with any array if the size of the serialization file is 2 GiB or more, see 49174af. Downgrading fastutil-core to 8.5.7 fixes the issue, the unit test also passes using the current head of fastutil (vigna/fastutil@b813824). See also question about release of fastutil 8.5.9. |
Fixed by downgrading to fastutil-core 8.5.7 |
- rename existing option `--no-strict-domain-validate` into `--multipart-suffixes-as-domains`, complete documentatin - add unit tests Log progress when processing large graphs HostToDomainGraph / JoinSortRanks: input/output is always UTF-8 (fixes #5): write output always as UTF-8
- rename existing option `--no-strict-domain-validate` into `--multipart-suffixes-as-domains`, complete documentatin - add unit tests Log progress when processing large graphs HostToDomainGraph / JoinSortRanks: input/output is always UTF-8 (fixes #5): write output always as UTF-8
space to execute unit test using large arrays
space to execute unit test using large arrays
- rename existing option `--no-strict-domain-validate` into `--multipart-suffixes-as-domains`, complete documentatin - add unit tests Log progress when processing large graphs HostToDomainGraph / JoinSortRanks: input/output is always UTF-8 (fixes #5): write output always as UTF-8
Thank you for reporting this. Fixed in fastutil 8.5.9. |
(loading of a serialized double array in a 2 GiB file fails) - reset array after storing it and call garbage collector to free memory - more verbose logging after steps in unit test
The text was updated successfully, but these errors were encountered: