Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Process large ASCII merger tree files in blocks #2

Open
abensonca opened this issue Apr 3, 2013 · 1 comment
Open

Process large ASCII merger tree files in blocks #2

abensonca opened this issue Apr 3, 2013 · 1 comment
Labels

Comments

@abensonca
Copy link
Collaborator

Original report by Andrew Benson (Bitbucket: abensonca, GitHub: abensonca).


Using, e.g. Simple_Merger_Tree_File_Maker.exe on very large input files will run out of memory. Could fix this by processing trees in batches.

@abensonca
Copy link
Collaborator Author

Original comment by Andrew Benson (Bitbucket: abensonca, GitHub: abensonca).


Another approach (suggested by Jaswant Yadav), would be to allocate space for just one property (e.g. mass, position, etc.) at a time, read that property, then store to file. This might be easier to code, but would require multiple passes through file I/O. It might also be less flexible overall - if you had a really big tree file then it could overflow memory even reading one property at a time, whereas processing in blocks might not.

@abensonca abensonca added minor enhancement New feature or request labels Aug 24, 2019
abensonca added a commit that referenced this issue Dec 2, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

1 participant