-
Notifications
You must be signed in to change notification settings - Fork 12
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Slow serialization with large Array data (> 60 Mbytes) #269
Comments
Interesting. I don't see any reason for Fuel to do that. There must be another mechanism at play. I currently don't have to look into this tough, in a week or two maybe. @tinchodias, do you have some time to spare maybe? |
Can you give me a rough timing estimate of what you mean by "slowly"? |
It was around 100K/sec |
This is very interesting. I found the issue to be time spent in |
Still investigating but interesting find: in Pharo 11 the string hashes appear to be badly distributed. |
Probably @guillep will be interested |
I'm not sure the hash distribution is bad per se, it's just that the hashes have only uneven numbers when using modulo 4096. It looks like |
Interesting finding :) |
I checked the usage of identity dictionaries. This kind of tells me that a better identity dictionary will drastically improve the performance of fuel. |
Brilliant, thanks! |
Oh nice, not just Fuel. The system relies a lot on IdentityDictionaries :) |
This issue has been automatically marked as stale because it has not had recent activity. It will remain open but will probably not come into focus. If you still think this should receive some attention, leave a comment. Thank you for your contributions. |
This issue has been automatically marked as stale because it has not had recent activity. It will remain open but will probably not come into focus. If you still think this should receive some attention, leave a comment. Thank you for your contributions. |
This issue has been automatically marked as stale because it has not had recent activity. It will remain open but will probably not come into focus. If you still think this should receive some attention, leave a comment. Thank you for your contributions. |
It seems that Fuel takes a long time serializing an Array of Arrays containing multiple Strings each one. When serialization passes around 60 Mbytes, it starts writing data in chunks of 100Kbytes and very slowly:
To reproduce please download this Fuel serialization: https://filesender.renater.fr/?s=download&token=b3809275-6258-4ee4-8d93-3b8871b7a0bd
Materialize it in a fresh Pharo 11 image:
and then from the Inspector window evaluate:
We tested in Pharo 11 under macOS.
The text was updated successfully, but these errors were encountered: