Skip to content
This repository has been archived by the owner on Mar 9, 2019. It is now read-only.

Database file size not updating after reaching 1GB #743

Open
ghost opened this issue Feb 12, 2018 · 3 comments
Open

Database file size not updating after reaching 1GB #743

ghost opened this issue Feb 12, 2018 · 3 comments

Comments

@ghost
Copy link

ghost commented Feb 12, 2018

Hello guys,
there is a problem in file size updating in boltdb after reaching 1GB (it occurs during file remapping). The problem is still present in the master version.
The following are the technical details (before start, I want to say that my OS pageSize, on a Windows 64 bit machine is equal to 4.096 bytes):
When boltdb needs to allocate new "junk" to store file, this method is called:
immagine
as you can see, when is needed to increase the store file is called the method "mmap" with a minimum size reallocation parameter. Inside "mmap" is called the method "mmapSize", that calculate the new store file size based on the minimum one passed as parameter:
immagine
as you can see from my red comments, there are 2 different calculation methods inside "mmapSize". The first one calculate the new store size following this rule, based on the minimum size parameter:
2^15 = 32.768 bytes
2^16 = 65.536 bytes
2^17 = 131.072 bytes
2^18 = 262.144 bytes
2^19 = 524.288 bytes
2^20 = 1.048.576 bytes
2^21 = 2.097.152 bytes
2^22 = 4.194.304 bytes
2^23 = 8.388.608 bytes
2^24 = 16.777.216 bytes
2^25 = 33.554.432 bytes
2^26 = 67.108.864 bytes
2^27 = 134.217.728 bytes
2^28 = 268.435.456 bytes
2^29 = 536.870.912 bytes
2^30 = 1.073.741.824 bytes
and this method works well.
The problems starts when the second calculation method take control over remap size, because the minimum size that is passed as parameter when 1GB is reached was "1.073.741.824", and not as I was expecting a value greater than "1.073.741.824".
So, when 1GB is reached, simply the mmapSize return the same minimum size passed as parameter (1.073.741.824), because following codes:
immagine
are never reached.

Can you please verify?

P.s.: I want to reference this bug on gitea project: go-gitea/gitea#3457

@ghost
Copy link
Author

ghost commented Feb 19, 2018

Are someone checking this issue?

@heyitsanthony
Copy link

heyitsanthony commented Feb 19, 2018

@giudon this doesn't look like a bug; the code allocates 1GB chunks after the first 1GB. If given a 1GB minimum size, it should return 1GB. If given a 1.01GB minimum size, the returned size should be 2GB. So long as mmapSize(size) >= size, it should work OK.

@ghost
Copy link
Author

ghost commented Feb 21, 2018

@heyitsanthony yes, I was expecting that boltdb work like your description, as I also seen from code, but we are also experiencing the issue that I described above. There is something that is not working properly

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant