New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
space:truncate() (and sometimes even space:delete()) doesn't work when memtx_memory is reached #3807
Comments
Same for me Tarantool 1.9.2-7-g3736c3798 Also reproduces in Tarantool 1.10.2-66-g2972cfcee, Ubuntu 18.04.1 LTS Sometimes this bug affects space:delete() operation. It falls with same error. After instance being restarted delete becomes possible. |
In bps_tree_create_leaf we use matras_alloc in case bps_tree_garbage_pop didn't work out. However it also might not succeed. Then we need to return NULL instead of dereferencing NULL pointer. Prerequisites: #3807
memtx_index_extent_reserve was always being called from memtx_space_replace_all_keys. It could lead to slab allocator failure while reaching memtx_memory limit, which is especially distressing on space:truncate() and space:delete() invocations. we Now it is not being called if not needed, including space:truncate() and space:delete() cases. Part of #3807
Trying to perform space:truncate() while reaching memtx_memory limit we could experience slab allocator failure on allocation for service tuple being upserted into service space to keep record of truncations. This behavior seems to be quite surprising for users. Now we are using calloc() to allocate tuples in BOX_TRUNCATE_ID space. Closes: #3807
Trying to perform space:truncate() while reaching memtx_memory limit we could experience slab allocator failure on allocation for service tuple being upserted into service space to keep record of truncations. This behavior seems to be quite surprising for users. Now we are using calloc() to allocate tuples in BOX_TRUNCATE_ID space. Closes: #3807
In bps_tree_create_leaf we use matras_alloc in case bps_tree_garbage_pop didn't work out. However it also might not succeed. Then we need to return NULL instead of dereferencing NULL pointer. Part of: #3807
In bps_tree_create_leaf we use matras_alloc in case bps_tree_garbage_pop didn't work out. However it also might not succeed. Then we need to return NULL instead of dereferencing NULL pointer. Part of: #3807
Trying to perform space:truncate() and space:delete() while reaching memtx_memory limit we could experience slab allocator failure. This behavior seems to be quite surprising for users. Now we are increasing memtx quota if needed for truncation or deletion. After performing it quota is being set back to the previous value if possible, while it should be so for almost any case, since we are meant to free some space during deletion or truncation. Closes #3807
Trying to perform space:truncate() and space:delete() while reaching memtx_memory limit we could experience slab allocator failure. This behavior seems to be quite surprising for users. Now we are increasing memtx quota if needed for truncation or deletion. After performing it quota is being set back to the previous value if possible, while it should be so for almost any case, since we are meant to free some space during deletion or truncation. Closes #3807
Trying to perform space:truncate() and space:delete() while reaching memtx_memory limit we could experience slab allocator failure. This behavior seems to be quite surprising for users. Now we are increasing memtx quota if needed for truncation or deletion. After performing it quota is being set back to the previous value if possible, while it should be so for almost any case, since we are meant to free some space during deletion or truncation. Closes #3807
In bps_tree_create_leaf we use matras_alloc in case bps_tree_garbage_pop didn't work out. However it also might not succeed. Then we need to return NULL instead of dereferencing NULL pointer. Part of: #3807
Trying to perform space:truncate() and space:delete() while reaching memtx_memory limit we could experience slab allocator failure. This behavior seems to be quite surprising for users. Now we are increasing memtx quota if needed for truncation or deletion. After performing it quota is being set back to the previous value if possible, while it should be so for almost any case, since we are meant to free some space during deletion or truncation. Closes #3807
By default the quota is strict. If it is set to false, quota_use will allow to overuse the total available memory limit. Part of tarantool/tarantool#3807
In bps_tree_create_leaf we use matras_alloc in case bps_tree_garbage_pop didn't work out. However it also might not succeed. Then we need to return NULL instead of dereferencing NULL pointer. Part of #3807
Trying to perform space:truncate() and space:delete() while reaching memtx_memory limit we could experience slab allocator failure. This behavior seems to be quite surprising for users. Now we are allowing to overuse memtx quota if needed for truncation or deletion, using flag in struct quota. After performing truncation or deletion we reset the flag so that quota can be overused any more. Closes #3807
By default the quota is enabled. If it is set to false, quota_use will allow to overuse the total available memory limit. Part of tarantool/tarantool#3807
By default the quota is enabled. If it is set to false, quota_use will allow to overuse the total available memory limit. Part of tarantool/tarantool#3807
In bps_tree_create_leaf we use matras_alloc in case bps_tree_garbage_pop didn't work out. However it also might not succeed. Then we need to return NULL instead of dereferencing NULL pointer. Part of #3807
Trying to perform space:truncate() and space:delete() while reaching memtx_memory limit we could experience slab allocator failure. This behavior seems to be quite surprising for users. Now we are allowing to overuse memtx quota if needed for truncation or deletion, using flag in struct quota. After performing truncation or deletion we reset the flag so that quota can't be overused any more. Closes #3807
Trying to perform space:truncate() and space:delete() while reaching memtx_memory limit we could experience slab allocator failure. Now it is solved through quota overuse tech. This commit introduces corresponding tests. Part of #3807
By default the quota is enabled. If it is set to false, quota_use will allow to overuse the total available memory limit. In case of disabled quota smalloc() allocates (and frees) large slabs using malloc so that the quota will be able to shrink back after those slabs are freed. Test introduced. Part of tarantool/tarantool#3807
Trying to perform space:truncate() while reaching memtx_memory limit we could experience slab allocator failure. This behavior seems to be quite surprising for users. Now we are allowing to overuse memtx quota for tuples in space _truncate using flag in struct quota. Truncate tuples are only being allocated with large slabs using malloc so that the quota can shrink back when they are freed. Closes #3807
By default the quota is enabled. If it is set to false, quota_use will allow to overuse the total available memory limit. In case of disabled quota smalloc() allocates (and frees) large slabs using malloc so that the quota will be able to shrink back after those slabs are freed. Test introduced. Part of tarantool/tarantool#3807
Trying to perform space:truncate() while reaching memtx_memory limit we could experience slab allocator failure. This behavior seems to be quite surprising for users. Now we are allowing to overuse memtx quota for tuples in space _truncate using flag in struct quota. Truncate tuples are only being allocated with large slabs using malloc so that the quota can shrink back when they are freed. Closes #3807
By default the quota is enabled. If it is set to false, quota_use will allow to overuse the total available memory limit. In case of disabled quota smalloc() allocates (and frees) large slabs using malloc so that the quota will be able to shrink back after those slabs are freed. Test introduced. Part of tarantool/tarantool#3807
Trying to perform space:truncate() while reaching memtx_memory limit we could experience slab allocator failure. This behavior seems to be quite surprising for users. Now we are allowing to overuse memtx quota for tuples in space _truncate using flag in struct quota. Truncate tuples are only being allocated with large slabs using malloc so that the quota can shrink back when they are freed. Closes #3807
By default the quota is enabled. If it is set to false, quota_use will allow to overuse the total available memory limit. In case of disabled quota smalloc() allocates (and frees) large slabs using malloc so that the quota will be able to shrink back after those slabs are freed. Test introduced. Part of tarantool/tarantool#3807
Trying to perform space:truncate() while reaching memtx_memory limit we could experience slab allocator failure. This behavior seems to be quite surprising for users. Now we are allowing to overuse memtx quota for tuples in space _truncate using flag in struct quota. Truncate tuples are only being allocated with large slabs using malloc so that the quota can shrink back when they are freed. Closes #3807
Trying to perform space:truncate() while reaching memtx_memory limit we could experience slab allocator failure. This behavior seems to be quite surprising for users. Now we are allowing to overuse memtx quota for tuples in space _truncate using flag in struct quota. Truncate tuples are only being allocated with large slabs using malloc so that the quota can shrink back when they are freed. Closes #3807
Reproducer for truncate: local yaml = require('yaml')
box.cfg({memtx_memory = 32*1024*1024})
local space = box.schema.space.create('test_space', { if_not_exists = true })
space:format({{ name = 'key', type = 'unsigned'}, { name = 'value', type = 'string'}})
space:create_index('primary', { parts = {{ field = 1, type = 'unsigned' }, { field = 2, type = 'string' }}, if_not_exists = true })
space:create_index('secondary', { parts = {{ field = 2, type = 'string' }}, if_not_exists = true })
local count = space:len()
while true do
count = count + 1
if not select(1, pcall(function() box.space.test_space:insert({count, 'Key number: '..tostring(count)}) end)) then
break
end
if count % 10000 == 0 then
print(count)
end
end
print('"test_space" length: '.. tostring(box.space.test_space:len()))
print('Arena:\n'..yaml.encode(box.slab.info()))
print('Try to truncate space:')
box.space.test_space:truncate() Log: |
Same issue at TEST stand on pretty huge (40gb) instance (and multiple replicasets). Used workaround was pretty time-consuming:
|
We hit tarantool#3807 in release/2.11 for release ASAN build with ASAN-friendly small allocators. Follow-up tarantool#7327 NO_CHANGELOG=internal NO_DOC=internal
We hit tarantool#3807 in release/2.11 for release ASAN build with ASAN-friendly small allocators. Follow-up tarantool#7327 NO_CHANGELOG=internal NO_DOC=internal
We hit tarantool#3807 in release/2.11 for release ASAN build with ASAN-friendly small allocators. Follow-up tarantool#7327 NO_CHANGELOG=internal NO_DOC=internal
Tarantool version: 1.10.1-370-ge93dc30
OS version: centos 7
Bug description:
Sometimes when memtx_memory limit is reached space:truncate() doesn't work with error
Failed to allocate 21 bytes in slab allocator for memtx_tuple
.The text was updated successfully, but these errors were encountered: