fifo: fix data race for put()/take() with vinyl #64
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Pull Request Test Coverage Report for Build 6096673092
💛 - Coveralls |
LeonidVas
approved these changes
Aug 10, 2023
oleg-jukovec
force-pushed
the
oleg-jukovec/gh-no-fifo-fix-vinyl
branch
2 times, most recently
from
12c6414
to
2289364
Compare
better0fdead
approved these changes
Aug 10, 2023
oleg-jukovec
force-pushed
the
oleg-jukovec/gh-no-fifo-fix-vinyl
branch
from
2289364
to
170a299
Compare
put()/take() contains two parts: 1. Get a tuple from an index. 2. Insert a next tuple (with id based on the data from the first call)/update the current one. For the vinyl engine it could lead to yield on the second step. As result we could put a tuple with the same id or got the same task multiple times. This is a potential problem. It is very unlikely to get yield on the second place because changes occur in the same place in memory. So it is should not to be flushed from memory to disk right after getting a tuple. But it could happen on practice, see a problem with auto increment and concurrent lookups [1]. This issue has been fixed before for fifottl driver [2][3]. Here I tried to fix it in a similar code style to make it easier to maintain the code. 1. tarantool/tarantool#389 2. #28 3. #30
By default idx:max() or idx:min() have read confirmed isolation level. It could lead to a task duplication or double task take when we already insert or update a task, commited, but it is not yet confirmed. See also: 1. tarantool/queue#207 2. tarantool/queue#211
oleg-jukovec
force-pushed
the
oleg-jukovec/gh-no-fifo-fix-vinyl
branch
from
170a299
to
c5cfb9b
Compare
oleg-jukovec
added a commit
that referenced
this pull request
Apr 16, 2024
Overview
The release introduces roles for Tarantool 3 and improves the module
metrics.
Breaking changes
Metric `sharded_queue_calls` renamed to
`tnt_sharded_queue_api_statistics_calls_total` (#71). The metric
now has labels in the format
`{name = "tube_name", state = "call_type"}` instead of
`{name = "tube_name", status = "call_type"}`.
Metric `sharded_queue_tasks` renamed to
`tnt_sharded_queue_api_statistics_tasks` (#71). The metric now
has labels in the format `{name = "tube_name", state = "task_state"}`
instead of `{name = "tube_name", status = "task_state"}`.
The dependency `cartridge` is removed from the `rockspec` since
the module does not require it to work with Tarantool 3 (#68).
New features
Role `roles.sharded-queue-router` for Tarantool 3 (#68).
Role `roles.sharded-queue-storage` for Tarantool 3 (#68).
Metric `tnt_sharded_queue_api_role_stats` is a summary with
quantiles of `sharded_queue.api` role API calls (#71).
The metric includes a counter of API calls and errors.
The metric contains labels in the following format:
`{name = "tube_name", method = "api_call_method", status = "ok" or "error"}`
Metric `tnt_sharded_queue_storage_role_stats` is a summary with
quantiles of `sharded_queue.storage` role API calls (#71).
The metric includes a counter of API calls and errors.
The metric contains labels in the following format:
`{name = "tube_name", method = "api_call_method", status = "ok" or "error"}`
Metric `tnt_sharded_queue_storage_statistics_calls_total` as
an equivalent of `tnt_sharded_queue_api_statistics_calls_total`
for the `sharded_queue.storage` role.
Values have the same meaning as the `queue` statistics `calls`
table.
The metric contains labels in the following format:
`{name = "tube_name", state = "call_type"}`
Metric `tnt_sharded_queue_storage_statistics_tasks` as an
equivalent of `tnt_sharded_queue_api_statistics_tasks` for
the `sharded_queue.storage` role (#71).
Values have the same meaning as the `queue` statistics `tasks`
table.
The metric contains labels in the following format:
`{name = "tube_name", state = "task_state"}`
Bugfixes
Data race with fifo driver for put()/take() methods with vinyl
engine (#64).
Merged
oleg-jukovec
added a commit
that referenced
this pull request
Apr 16, 2024
Overview
The release introduces roles for Tarantool 3 and improves the module
metrics.
Breaking changes
Metric `sharded_queue_calls` renamed to
`tnt_sharded_queue_api_statistics_calls_total` (#71). The metric
now has labels in the format
`{name = "tube_name", state = "call_type"}` instead of
`{name = "tube_name", status = "call_type"}`.
Metric `sharded_queue_tasks` renamed to
`tnt_sharded_queue_api_statistics_tasks` (#71). The metric now
has labels in the format `{name = "tube_name", state = "task_state"}`
instead of `{name = "tube_name", status = "task_state"}`.
The dependency `cartridge` is removed from the `rockspec` since
the module does not require it to work with Tarantool 3 (#68).
New features
Role `roles.sharded-queue-router` for Tarantool 3 (#68).
Role `roles.sharded-queue-storage` for Tarantool 3 (#68).
Metric `tnt_sharded_queue_api_role_stats` is a summary with
quantiles of `sharded_queue.api` role API calls (#71).
The metric includes a counter of API calls and errors.
The metric contains labels in the following format:
`{name = "tube_name", method = "api_call_method", status = "ok" or "error"}`
Metric `tnt_sharded_queue_storage_role_stats` is a summary with
quantiles of `sharded_queue.storage` role API calls (#71).
The metric includes a counter of API calls and errors.
The metric contains labels in the following format:
`{name = "tube_name", method = "api_call_method", status = "ok" or "error"}`
Metric `tnt_sharded_queue_storage_statistics_calls_total` as
an equivalent of `tnt_sharded_queue_api_statistics_calls_total`
for the `sharded_queue.storage` role (#71).
Values have the same meaning as the `queue` statistics `calls`
table.
The metric contains labels in the following format:
`{name = "tube_name", state = "call_type"}`
Metric `tnt_sharded_queue_storage_statistics_tasks` as an
equivalent of `tnt_sharded_queue_api_statistics_tasks` for
the `sharded_queue.storage` role (#71).
Values have the same meaning as the `queue` statistics `tasks`
table.
The metric contains labels in the following format:
`{name = "tube_name", state = "task_state"}`
Bugfixes
Data race with fifo driver for put()/take() methods with vinyl
engine (#64).
oleg-jukovec
added a commit
that referenced
this pull request
Apr 16, 2024
Overview
The release introduces roles for Tarantool 3 and improves the module
metrics.
Breaking changes
Metric `sharded_queue_calls` renamed to
`tnt_sharded_queue_api_statistics_calls_total` (#71). The metric
now has labels in the format
`{name = "tube_name", state = "call_type"}` instead of
`{name = "tube_name", status = "call_type"}`.
Metric `sharded_queue_tasks` renamed to
`tnt_sharded_queue_api_statistics_tasks` (#71). The metric now
has labels in the format `{name = "tube_name", state = "task_state"}`
instead of `{name = "tube_name", status = "task_state"}`.
The dependency `cartridge` is removed from the `rockspec` since
the module does not require it to work with Tarantool 3 (#68).
New features
Role `roles.sharded-queue-router` for Tarantool 3 (#68).
Role `roles.sharded-queue-storage` for Tarantool 3 (#68).
Metric `tnt_sharded_queue_api_role_stats` is a summary with
quantiles of `sharded_queue.api` role API calls (#71).
The metric includes a counter of API calls and errors.
The metric contains labels in the following format:
`{name = "tube_name", method = "api_call_method", status = "ok" or "error"}`
Metric `tnt_sharded_queue_storage_role_stats` is a summary with
quantiles of `sharded_queue.storage` role API calls (#71).
The metric includes a counter of API calls and errors.
The metric contains labels in the following format:
`{name = "tube_name", method = "api_call_method", status = "ok" or "error"}`
Metric `tnt_sharded_queue_storage_statistics_calls_total` as
an equivalent of `tnt_sharded_queue_api_statistics_calls_total`
for the `sharded_queue.storage` role (#71).
Values have the same meaning as the `queue` statistics `calls`
table.
The metric contains labels in the following format:
`{name = "tube_name", state = "call_type"}`
Metric `tnt_sharded_queue_storage_statistics_tasks` as an
equivalent of `tnt_sharded_queue_api_statistics_tasks` for
the `sharded_queue.storage` role (#71).
Values have the same meaning as the `queue` statistics `tasks`
table.
The metric contains labels in the following format:
`{name = "tube_name", state = "task_state"}`
Bugfixes
Data race with fifo driver for put()/take() methods with vinyl
engine (#64).
oleg-jukovec
added a commit
that referenced
this pull request
Apr 16, 2024
Overview
The release introduces roles for Tarantool 3 and improves the module
metrics.
Breaking changes
Metric `sharded_queue_calls` renamed to
`tnt_sharded_queue_api_statistics_calls_total` (#71). The metric
now has labels in the format
`{name = "tube_name", state = "call_type"}` instead of
`{name = "tube_name", status = "call_type"}`.
Metric `sharded_queue_tasks` renamed to
`tnt_sharded_queue_api_statistics_tasks` (#71). The metric now
has labels in the format `{name = "tube_name", state = "task_state"}`
instead of `{name = "tube_name", status = "task_state"}`.
The dependency `cartridge` is removed from the `rockspec` since
the module does not require it to work with Tarantool 3 (#68).
New features
Role `roles.sharded-queue-router` for Tarantool 3 (#68).
Role `roles.sharded-queue-storage` for Tarantool 3 (#68).
Metric `tnt_sharded_queue_api_role_stats` is a summary with
quantiles of `sharded_queue.api` role API calls (#71).
The metric includes a counter of API calls and errors.
The metric contains labels in the following format:
`{name = "tube_name", method = "api_call_method", status = "ok" or "error"}`
Metric `tnt_sharded_queue_storage_role_stats` is a summary with
quantiles of `sharded_queue.storage` role API calls (#71).
The metric includes a counter of API calls and errors.
The metric contains labels in the following format:
`{name = "tube_name", method = "api_call_method", status = "ok" or "error"}`
Metric `tnt_sharded_queue_storage_statistics_calls_total` as
an equivalent of `tnt_sharded_queue_api_statistics_calls_total`
for the `sharded_queue.storage` role (#71).
Values have the same meaning as the `queue` statistics `calls`
table.
The metric contains labels in the following format:
`{name = "tube_name", state = "call_type"}`
Metric `tnt_sharded_queue_storage_statistics_tasks` as an
equivalent of `tnt_sharded_queue_api_statistics_tasks` for
the `sharded_queue.storage` role (#71).
Values have the same meaning as the `queue` statistics `tasks`
table.
The metric contains labels in the following format:
`{name = "tube_name", state = "task_state"}`
Bugfixes
Data race with fifo driver for put()/take() methods with vinyl
engine (#64).
oleg-jukovec
added a commit
that referenced
this pull request
Apr 17, 2024
Overview
The release introduces roles for Tarantool 3 and improves the module
metrics.
Breaking changes
Metric `sharded_queue_calls` renamed to
`tnt_sharded_queue_api_statistics_calls_total` (#71). The metric
now has labels in the format
`{name = "tube_name", state = "call_type"}` instead of
`{name = "tube_name", status = "call_type"}`.
Metric `sharded_queue_tasks` renamed to
`tnt_sharded_queue_api_statistics_tasks` (#71). The metric now
has labels in the format `{name = "tube_name", state = "task_state"}`
instead of `{name = "tube_name", status = "task_state"}`.
The dependency `cartridge` is removed from the `rockspec` since
the module does not require it to work with Tarantool 3 (#68).
New features
Role `roles.sharded-queue-router` for Tarantool 3 (#68).
Role `roles.sharded-queue-storage` for Tarantool 3 (#68).
Metric `tnt_sharded_queue_api_role_stats` is a summary with
quantiles of `sharded_queue.api` role API calls (#71).
The metric includes a counter of API calls and errors.
The metric contains labels in the following format:
`{name = "tube_name", method = "api_call_method", status = "ok" or "error"}`
Metric `tnt_sharded_queue_storage_role_stats` is a summary with
quantiles of `sharded_queue.storage` role API calls (#71).
The metric includes a counter of API calls and errors.
The metric contains labels in the following format:
`{name = "tube_name", method = "api_call_method", status = "ok" or "error"}`
Metric `tnt_sharded_queue_storage_statistics_calls_total` as
an equivalent of `tnt_sharded_queue_api_statistics_calls_total`
for the `sharded_queue.storage` role (#71).
Values have the same meaning as the `queue` statistics `calls`
table.
The metric contains labels in the following format:
`{name = "tube_name", state = "call_type"}`
Metric `tnt_sharded_queue_storage_statistics_tasks` as an
equivalent of `tnt_sharded_queue_api_statistics_tasks` for
the `sharded_queue.storage` role (#71).
Values have the same meaning as the `queue` statistics `tasks`
table.
The metric contains labels in the following format:
`{name = "tube_name", state = "task_state"}`
Bugfixes
Data race with fifo driver for put()/take() methods with vinyl
engine (#64).
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
put()/take() contains two parts:
For the vinyl engine it could lead to yield on the second step. As result we could put a tuple with the same id or got the same task multiple times.
This is a potential problem. It is very unlikely to get yield on the second place because changes occur in the same place in memory. So it is should not to be flushed from memory to disk right after getting a tuple. But it could happen on practice, see a problem with auto increment and concurrent lookups [1].
This issue has been fixed before for fifottl driver [2][3]. Here I tried to fix it in a similar code style to make it easier to maintain the code.
drivers: fix concurrency with mvcc
By default idx:max() or idx:min() have read confirmed isolation level.
It could lead to a task duplication or double task take when we
already insert or update a task, commited, but it is not yet
confirmed.
See also: