Possible race condition when tasks stores to the queue

[strangeman]

I'm sorry for the possibly improper report, but I cannot reproduce it on the repeatable basis.

Sometimes, when we have many launched tasks in the same project (one running, others waiting in the queue) we experience strange task crashes on ansible-galaxy install step.

Log example:

20:52:42: Preparing: 6316
20:52:42: Prepare task with template: Update Some Service on Test

20:52:42: access key ansible installed
20:52:42: Updating repository ssh://git@xxxxxxxxxxx/ansible/ansible-playbooks.git
20:52:42: Warning: Permanently added '[xxxxxxxxxxx],[xxxxxxxxxxx]' (RSA) to the list of known hosts.
20:52:42: From ssh://git@xxxxxxxxxxx/ansible/ansible-playbooks.git
20:52:42:  * branch            master     -> FETCH_HEAD
20:52:42: Already up-to-date.
20:52:42: access key ansible installed
20:52:42: installing static inventory
20:52:43: - changing role nginx from master to master
20:52:43: - extracting nginx to /opt/semaphore/repository_2/roles/nginx
20:52:43: - nginx (master) was installed successfully

<Omitted, other roles installations here>

20:53:49: Started: 6316
20:53:49: Run task with template: Update Some Service on Test

20:54:01: ERROR! the role 'nginx' was not found in /opt/semaphore/repository_2/roles:/opt/semaphore/repository_2/roles:/opt/semaphore/roles:/opt/semaphore/repository_2/roles:/opt/semaphore/repository_2
20:54:01:
20:54:01: The error appears to have been in '/opt/semaphore/repository_2/roles/app-reverse-proxy/meta/main.yml': line 16, column 5, but may
20:54:01: be elsewhere in the file depending on the exact syntax problem.
20:54:01:
20:54:01: The offending line appears to be:
20:54:01:
20:54:01: dependencies:
20:54:01:   - role: nginx
20:54:01:     ^ here
20:54:01: Running playbook failed: exit status 1

I thought (but this is only an assumption), that there is the issue somewhere in queue logic: next task trying install roles instead of waiting and delete role folder (because ansible-galaxy runs with --force flag) at the same moment when this role needed to the task.

[strangeman]

Closed by #546