[Lock] Remove released semaphore

[jderusse]

Q	A
Branch?	3.4
Bug fix?	yes
New feature?	no
BC breaks?	no
Deprecations?	no
Tests pass?	yes
Fixed tickets	#27356
License	MIT
Doc PR	NA

This PR remove the semaphore with sem_remove. By removing without releasing the semaphore, all pending blocking acquiring will fail that's why the acquire method has also been update to handle such case

[stof]

Can we have tests reproducing the issue ? AFAIK, all tests of the SemaphoreStore are green with the existing implementation, so having it non-working means that the testsuite is not covering things properly.

[stof]

and you worked on an outdated version of Symfony master (or you targeted the wrong branch when opening the PR), as this conflicts.

[jderusse]

Added a test case and fix target branch

[stof]

I'd rather see a test about the behavior of the store (to prevent the bug beign reported) rather than a test enforcing this specific implementation (which does not prove that this implementation is the right one, but forbids replacing it)

[jderusse]

@stof unless I missed a point, the bug reported is semaphores are not removed which fill the system device and end with an error "no space left on device". It's kinda tracking a memory leak or a un-removed files which fill a disk, that's why the bug exists whereas every tests were green.

The test asserts that amount of semaphores used on the OS does not increase when the lock is acquired then released, regardless of the implementation.
Moreover a second assertion validate the test itself by checking that the amount of semaphore has been increased when the lock was acquired, which proves that this is the right way to count the amount of semaphore used by the OS

An other way to test it would be to reproduce the reported bug. Meaning either

• creating more than 32 000 semaphores (default value) which take ~ 2 minutes on my machine
• reducing this limit by changing a kernel parameter in /etc/sysctl.conf (requires sudo and restart)

Do you see another way to test the behavior?

[stof]

Is sem_release a useless API then ?

[jderusse]

Wouldn't say so.

sem_release is the right way to release a lock, next pending sem_acquire will softly take the lock. BUT, it does not free the resource. If nobody want to acquire the lock, the resource will stay here (consuming 1 slot on the 32 000 available). AND because of cross concurrency, you can not be sure that nobody else use the semaphore, then you can not remove the resource safely...

sem_remove is the brute way to the release the resource. By removing it: every pending sem_acquire will fail.

If your application can control the amount of distinguished resource, you should sem_release. If not, the single workaround I found is to use the sem_remove and deal with the failure.

For the record, we have exactly the same issue with the original implementation of flock, the file is not removed and consume 1 inode, but with more than millions of inodes available, this bug has not yet been reported...

[nicolas-grekas]

Just wondering: can this happen at all now? do we need some sort of timeout in the while above?

[jderusse]

yes, if \PHP_VERSION_ID >= 50601 and !$blocking

[nicolas-grekas]

the value of $blocking is known here

[jderusse]

fixed

[nicolas-grekas]

This loop can be moved outside of the if/else

[nicolas-grekas]

By swapping the order of the if/else, we can simplify the condition: if (PHP>=...) elseif !$blovking throw

[jderusse]

done

[nicolas-grekas]

Thank you @jderusse.

[pdias]

I do not know if there is anything to do with this topic, but in a console command I do this:

``
protected function execute(InputInterface $input, OutputInterface $output)
{
if (!$this->lock()) {
$output->writeln('The command is already running in another process.');
return 0;
}

    "...My task…"        

    $this->release();
}

``

And i have this warning:

PHP Fatal error: Uncaught Symfony\Component\Debug\Exception\ContextErrorException: Warning: sem_remove(): failed for SysV sempphore 139798696744064: Operation not permitted in /home/dev-my/vendor/symfony/symfony/src/Symfony/Component/Lock/Store/SemaphoreStore.php:113 Stack trace: #0 /home/dev-my/vendor/symfony/symfony/src/Symfony/Component/Lock/Lock.php(147): Symfony\Component\Lock\Store\SemaphoreStore->delete(Object(Symfony\Component\Lock\Key)) #1 /home/dev-my/vendor/symfony/symfony/src/Symfony/Component/Lock/Lock.php(63): Symfony\Component\Lock\Lock->release() #2 [internal function]: Symfony\Component\Lock\Lock->__destruct() #3 {main} thrown in /home/dev-my/vendor/symfony/symfony/src/Symfony/Component/Lock/Store/SemaphoreStore.php on line 113

I'm using SF3.4.

Thanks.