Named mutex not supported on Unix #5211

drieseng · 2016-02-28T09:41:31Z

Creating a named mutex on Unix currently throws a PlatformNotSupportedException stating "Named synchronization primitives are not supported on this platform".

Is this scheduled to arrive ? I'm sure @migueldeicaza will be eager to port this from Mono to .NET Core :p

Personally, I don't think it makes sense to expose APIs on both Windows and Unix, but only implement them on one of them. In what way does the Portability Analyzer even take this into account ?

mikedn · 2016-02-28T10:21:57Z

Related to #4370

drieseng · 2016-02-28T11:29:39Z

Duplicate of issue #4370.

mikedn · 2016-02-28T11:34:54Z

I wouldn't say it's a duplicate. That issue was resolved by removing the functionality. It would be nice to actually fix the actual problem and restore the functionality.

stephentoub · 2016-02-28T12:26:14Z

I'm sure @migueldeicaza will be eager to port this from Mono to .NET Core :p

Miguel can correct me if I'm wrong, but I don't believe Mono implements this either, or more specifically I believe they implement it the same general manner that the coreclr did before it was disabled to throw those exceptions: the names are only process-wide, not cross-process. That's why we decided to throw exceptions on coreclr; otherwise, since names are typically used for cross-process synchronization, you use the names thinking you're getting cross-process synchronization, but in reality you've just got a bunch of race conditions silently waiting to cause problems.

drieseng · 2016-02-29T18:39:41Z

Reopened because of @mikedn comment.

kouvel · 2016-05-05T19:40:25Z

My search for ways to add a cross-process mutex didn't yield many options. I'm currently planning on this:

On Linux, we have the pthread robust mutex that can be allocated in shared memory, so that's great
Elsewhere, it seems like file locks are the only decent option

Currently, we are looking for the following features:

Track lifetime across processes and users. One process/user should be able to create the mutex, and another process/user should be able to delete it. This eliminates the System V semaphore since only the creator or owner can delete a semaphore set.
Detect when no processes are using the mutex so that it can be deleted or reinitialized. File locks allow this detection.
A way to detect an abandoned mutex (similar to pthread robust mutex) even if the process owning the lock crashes or is killed.
Be able to do a timed wait. On OSX, this eliminates pthread named semaphore and pshared rwlock. For file locks too, we would need to poll for the lock in loop, sleeping in-between, which has other issues but may be workable if it's the only option. I also looked into Mach semaphores and kqueue, but their lifetime seems to be tied to the creator process.

Any other ideas?

CC @kangaroo

janvorli · 2016-05-05T20:39:24Z

cc @krytarowski for NetBSD

krytarowski · 2016-05-06T02:29:50Z

I don't have anything portable offhand, other than maybe <semaphore.h> or pthread_mutexattr_init(3).

Fixes #3422 - On systems that support pthread process-shared robust recursive mutexes, they will be used - On other systems, file locks are used. File locks unfortunately don't have a timeout in the blocking wait call, and I didn't find any other sync object with a timed wait with the necessary properties, so polling is done for timed waits. Shared memory files: - Session-scoped mutexes (name not prefixed, or prefixed with Local\) go in /tmp/coreclr/shm/session<sessionId>/<mutexName> - Globally-scoped mutexes (name prefixed with Global\) go in /tmp/coreclr/shm/global/<mutexName> - Contains shared state, and is mmap'ped into the process, see SharedMemorySharedDataHeader and NamedMutexSharedData for data stored - Creation and deletion is synchronized using an exclusive file lock on the shm directory - Any process using the shared memory file holds a shared file lock on the shared memory file - Upon creation, if the shared memory file already exists, an exclusive file lock is attempted on it, to see if the file data is valid. If no other processes have the mutex open, the file is reinitialized. - Upon releasing the last reference to a mutex in a process, it will try to get an exclusive lock on the shared memory file to see if any other processes have the mutex opened. If not, the file is deleted, along with the session directory if it's empty. The coreclr and shm directories are not deleted. - This allows managing the lifetime of mutex state based on active processes that have the mutex open. Depending on how the process terminated, the file may still be left over in the tmp directory, I haven't found anything that can be done about that. Lock files when using file locks: - In addition to the shared memory file, we need another file for the actual synchronization file lock, since a file lock on the shared memory file is used for lifetime purposes. - These files go in /tmp/coreclr/lockfiles/session<sessionId>|global/<mutexName> - The file is empty, and is only used for file locks Process data - See SharedMemoryProcessDataHeader and NamedMutexProcessData for data stored - Per mutex name, there is only one instance of process data that is ref-counted. They are currently stored in a linked list in SharedMemoryManager. It should use a hash table, but of the many hash table implementations that are already there, none seem to be easily usable in the PAL. I'll look into that and will fix later. - Refers to the associated shared memory, and knows how to clean up both the process data and shared data - When using file locks for synchronization, a process-local mutex is also created for synchronizing threads, since file locks are owned at the file descriptor level and there is only one open file descriptor in the process per mutex name. The process-local mutex is locked around the file lock, so that only one thread per process is ever trying to flock on a given file descriptor. Abandon detection - When a lock is acquired, the process data is added to a linked list on the owning thread - When a thread exits, the list is walked, each mutex is flagged as abandoned and released - For detecting process abruptly terminating, pthread robust mutexes give us that. When using file locks, the file lock is automatically released by the system. Upon acquiring a lock, the lock owner info in the shared memory is checked to see if the mutex was abandoned. Miscellaneous - CreateMutex and OpenMutex both create new handles for each mutex opened. Each handle just refers to the process data header for the mutex name. - Some of the above features are already available in the PAL, but not quite in a way that I can use for this purpose. The existing shared memory, naming, and waiting infrastructure is not suitable for this purpose, and is not used.

Related to dotnet/coreclr#5030 Related to dotnet/coreclr#3422

Fixes #3422 - On systems that support pthread process-shared robust recursive mutexes, they will be used - On other systems, file locks are used. File locks unfortunately don't have a timeout in the blocking wait call, and I didn't find any other sync object with a timed wait with the necessary properties, so polling is done for timed waits. Shared memory files: - Session-scoped mutexes (name not prefixed, or prefixed with Local\) go in /tmp/coreclr/shm/session<sessionId>/<mutexName> - Globally-scoped mutexes (name prefixed with Global\) go in /tmp/coreclr/shm/global/<mutexName> - Contains shared state, and is mmap'ped into the process, see SharedMemorySharedDataHeader and NamedMutexSharedData for data stored - Creation and deletion is synchronized using an exclusive file lock on the shm directory - Any process using the shared memory file holds a shared file lock on the shared memory file - Upon creation, if the shared memory file already exists, an exclusive file lock is attempted on it, to see if the file data is valid. If no other processes have the mutex open, the file is reinitialized. - Upon releasing the last reference to a mutex in a process, it will try to get an exclusive lock on the shared memory file to see if any other processes have the mutex opened. If not, the file is deleted, along with the session directory if it's empty. The coreclr and shm directories are not deleted. - This allows managing the lifetime of mutex state based on active processes that have the mutex open. Depending on how the process terminated, the file may still be left over in the tmp directory, I haven't found anything that can be done about that. Lock files when using file locks: - In addition to the shared memory file, we need another file for the actual synchronization file lock, since a file lock on the shared memory file is used for lifetime purposes. - These files go in /tmp/coreclr/lockfiles/session<sessionId>|global/<mutexName> - The file is empty, and is only used for file locks Process data - See SharedMemoryProcessDataHeader and NamedMutexProcessData for data stored - Per mutex name, there is only one instance of process data that is ref-counted. They are currently stored in a linked list in SharedMemoryManager. It should use a hash table, but of the many hash table implementations that are already there, none seem to be easily usable in the PAL. I'll look into that and will fix later. - Refers to the associated shared memory, and knows how to clean up both the process data and shared data - When using file locks for synchronization, a process-local mutex is also created for synchronizing threads, since file locks are owned at the file descriptor level and there is only one open file descriptor in the process per mutex name. The process-local mutex is locked around the file lock, so that only one thread per process is ever trying to flock on a given file descriptor. Abandon detection - When a lock is acquired, the process data is added to a linked list on the owning thread - When a thread exits, the list is walked, each mutex is flagged as abandoned and released - For detecting process abruptly terminating, pthread robust mutexes give us that. When using file locks, the file lock is automatically released by the system. Upon acquiring a lock, the lock owner info in the shared memory is checked to see if the mutex was abandoned. Miscellaneous - CreateMutex and OpenMutex both create new handles for each mutex opened. Each handle just refers to the process data header for the mutex name. - Some of the above features are already available in the PAL, but not quite in a way that I can use for this purpose. The existing shared memory, naming, and waiting infrastructure is not suitable for this purpose, and is not used.

The tests removed would fail soon after dotnet/coreclr#5030 is merged. Related to dotnet/coreclr#5030 Related to dotnet#8625 Related to dotnet/coreclr#3422

Fixes #3422 - On systems that support pthread process-shared robust recursive mutexes, they will be used - On other systems, file locks are used. File locks unfortunately don't have a timeout in the blocking wait call, and I didn't find any other sync object with a timed wait with the necessary properties, so polling is done for timed waits. Shared memory files: - Session-scoped mutexes (name not prefixed, or prefixed with Local\) go in /tmp/coreclr/shm/session<sessionId>/<mutexName> - Globally-scoped mutexes (name prefixed with Global\) go in /tmp/coreclr/shm/global/<mutexName> - Contains shared state, and is mmap'ped into the process, see SharedMemorySharedDataHeader and NamedMutexSharedData for data stored - Creation and deletion is synchronized using an exclusive file lock on the shm directory - Any process using the shared memory file holds a shared file lock on the shared memory file - Upon creation, if the shared memory file already exists, an exclusive file lock is attempted on it, to see if the file data is valid. If no other processes have the mutex open, the file is reinitialized. - Upon releasing the last reference to a mutex in a process, it will try to get an exclusive lock on the shared memory file to see if any other processes have the mutex opened. If not, the file is deleted, along with the session directory if it's empty. The coreclr and shm directories are not deleted. - This allows managing the lifetime of mutex state based on active processes that have the mutex open. Depending on how the process terminated, the file may still be left over in the tmp directory, I haven't found anything that can be done about that. Lock files when using file locks: - In addition to the shared memory file, we need another file for the actual synchronization file lock, since a file lock on the shared memory file is used for lifetime purposes. - These files go in /tmp/coreclr/lockfiles/session<sessionId>|global/<mutexName> - The file is empty, and is only used for file locks Process data - See SharedMemoryProcessDataHeader and NamedMutexProcessData for data stored - Per mutex name, there is only one instance of process data that is ref-counted. They are currently stored in a linked list in SharedMemoryManager. It should use a hash table, but of the many hash table implementations that are already there, none seem to be easily usable in the PAL. I'll look into that and will fix later. - Refers to the associated shared memory, and knows how to clean up both the process data and shared data - When using file locks for synchronization, a process-local mutex is also created for synchronizing threads, since file locks are owned at the file descriptor level and there is only one open file descriptor in the process per mutex name. The process-local mutex is locked around the file lock, so that only one thread per process is ever trying to flock on a given file descriptor. Abandon detection - When a lock is acquired, the process data is added to a linked list on the owning thread - When a thread exits, the list is walked, each mutex is flagged as abandoned and released - For detecting process abruptly terminating, pthread robust mutexes give us that. When using file locks, the file lock is automatically released by the system. Upon acquiring a lock, the lock owner info in the shared memory is checked to see if the mutex was abandoned. Miscellaneous - CreateMutex and OpenMutex both create new handles for each mutex opened. Each handle just refers to the process data header for the mutex name. - Some of the above features are already available in the PAL, but not quite in a way that I can use for this purpose. The existing shared memory, naming, and waiting infrastructure is not suitable for this purpose, and is not used.

Related to dotnet/coreclr#5030 Related to dotnet/coreclr#3422

stephentoub · 2016-06-05T12:23:22Z

Mono's create named semaphore

Mono's implementation is not cross-process.

ChangePlaces · 2017-02-17T21:47:30Z

I'm confused, what's the current state of named Mutexes in core? Will they work on windows and linux?

krytarowski · 2017-02-17T21:51:30Z

There was an implementation with POSIX robust mutexes on Linux.

ChangePlaces · 2017-02-17T21:59:53Z

so that's a yes? It'd be really nice if yes as I can get rid of the file locking hacks!

krytarowski · 2017-02-18T01:59:10Z

I guess so, this is why I need to implement POSIX robust mutexes on my platform (NetBSD).

ChangePlaces · 2017-02-18T02:51:52Z

I've just tried on Ubuntu 14.04 and the answer is a no

stephentoub · 2017-02-18T04:51:16Z

I've just tried on Ubuntu 14.04 and the answer is a no

What code did you try? Named mutexes should work fine on in Ubuntu.

ChangePlaces · 2017-02-18T05:18:01Z

Interesting, this is my code, other processes are not honouring the mutex and are acquiring the mutex and repeating the same work. It works in Windows as expected.

using (var m = new Mutex(false, mutexName))
{
    var hasMutex = false;
    try
    {
	if (m.WaitOne(2))
	{
	    hasMutex = true;
	    ProcessIterationAsync(currentOutputFileNameBase, c, _config.CountryCode).Wait();
	}
    }
    catch (AbandonedMutexException e)
    {
	continue;
    }
    catch (Exception e)
    {
        continue;
    }
    finally
    {
	if (hasMutex)
	{
	    m.ReleaseMutex();
	}
    }
}

kouvel · 2017-02-18T08:33:34Z

What name did you use? By default, names have session scope and sessions are more granular on Unix (each terminal gets its own session). If you haven't already, please try adding a "Global" prefix to the name minus the quotes.

There is build-time code to detect if the system supports POSIX robust mutexes, see https://github.com/dotnet/coreclr/blob/master/src/pal/src/configure.cmake#L997 and https://github.com/dotnet/coreclr/blob/master/src/pal/src/configure.cmake#L1020. Based on that, it would use POSIX robust mutexes or file locks, but it should work either way.

kouvel · 2017-02-18T08:40:37Z

Although unrelated, in the above code, when an AbandonedMutexException is thrown, the thread is still given ownership of the lock, so you'd have to ReleaseMutex on that path as well

ChangePlaces · 2017-02-18T10:10:24Z

I prefixed the name with "Global\"as suggested here to no avail. I am running the processes in multiple screen instances. This is with RC4 (installed with apt-get)

kouvel · 2017-02-19T06:27:40Z

Would it be possible to provide a small runnable repro? I can give it a try and see what's going on.

ChangePlaces · 2017-02-21T02:22:15Z

I took this down to the most basic example I could make and it worked... so I worked backwards and fixed. It was the missing "Global\" along with a few other things - User error - thanks and sorry!

kouvel · 2017-02-21T18:35:26Z

Ok great

nvoronchev · 2019-03-14T15:34:50Z

Hi!

I have try the following program in Debian 9 and it seems mutex is not global (cross-process):

using System;
using System.Threading;

namespace test_mutex
{
    class Program
    {
        static void Main(string[] args)
        {
            Console.WriteLine("Started! Try to Mutex.ctor...");
            var m = new Mutex(false, @"Global\aaaaa");

            Console.WriteLine("We have the Mutex! Try to lock mutex...");
            m.WaitOne();

            Console.WriteLine("Locked! Sleep...");
            for (int i = 5; i > 0; --i)
            {
                Console.WriteLine("  {0}", i);
                Thread.Sleep(1000);
            }

            Console.WriteLine("Sleep done! Release mutex...");
            m.ReleaseMutex();

            Console.WriteLine("Done.");
            m.Close();
        }
    }
}

I started two instances of the application and both of them counting down simultaneously.

Would you be so kind to clarify is it my mistake or Mono bug?

stephentoub · 2019-03-14T15:36:51Z

Would you be so kind to clarify is it my mistake or Mono bug?

Are you using .NET Core or Mono?

nvoronchev · 2019-03-14T15:39:01Z

@stephentoub, I try Mono 5.18.0.240 and 6.1.0 (master/72512cfbcbc Thu Feb 21 19:00:53 MSK 2019).

stephentoub · 2019-03-14T15:40:15Z

Thanks. This issue / repo are for coreclr. Can you open an issue instead in http://github.com/mono/mono?

nvoronchev · 2019-03-14T16:17:30Z

@stephentoub, Oh, I see. I chose wrong project. My apologies!

stephentoub · 2019-03-14T17:10:51Z

No worries; it's confusing :)

ritchxu · 2019-12-21T04:24:27Z

Sorry for pinging on old thread. What is the situation for Semaphore? Which seems to still only support named instance on Windows.

stephentoub · 2020-01-06T13:53:16Z

Which seems to still only support named instance on Windows.

Correct.

josegomez · 2020-04-20T23:32:53Z

Would you be so kind to clarify is it my mistake or Mono bug?

Are you using .NET Core or Mono?

@stephentoub is there documentation about how global mutex works on each platform supported by net core?
I’m trying to get a better understanding of the implementation for Linux, OS X and Win any pointers in the right direction would be appreciated (netcore 3.1)

stephentoub · 2020-04-24T22:10:38Z

is there documentation about how global mutex works on each platform supported by net core?
I’m trying to get a better understanding of the implementation for Linux, OS X and Win any pointers in the right direction would be appreciated (netcore 3.1)

Documentation about the implementation? I don't know if @kouvel published a design doc anywhere for the Unix implementations; he might be able to point you to something. Your best bet though is probably just the implementation source code.

josegomez · 2020-04-25T00:52:33Z

Your best bet though is probably just the implementation source code.

Any way you can point me to it? The code base is extensive and so far I've had no luck with it.

kouvel · 2020-04-27T17:02:38Z

The closest summary is probably from the original PR: dotnet/coreclr#5030. I thought they were in the code somewhere but apparently not, I have filed an issue to get those into code.

Most of the relevant code is here:

https://github.com/dotnet/runtime/blob/master/src/coreclr/src/pal/src/sharedmemory/sharedmemory.cpp
Below

runtime/src/coreclr/src/pal/src/synchobj/mutex.cpp

Line 926 in 24bfe6c

////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////

Roughly, they should behave the same way as on Windows. There may be some corner cases where they behave slightly differently, for example when the last handle to a mutex object is closed while the lock is held by a different thread.

kouvel · 2020-04-27T17:04:53Z

On Windows the native objects are used (CreateMutex, etc.), on Linuxes where pthread robust mutexes are supported, they are used, and on OSX file locks are used.

drieseng closed this as completed Feb 28, 2016

drieseng reopened this Feb 29, 2016

stephentoub assigned kouvel Apr 3, 2016

kouvel referenced this issue in kouvel/corefx May 17, 2016

Fix some named mutex tests

cf69142

Related to dotnet/coreclr#5030 Related to dotnet/coreclr#3422

kouvel referenced this issue in kouvel/corefx May 18, 2016

Remove named mutex PNSE tests that will fail soon

4c4924d

The tests removed would fail soon after dotnet/coreclr#5030 is merged. Related to dotnet/coreclr#5030 Related to dotnet#8625 Related to dotnet/coreclr#3422

kouvel closed this as completed in dotnet/coreclr#5030 May 20, 2016

kouvel referenced this issue in kouvel/corefx May 20, 2016

Fix some named mutex tests

719ed4b

Related to dotnet/coreclr#5030 Related to dotnet/coreclr#3422

kouvel referenced this issue in kouvel/corefx May 20, 2016

Fix some named mutex tests

c60384c

Related to dotnet/coreclr#5030 Related to dotnet/coreclr#3422

kouvel referenced this issue in kouvel/corefx May 20, 2016

Fix some named mutex tests

b25405a

Related to dotnet/coreclr#5030 Related to dotnet/coreclr#3422

kouvel referenced this issue in kouvel/corefx May 20, 2016

Fix some named mutex tests

e68b52c

Related to dotnet/coreclr#5030 Related to dotnet/coreclr#3422

msftgits transferred this issue from dotnet/coreclr Jan 30, 2020

msftgits added this to the 1.0.0-rtm milestone Jan 30, 2020

dotnet locked as resolved and limited conversation to collaborators Jan 2, 2021

Named mutex not supported on Unix #5211

Named mutex not supported on Unix #5211

Comments

drieseng commented Feb 28, 2016

mikedn commented Feb 28, 2016

drieseng commented Feb 28, 2016

mikedn commented Feb 28, 2016

stephentoub commented Feb 28, 2016

drieseng commented Feb 29, 2016

kouvel commented May 5, 2016

janvorli commented May 5, 2016

krytarowski commented May 6, 2016

stephentoub commented Jun 5, 2016 • edited

ChangePlaces commented Feb 17, 2017

krytarowski commented Feb 17, 2017

ChangePlaces commented Feb 17, 2017 • edited

krytarowski commented Feb 18, 2017

ChangePlaces commented Feb 18, 2017

stephentoub commented Feb 18, 2017

ChangePlaces commented Feb 18, 2017 • edited

kouvel commented Feb 18, 2017

kouvel commented Feb 18, 2017

ChangePlaces commented Feb 18, 2017 • edited

kouvel commented Feb 19, 2017

ChangePlaces commented Feb 21, 2017 • edited

kouvel commented Feb 21, 2017 • edited

nvoronchev commented Mar 14, 2019

stephentoub commented Mar 14, 2019

nvoronchev commented Mar 14, 2019

stephentoub commented Mar 14, 2019

nvoronchev commented Mar 14, 2019

stephentoub commented Mar 14, 2019

ritchxu commented Dec 21, 2019

stephentoub commented Jan 6, 2020

josegomez commented Apr 20, 2020 • edited

stephentoub commented Apr 24, 2020

josegomez commented Apr 25, 2020

kouvel commented Apr 27, 2020

kouvel commented Apr 27, 2020

stephentoub commented Jun 5, 2016 •

edited

ChangePlaces commented Feb 17, 2017 •

edited

ChangePlaces commented Feb 18, 2017 •

edited

ChangePlaces commented Feb 18, 2017 •

edited

ChangePlaces commented Feb 21, 2017 •

edited

kouvel commented Feb 21, 2017 •

edited

josegomez commented Apr 20, 2020 •

edited