Named mutex not supported on Unix #3422

Closed
drieseng opened this Issue Feb 28, 2016 · 23 comments

Comments

Projects
None yet
@drieseng

Creating a named mutex on Unix currently throws a PlatformNotSupportedException stating "Named synchronization primitives are not supported on this platform".

Is this scheduled to arrive ? I'm sure @migueldeicaza will be eager to port this from Mono to .NET Core :p

Personally, I don't think it makes sense to expose APIs on both Windows and Unix, but only implement them on one of them. In what way does the Portability Analyzer even take this into account ?

@mikedn

This comment has been minimized.

Show comment
Hide comment
@mikedn

mikedn Feb 28, 2016

Contributor

Related to #1237

Contributor

mikedn commented Feb 28, 2016

Related to #1237

@drieseng drieseng closed this Feb 28, 2016

@drieseng

This comment has been minimized.

Show comment
Hide comment
@drieseng

drieseng Feb 28, 2016

Duplicate of issue #1237.

Duplicate of issue #1237.

@mikedn

This comment has been minimized.

Show comment
Hide comment
@mikedn

mikedn Feb 28, 2016

Contributor

I wouldn't say it's a duplicate. That issue was resolved by removing the functionality. It would be nice to actually fix the actual problem and restore the functionality.

Contributor

mikedn commented Feb 28, 2016

I wouldn't say it's a duplicate. That issue was resolved by removing the functionality. It would be nice to actually fix the actual problem and restore the functionality.

@stephentoub

This comment has been minimized.

Show comment
Hide comment
@stephentoub

stephentoub Feb 28, 2016

Member

I'm sure @migueldeicaza will be eager to port this from Mono to .NET Core :p

Miguel can correct me if I'm wrong, but I don't believe Mono implements this either, or more specifically I believe they implement it the same general manner that the coreclr did before it was disabled to throw those exceptions: the names are only process-wide, not cross-process. That's why we decided to throw exceptions on coreclr; otherwise, since names are typically used for cross-process synchronization, you use the names thinking you're getting cross-process synchronization, but in reality you've just got a bunch of race conditions silently waiting to cause problems.

Member

stephentoub commented Feb 28, 2016

I'm sure @migueldeicaza will be eager to port this from Mono to .NET Core :p

Miguel can correct me if I'm wrong, but I don't believe Mono implements this either, or more specifically I believe they implement it the same general manner that the coreclr did before it was disabled to throw those exceptions: the names are only process-wide, not cross-process. That's why we decided to throw exceptions on coreclr; otherwise, since names are typically used for cross-process synchronization, you use the names thinking you're getting cross-process synchronization, but in reality you've just got a bunch of race conditions silently waiting to cause problems.

@drieseng

This comment has been minimized.

Show comment
Hide comment
@drieseng

drieseng Feb 29, 2016

Reopened because of @mikedn comment.

Reopened because of @mikedn comment.

@drieseng drieseng reopened this Feb 29, 2016

@stephentoub stephentoub added this to the 1.0.0-rtm milestone Apr 3, 2016

@sergiy-k sergiy-k added the Feature label May 5, 2016

@kouvel

This comment has been minimized.

Show comment
Hide comment
@kouvel

kouvel May 5, 2016

Member

My search for ways to add a cross-process mutex didn't yield many options. I'm currently planning on this:

  • On Linux, we have the pthread robust mutex that can be allocated in shared memory, so that's great
  • Elsewhere, it seems like file locks are the only decent option

Currently, we are looking for the following features:

  • Track lifetime across processes and users. One process/user should be able to create the mutex, and another process/user should be able to delete it. This eliminates the System V semaphore since only the creator or owner can delete a semaphore set.
  • Detect when no processes are using the mutex so that it can be deleted or reinitialized. File locks allow this detection.
  • A way to detect an abandoned mutex (similar to pthread robust mutex) even if the process owning the lock crashes or is killed.
  • Be able to do a timed wait. On OSX, this eliminates pthread named semaphore and pshared rwlock. For file locks too, we would need to poll for the lock in loop, sleeping in-between, which has other issues but may be workable if it's the only option. I also looked into Mach semaphores and kqueue, but their lifetime seems to be tied to the creator process.

Any other ideas?

CC @kangaroo

Member

kouvel commented May 5, 2016

My search for ways to add a cross-process mutex didn't yield many options. I'm currently planning on this:

  • On Linux, we have the pthread robust mutex that can be allocated in shared memory, so that's great
  • Elsewhere, it seems like file locks are the only decent option

Currently, we are looking for the following features:

  • Track lifetime across processes and users. One process/user should be able to create the mutex, and another process/user should be able to delete it. This eliminates the System V semaphore since only the creator or owner can delete a semaphore set.
  • Detect when no processes are using the mutex so that it can be deleted or reinitialized. File locks allow this detection.
  • A way to detect an abandoned mutex (similar to pthread robust mutex) even if the process owning the lock crashes or is killed.
  • Be able to do a timed wait. On OSX, this eliminates pthread named semaphore and pshared rwlock. For file locks too, we would need to poll for the lock in loop, sleeping in-between, which has other issues but may be workable if it's the only option. I also looked into Mach semaphores and kqueue, but their lifetime seems to be tied to the creator process.

Any other ideas?

CC @kangaroo

@janvorli

This comment has been minimized.

Show comment
Hide comment
@janvorli

janvorli May 5, 2016

Member

cc @krytarowski for NetBSD

Member

janvorli commented May 5, 2016

cc @krytarowski for NetBSD

@krytarowski

This comment has been minimized.

Show comment
Hide comment
@krytarowski

krytarowski May 6, 2016

Contributor

I don't have anything portable offhand, other than maybe <semaphore.h> or pthread_mutexattr_init(3).

Contributor

krytarowski commented May 6, 2016

I don't have anything portable offhand, other than maybe <semaphore.h> or pthread_mutexattr_init(3).

kouvel added a commit to kouvel/coreclr that referenced this issue May 17, 2016

Add named mutex for cross-process synchronization
Fixes #3422

- On systems that support pthread process-shared robust recursive mutexes, they will be used
- On other systems, file locks are used. File locks unfortunately don't have a timeout in the blocking wait call, and I didn't find any other sync object with a timed wait with the necessary properties, so polling is done for timed waits.

Shared memory files:
- Session-scoped mutexes (name not prefixed, or prefixed with Local\) go in /tmp/coreclr/shm/session<sessionId>/<mutexName>
- Globally-scoped mutexes (name prefixed with Global\) go in /tmp/coreclr/shm/global/<mutexName>
- Contains shared state, and is mmap'ped into the process, see SharedMemorySharedDataHeader and NamedMutexSharedData for data stored
- Creation and deletion is synchronized using an exclusive file lock on the shm directory
- Any process using the shared memory file holds a shared file lock on the shared memory file
- Upon creation, if the shared memory file already exists, an exclusive file lock is attempted on it, to see if the file data is valid. If no other processes have the mutex open, the file is reinitialized.
- Upon releasing the last reference to a mutex in a process, it will try to get an exclusive lock on the shared memory file to see if any other processes have the mutex opened. If not, the file is deleted, along with the session directory if it's empty. The coreclr and shm directories are not deleted.
- This allows managing the lifetime of mutex state based on active processes that have the mutex open. Depending on how the process terminated, the file may still be left over in the tmp directory, I haven't found anything that can be done about that.

Lock files when using file locks:
- In addition to the shared memory file, we need another file for the actual synchronization file lock, since a file lock on the shared memory file is used for lifetime purposes.
- These files go in /tmp/coreclr/lockfiles/session<sessionId>|global/<mutexName>
- The file is empty, and is only used for file locks

Process data
- See SharedMemoryProcessDataHeader and NamedMutexProcessData for data stored
- Per mutex name, there is only one instance of process data that is ref-counted. They are currently stored in a linked list in SharedMemoryManager. It should use a hash table, but of the many hash table implementations that are already there, none seem to be easily usable in the PAL. I'll look into that and will fix later.
- Refers to the associated shared memory, and knows how to clean up both the process data and shared data
- When using file locks for synchronization, a process-local mutex is also created for synchronizing threads, since file locks are owned at the file descriptor level and there is only one open file descriptor in the process per mutex name. The process-local mutex is locked around the file lock, so that only one thread per process is ever trying to flock on a given file descriptor.

Abandon detection
- When a lock is acquired, the process data is added to a linked list on the owning thread
- When a thread exits, the list is walked, each mutex is flagged as abandoned and released
- For detecting process abruptly terminating, pthread robust mutexes give us that. When using file locks, the file lock is automatically released by the system. Upon acquiring a lock, the lock owner info in the shared memory is checked to see if the mutex was abandoned.

Miscellaneous
- CreateMutex and OpenMutex both create new handles for each mutex opened. Each handle just refers to the process data header for the mutex name.
- Some of the above features are already available in the PAL, but not quite in a way that I can use for this purpose. The existing shared memory, naming, and waiting infrastructure is not suitable for this purpose, and is not used.

kouvel added a commit to kouvel/corefx that referenced this issue May 17, 2016

@kouvel kouvel referenced this issue in dotnet/corefx May 17, 2016

Merged

Fix some named mutex tests #8625

kouvel added a commit to kouvel/coreclr that referenced this issue May 18, 2016

Add named mutex for cross-process synchronization
Fixes #3422

- On systems that support pthread process-shared robust recursive mutexes, they will be used
- On other systems, file locks are used. File locks unfortunately don't have a timeout in the blocking wait call, and I didn't find any other sync object with a timed wait with the necessary properties, so polling is done for timed waits.

Shared memory files:
- Session-scoped mutexes (name not prefixed, or prefixed with Local\) go in /tmp/coreclr/shm/session<sessionId>/<mutexName>
- Globally-scoped mutexes (name prefixed with Global\) go in /tmp/coreclr/shm/global/<mutexName>
- Contains shared state, and is mmap'ped into the process, see SharedMemorySharedDataHeader and NamedMutexSharedData for data stored
- Creation and deletion is synchronized using an exclusive file lock on the shm directory
- Any process using the shared memory file holds a shared file lock on the shared memory file
- Upon creation, if the shared memory file already exists, an exclusive file lock is attempted on it, to see if the file data is valid. If no other processes have the mutex open, the file is reinitialized.
- Upon releasing the last reference to a mutex in a process, it will try to get an exclusive lock on the shared memory file to see if any other processes have the mutex opened. If not, the file is deleted, along with the session directory if it's empty. The coreclr and shm directories are not deleted.
- This allows managing the lifetime of mutex state based on active processes that have the mutex open. Depending on how the process terminated, the file may still be left over in the tmp directory, I haven't found anything that can be done about that.

Lock files when using file locks:
- In addition to the shared memory file, we need another file for the actual synchronization file lock, since a file lock on the shared memory file is used for lifetime purposes.
- These files go in /tmp/coreclr/lockfiles/session<sessionId>|global/<mutexName>
- The file is empty, and is only used for file locks

Process data
- See SharedMemoryProcessDataHeader and NamedMutexProcessData for data stored
- Per mutex name, there is only one instance of process data that is ref-counted. They are currently stored in a linked list in SharedMemoryManager. It should use a hash table, but of the many hash table implementations that are already there, none seem to be easily usable in the PAL. I'll look into that and will fix later.
- Refers to the associated shared memory, and knows how to clean up both the process data and shared data
- When using file locks for synchronization, a process-local mutex is also created for synchronizing threads, since file locks are owned at the file descriptor level and there is only one open file descriptor in the process per mutex name. The process-local mutex is locked around the file lock, so that only one thread per process is ever trying to flock on a given file descriptor.

Abandon detection
- When a lock is acquired, the process data is added to a linked list on the owning thread
- When a thread exits, the list is walked, each mutex is flagged as abandoned and released
- For detecting process abruptly terminating, pthread robust mutexes give us that. When using file locks, the file lock is automatically released by the system. Upon acquiring a lock, the lock owner info in the shared memory is checked to see if the mutex was abandoned.

Miscellaneous
- CreateMutex and OpenMutex both create new handles for each mutex opened. Each handle just refers to the process data header for the mutex name.
- Some of the above features are already available in the PAL, but not quite in a way that I can use for this purpose. The existing shared memory, naming, and waiting infrastructure is not suitable for this purpose, and is not used.

kouvel added a commit to kouvel/coreclr that referenced this issue May 18, 2016

Add named mutex for cross-process synchronization
Fixes #3422

- On systems that support pthread process-shared robust recursive mutexes, they will be used
- On other systems, file locks are used. File locks unfortunately don't have a timeout in the blocking wait call, and I didn't find any other sync object with a timed wait with the necessary properties, so polling is done for timed waits.

Shared memory files:
- Session-scoped mutexes (name not prefixed, or prefixed with Local\) go in /tmp/coreclr/shm/session<sessionId>/<mutexName>
- Globally-scoped mutexes (name prefixed with Global\) go in /tmp/coreclr/shm/global/<mutexName>
- Contains shared state, and is mmap'ped into the process, see SharedMemorySharedDataHeader and NamedMutexSharedData for data stored
- Creation and deletion is synchronized using an exclusive file lock on the shm directory
- Any process using the shared memory file holds a shared file lock on the shared memory file
- Upon creation, if the shared memory file already exists, an exclusive file lock is attempted on it, to see if the file data is valid. If no other processes have the mutex open, the file is reinitialized.
- Upon releasing the last reference to a mutex in a process, it will try to get an exclusive lock on the shared memory file to see if any other processes have the mutex opened. If not, the file is deleted, along with the session directory if it's empty. The coreclr and shm directories are not deleted.
- This allows managing the lifetime of mutex state based on active processes that have the mutex open. Depending on how the process terminated, the file may still be left over in the tmp directory, I haven't found anything that can be done about that.

Lock files when using file locks:
- In addition to the shared memory file, we need another file for the actual synchronization file lock, since a file lock on the shared memory file is used for lifetime purposes.
- These files go in /tmp/coreclr/lockfiles/session<sessionId>|global/<mutexName>
- The file is empty, and is only used for file locks

Process data
- See SharedMemoryProcessDataHeader and NamedMutexProcessData for data stored
- Per mutex name, there is only one instance of process data that is ref-counted. They are currently stored in a linked list in SharedMemoryManager. It should use a hash table, but of the many hash table implementations that are already there, none seem to be easily usable in the PAL. I'll look into that and will fix later.
- Refers to the associated shared memory, and knows how to clean up both the process data and shared data
- When using file locks for synchronization, a process-local mutex is also created for synchronizing threads, since file locks are owned at the file descriptor level and there is only one open file descriptor in the process per mutex name. The process-local mutex is locked around the file lock, so that only one thread per process is ever trying to flock on a given file descriptor.

Abandon detection
- When a lock is acquired, the process data is added to a linked list on the owning thread
- When a thread exits, the list is walked, each mutex is flagged as abandoned and released
- For detecting process abruptly terminating, pthread robust mutexes give us that. When using file locks, the file lock is automatically released by the system. Upon acquiring a lock, the lock owner info in the shared memory is checked to see if the mutex was abandoned.

Miscellaneous
- CreateMutex and OpenMutex both create new handles for each mutex opened. Each handle just refers to the process data header for the mutex name.
- Some of the above features are already available in the PAL, but not quite in a way that I can use for this purpose. The existing shared memory, naming, and waiting infrastructure is not suitable for this purpose, and is not used.

kouvel added a commit to kouvel/coreclr that referenced this issue May 18, 2016

Add named mutex for cross-process synchronization
Fixes #3422

- On systems that support pthread process-shared robust recursive mutexes, they will be used
- On other systems, file locks are used. File locks unfortunately don't have a timeout in the blocking wait call, and I didn't find any other sync object with a timed wait with the necessary properties, so polling is done for timed waits.

Shared memory files:
- Session-scoped mutexes (name not prefixed, or prefixed with Local\) go in /tmp/coreclr/shm/session<sessionId>/<mutexName>
- Globally-scoped mutexes (name prefixed with Global\) go in /tmp/coreclr/shm/global/<mutexName>
- Contains shared state, and is mmap'ped into the process, see SharedMemorySharedDataHeader and NamedMutexSharedData for data stored
- Creation and deletion is synchronized using an exclusive file lock on the shm directory
- Any process using the shared memory file holds a shared file lock on the shared memory file
- Upon creation, if the shared memory file already exists, an exclusive file lock is attempted on it, to see if the file data is valid. If no other processes have the mutex open, the file is reinitialized.
- Upon releasing the last reference to a mutex in a process, it will try to get an exclusive lock on the shared memory file to see if any other processes have the mutex opened. If not, the file is deleted, along with the session directory if it's empty. The coreclr and shm directories are not deleted.
- This allows managing the lifetime of mutex state based on active processes that have the mutex open. Depending on how the process terminated, the file may still be left over in the tmp directory, I haven't found anything that can be done about that.

Lock files when using file locks:
- In addition to the shared memory file, we need another file for the actual synchronization file lock, since a file lock on the shared memory file is used for lifetime purposes.
- These files go in /tmp/coreclr/lockfiles/session<sessionId>|global/<mutexName>
- The file is empty, and is only used for file locks

Process data
- See SharedMemoryProcessDataHeader and NamedMutexProcessData for data stored
- Per mutex name, there is only one instance of process data that is ref-counted. They are currently stored in a linked list in SharedMemoryManager. It should use a hash table, but of the many hash table implementations that are already there, none seem to be easily usable in the PAL. I'll look into that and will fix later.
- Refers to the associated shared memory, and knows how to clean up both the process data and shared data
- When using file locks for synchronization, a process-local mutex is also created for synchronizing threads, since file locks are owned at the file descriptor level and there is only one open file descriptor in the process per mutex name. The process-local mutex is locked around the file lock, so that only one thread per process is ever trying to flock on a given file descriptor.

Abandon detection
- When a lock is acquired, the process data is added to a linked list on the owning thread
- When a thread exits, the list is walked, each mutex is flagged as abandoned and released
- For detecting process abruptly terminating, pthread robust mutexes give us that. When using file locks, the file lock is automatically released by the system. Upon acquiring a lock, the lock owner info in the shared memory is checked to see if the mutex was abandoned.

Miscellaneous
- CreateMutex and OpenMutex both create new handles for each mutex opened. Each handle just refers to the process data header for the mutex name.
- Some of the above features are already available in the PAL, but not quite in a way that I can use for this purpose. The existing shared memory, naming, and waiting infrastructure is not suitable for this purpose, and is not used.

kouvel added a commit to kouvel/coreclr that referenced this issue May 18, 2016

Add named mutex for cross-process synchronization
Fixes #3422

- On systems that support pthread process-shared robust recursive mutexes, they will be used
- On other systems, file locks are used. File locks unfortunately don't have a timeout in the blocking wait call, and I didn't find any other sync object with a timed wait with the necessary properties, so polling is done for timed waits.

Shared memory files:
- Session-scoped mutexes (name not prefixed, or prefixed with Local\) go in /tmp/coreclr/shm/session<sessionId>/<mutexName>
- Globally-scoped mutexes (name prefixed with Global\) go in /tmp/coreclr/shm/global/<mutexName>
- Contains shared state, and is mmap'ped into the process, see SharedMemorySharedDataHeader and NamedMutexSharedData for data stored
- Creation and deletion is synchronized using an exclusive file lock on the shm directory
- Any process using the shared memory file holds a shared file lock on the shared memory file
- Upon creation, if the shared memory file already exists, an exclusive file lock is attempted on it, to see if the file data is valid. If no other processes have the mutex open, the file is reinitialized.
- Upon releasing the last reference to a mutex in a process, it will try to get an exclusive lock on the shared memory file to see if any other processes have the mutex opened. If not, the file is deleted, along with the session directory if it's empty. The coreclr and shm directories are not deleted.
- This allows managing the lifetime of mutex state based on active processes that have the mutex open. Depending on how the process terminated, the file may still be left over in the tmp directory, I haven't found anything that can be done about that.

Lock files when using file locks:
- In addition to the shared memory file, we need another file for the actual synchronization file lock, since a file lock on the shared memory file is used for lifetime purposes.
- These files go in /tmp/coreclr/lockfiles/session<sessionId>|global/<mutexName>
- The file is empty, and is only used for file locks

Process data
- See SharedMemoryProcessDataHeader and NamedMutexProcessData for data stored
- Per mutex name, there is only one instance of process data that is ref-counted. They are currently stored in a linked list in SharedMemoryManager. It should use a hash table, but of the many hash table implementations that are already there, none seem to be easily usable in the PAL. I'll look into that and will fix later.
- Refers to the associated shared memory, and knows how to clean up both the process data and shared data
- When using file locks for synchronization, a process-local mutex is also created for synchronizing threads, since file locks are owned at the file descriptor level and there is only one open file descriptor in the process per mutex name. The process-local mutex is locked around the file lock, so that only one thread per process is ever trying to flock on a given file descriptor.

Abandon detection
- When a lock is acquired, the process data is added to a linked list on the owning thread
- When a thread exits, the list is walked, each mutex is flagged as abandoned and released
- For detecting process abruptly terminating, pthread robust mutexes give us that. When using file locks, the file lock is automatically released by the system. Upon acquiring a lock, the lock owner info in the shared memory is checked to see if the mutex was abandoned.

Miscellaneous
- CreateMutex and OpenMutex both create new handles for each mutex opened. Each handle just refers to the process data header for the mutex name.
- Some of the above features are already available in the PAL, but not quite in a way that I can use for this purpose. The existing shared memory, naming, and waiting infrastructure is not suitable for this purpose, and is not used.

kouvel added a commit to kouvel/coreclr that referenced this issue May 18, 2016

Add named mutex for cross-process synchronization
Fixes #3422

- On systems that support pthread process-shared robust recursive mutexes, they will be used
- On other systems, file locks are used. File locks unfortunately don't have a timeout in the blocking wait call, and I didn't find any other sync object with a timed wait with the necessary properties, so polling is done for timed waits.

Shared memory files:
- Session-scoped mutexes (name not prefixed, or prefixed with Local\) go in /tmp/coreclr/shm/session<sessionId>/<mutexName>
- Globally-scoped mutexes (name prefixed with Global\) go in /tmp/coreclr/shm/global/<mutexName>
- Contains shared state, and is mmap'ped into the process, see SharedMemorySharedDataHeader and NamedMutexSharedData for data stored
- Creation and deletion is synchronized using an exclusive file lock on the shm directory
- Any process using the shared memory file holds a shared file lock on the shared memory file
- Upon creation, if the shared memory file already exists, an exclusive file lock is attempted on it, to see if the file data is valid. If no other processes have the mutex open, the file is reinitialized.
- Upon releasing the last reference to a mutex in a process, it will try to get an exclusive lock on the shared memory file to see if any other processes have the mutex opened. If not, the file is deleted, along with the session directory if it's empty. The coreclr and shm directories are not deleted.
- This allows managing the lifetime of mutex state based on active processes that have the mutex open. Depending on how the process terminated, the file may still be left over in the tmp directory, I haven't found anything that can be done about that.

Lock files when using file locks:
- In addition to the shared memory file, we need another file for the actual synchronization file lock, since a file lock on the shared memory file is used for lifetime purposes.
- These files go in /tmp/coreclr/lockfiles/session<sessionId>|global/<mutexName>
- The file is empty, and is only used for file locks

Process data
- See SharedMemoryProcessDataHeader and NamedMutexProcessData for data stored
- Per mutex name, there is only one instance of process data that is ref-counted. They are currently stored in a linked list in SharedMemoryManager. It should use a hash table, but of the many hash table implementations that are already there, none seem to be easily usable in the PAL. I'll look into that and will fix later.
- Refers to the associated shared memory, and knows how to clean up both the process data and shared data
- When using file locks for synchronization, a process-local mutex is also created for synchronizing threads, since file locks are owned at the file descriptor level and there is only one open file descriptor in the process per mutex name. The process-local mutex is locked around the file lock, so that only one thread per process is ever trying to flock on a given file descriptor.

Abandon detection
- When a lock is acquired, the process data is added to a linked list on the owning thread
- When a thread exits, the list is walked, each mutex is flagged as abandoned and released
- For detecting process abruptly terminating, pthread robust mutexes give us that. When using file locks, the file lock is automatically released by the system. Upon acquiring a lock, the lock owner info in the shared memory is checked to see if the mutex was abandoned.

Miscellaneous
- CreateMutex and OpenMutex both create new handles for each mutex opened. Each handle just refers to the process data header for the mutex name.
- Some of the above features are already available in the PAL, but not quite in a way that I can use for this purpose. The existing shared memory, naming, and waiting infrastructure is not suitable for this purpose, and is not used.

kouvel added a commit to kouvel/corefx that referenced this issue May 18, 2016

Remove named mutex PNSE tests that will fail soon
The tests removed would fail soon after dotnet/coreclr#5030 is merged.

Related to dotnet/coreclr#5030
Related to #8625
Related to dotnet/coreclr#3422

kouvel added a commit to kouvel/coreclr that referenced this issue May 19, 2016

Add named mutex for cross-process synchronization
Fixes #3422

- On systems that support pthread process-shared robust recursive mutexes, they will be used
- On other systems, file locks are used. File locks unfortunately don't have a timeout in the blocking wait call, and I didn't find any other sync object with a timed wait with the necessary properties, so polling is done for timed waits.

Shared memory files:
- Session-scoped mutexes (name not prefixed, or prefixed with Local\) go in /tmp/coreclr/shm/session<sessionId>/<mutexName>
- Globally-scoped mutexes (name prefixed with Global\) go in /tmp/coreclr/shm/global/<mutexName>
- Contains shared state, and is mmap'ped into the process, see SharedMemorySharedDataHeader and NamedMutexSharedData for data stored
- Creation and deletion is synchronized using an exclusive file lock on the shm directory
- Any process using the shared memory file holds a shared file lock on the shared memory file
- Upon creation, if the shared memory file already exists, an exclusive file lock is attempted on it, to see if the file data is valid. If no other processes have the mutex open, the file is reinitialized.
- Upon releasing the last reference to a mutex in a process, it will try to get an exclusive lock on the shared memory file to see if any other processes have the mutex opened. If not, the file is deleted, along with the session directory if it's empty. The coreclr and shm directories are not deleted.
- This allows managing the lifetime of mutex state based on active processes that have the mutex open. Depending on how the process terminated, the file may still be left over in the tmp directory, I haven't found anything that can be done about that.

Lock files when using file locks:
- In addition to the shared memory file, we need another file for the actual synchronization file lock, since a file lock on the shared memory file is used for lifetime purposes.
- These files go in /tmp/coreclr/lockfiles/session<sessionId>|global/<mutexName>
- The file is empty, and is only used for file locks

Process data
- See SharedMemoryProcessDataHeader and NamedMutexProcessData for data stored
- Per mutex name, there is only one instance of process data that is ref-counted. They are currently stored in a linked list in SharedMemoryManager. It should use a hash table, but of the many hash table implementations that are already there, none seem to be easily usable in the PAL. I'll look into that and will fix later.
- Refers to the associated shared memory, and knows how to clean up both the process data and shared data
- When using file locks for synchronization, a process-local mutex is also created for synchronizing threads, since file locks are owned at the file descriptor level and there is only one open file descriptor in the process per mutex name. The process-local mutex is locked around the file lock, so that only one thread per process is ever trying to flock on a given file descriptor.

Abandon detection
- When a lock is acquired, the process data is added to a linked list on the owning thread
- When a thread exits, the list is walked, each mutex is flagged as abandoned and released
- For detecting process abruptly terminating, pthread robust mutexes give us that. When using file locks, the file lock is automatically released by the system. Upon acquiring a lock, the lock owner info in the shared memory is checked to see if the mutex was abandoned.

Miscellaneous
- CreateMutex and OpenMutex both create new handles for each mutex opened. Each handle just refers to the process data header for the mutex name.
- Some of the above features are already available in the PAL, but not quite in a way that I can use for this purpose. The existing shared memory, naming, and waiting infrastructure is not suitable for this purpose, and is not used.

kouvel added a commit to kouvel/coreclr that referenced this issue May 19, 2016

Add named mutex for cross-process synchronization
Fixes #3422

- On systems that support pthread process-shared robust recursive mutexes, they will be used
- On other systems, file locks are used. File locks unfortunately don't have a timeout in the blocking wait call, and I didn't find any other sync object with a timed wait with the necessary properties, so polling is done for timed waits.

Shared memory files:
- Session-scoped mutexes (name not prefixed, or prefixed with Local\) go in /tmp/coreclr/shm/session<sessionId>/<mutexName>
- Globally-scoped mutexes (name prefixed with Global\) go in /tmp/coreclr/shm/global/<mutexName>
- Contains shared state, and is mmap'ped into the process, see SharedMemorySharedDataHeader and NamedMutexSharedData for data stored
- Creation and deletion is synchronized using an exclusive file lock on the shm directory
- Any process using the shared memory file holds a shared file lock on the shared memory file
- Upon creation, if the shared memory file already exists, an exclusive file lock is attempted on it, to see if the file data is valid. If no other processes have the mutex open, the file is reinitialized.
- Upon releasing the last reference to a mutex in a process, it will try to get an exclusive lock on the shared memory file to see if any other processes have the mutex opened. If not, the file is deleted, along with the session directory if it's empty. The coreclr and shm directories are not deleted.
- This allows managing the lifetime of mutex state based on active processes that have the mutex open. Depending on how the process terminated, the file may still be left over in the tmp directory, I haven't found anything that can be done about that.

Lock files when using file locks:
- In addition to the shared memory file, we need another file for the actual synchronization file lock, since a file lock on the shared memory file is used for lifetime purposes.
- These files go in /tmp/coreclr/lockfiles/session<sessionId>|global/<mutexName>
- The file is empty, and is only used for file locks

Process data
- See SharedMemoryProcessDataHeader and NamedMutexProcessData for data stored
- Per mutex name, there is only one instance of process data that is ref-counted. They are currently stored in a linked list in SharedMemoryManager. It should use a hash table, but of the many hash table implementations that are already there, none seem to be easily usable in the PAL. I'll look into that and will fix later.
- Refers to the associated shared memory, and knows how to clean up both the process data and shared data
- When using file locks for synchronization, a process-local mutex is also created for synchronizing threads, since file locks are owned at the file descriptor level and there is only one open file descriptor in the process per mutex name. The process-local mutex is locked around the file lock, so that only one thread per process is ever trying to flock on a given file descriptor.

Abandon detection
- When a lock is acquired, the process data is added to a linked list on the owning thread
- When a thread exits, the list is walked, each mutex is flagged as abandoned and released
- For detecting process abruptly terminating, pthread robust mutexes give us that. When using file locks, the file lock is automatically released by the system. Upon acquiring a lock, the lock owner info in the shared memory is checked to see if the mutex was abandoned.

Miscellaneous
- CreateMutex and OpenMutex both create new handles for each mutex opened. Each handle just refers to the process data header for the mutex name.
- Some of the above features are already available in the PAL, but not quite in a way that I can use for this purpose. The existing shared memory, naming, and waiting infrastructure is not suitable for this purpose, and is not used.

kouvel added a commit to kouvel/coreclr that referenced this issue May 19, 2016

Add named mutex for cross-process synchronization
Fixes #3422

- On systems that support pthread process-shared robust recursive mutexes, they will be used
- On other systems, file locks are used. File locks unfortunately don't have a timeout in the blocking wait call, and I didn't find any other sync object with a timed wait with the necessary properties, so polling is done for timed waits.

Shared memory files:
- Session-scoped mutexes (name not prefixed, or prefixed with Local\) go in /tmp/coreclr/shm/session<sessionId>/<mutexName>
- Globally-scoped mutexes (name prefixed with Global\) go in /tmp/coreclr/shm/global/<mutexName>
- Contains shared state, and is mmap'ped into the process, see SharedMemorySharedDataHeader and NamedMutexSharedData for data stored
- Creation and deletion is synchronized using an exclusive file lock on the shm directory
- Any process using the shared memory file holds a shared file lock on the shared memory file
- Upon creation, if the shared memory file already exists, an exclusive file lock is attempted on it, to see if the file data is valid. If no other processes have the mutex open, the file is reinitialized.
- Upon releasing the last reference to a mutex in a process, it will try to get an exclusive lock on the shared memory file to see if any other processes have the mutex opened. If not, the file is deleted, along with the session directory if it's empty. The coreclr and shm directories are not deleted.
- This allows managing the lifetime of mutex state based on active processes that have the mutex open. Depending on how the process terminated, the file may still be left over in the tmp directory, I haven't found anything that can be done about that.

Lock files when using file locks:
- In addition to the shared memory file, we need another file for the actual synchronization file lock, since a file lock on the shared memory file is used for lifetime purposes.
- These files go in /tmp/coreclr/lockfiles/session<sessionId>|global/<mutexName>
- The file is empty, and is only used for file locks

Process data
- See SharedMemoryProcessDataHeader and NamedMutexProcessData for data stored
- Per mutex name, there is only one instance of process data that is ref-counted. They are currently stored in a linked list in SharedMemoryManager. It should use a hash table, but of the many hash table implementations that are already there, none seem to be easily usable in the PAL. I'll look into that and will fix later.
- Refers to the associated shared memory, and knows how to clean up both the process data and shared data
- When using file locks for synchronization, a process-local mutex is also created for synchronizing threads, since file locks are owned at the file descriptor level and there is only one open file descriptor in the process per mutex name. The process-local mutex is locked around the file lock, so that only one thread per process is ever trying to flock on a given file descriptor.

Abandon detection
- When a lock is acquired, the process data is added to a linked list on the owning thread
- When a thread exits, the list is walked, each mutex is flagged as abandoned and released
- For detecting process abruptly terminating, pthread robust mutexes give us that. When using file locks, the file lock is automatically released by the system. Upon acquiring a lock, the lock owner info in the shared memory is checked to see if the mutex was abandoned.

Miscellaneous
- CreateMutex and OpenMutex both create new handles for each mutex opened. Each handle just refers to the process data header for the mutex name.
- Some of the above features are already available in the PAL, but not quite in a way that I can use for this purpose. The existing shared memory, naming, and waiting infrastructure is not suitable for this purpose, and is not used.

kouvel added a commit to kouvel/coreclr that referenced this issue May 19, 2016

Add named mutex for cross-process synchronization
Fixes #3422

- On systems that support pthread process-shared robust recursive mutexes, they will be used
- On other systems, file locks are used. File locks unfortunately don't have a timeout in the blocking wait call, and I didn't find any other sync object with a timed wait with the necessary properties, so polling is done for timed waits.

Shared memory files:
- Session-scoped mutexes (name not prefixed, or prefixed with Local\) go in /tmp/coreclr/shm/session<sessionId>/<mutexName>
- Globally-scoped mutexes (name prefixed with Global\) go in /tmp/coreclr/shm/global/<mutexName>
- Contains shared state, and is mmap'ped into the process, see SharedMemorySharedDataHeader and NamedMutexSharedData for data stored
- Creation and deletion is synchronized using an exclusive file lock on the shm directory
- Any process using the shared memory file holds a shared file lock on the shared memory file
- Upon creation, if the shared memory file already exists, an exclusive file lock is attempted on it, to see if the file data is valid. If no other processes have the mutex open, the file is reinitialized.
- Upon releasing the last reference to a mutex in a process, it will try to get an exclusive lock on the shared memory file to see if any other processes have the mutex opened. If not, the file is deleted, along with the session directory if it's empty. The coreclr and shm directories are not deleted.
- This allows managing the lifetime of mutex state based on active processes that have the mutex open. Depending on how the process terminated, the file may still be left over in the tmp directory, I haven't found anything that can be done about that.

Lock files when using file locks:
- In addition to the shared memory file, we need another file for the actual synchronization file lock, since a file lock on the shared memory file is used for lifetime purposes.
- These files go in /tmp/coreclr/lockfiles/session<sessionId>|global/<mutexName>
- The file is empty, and is only used for file locks

Process data
- See SharedMemoryProcessDataHeader and NamedMutexProcessData for data stored
- Per mutex name, there is only one instance of process data that is ref-counted. They are currently stored in a linked list in SharedMemoryManager. It should use a hash table, but of the many hash table implementations that are already there, none seem to be easily usable in the PAL. I'll look into that and will fix later.
- Refers to the associated shared memory, and knows how to clean up both the process data and shared data
- When using file locks for synchronization, a process-local mutex is also created for synchronizing threads, since file locks are owned at the file descriptor level and there is only one open file descriptor in the process per mutex name. The process-local mutex is locked around the file lock, so that only one thread per process is ever trying to flock on a given file descriptor.

Abandon detection
- When a lock is acquired, the process data is added to a linked list on the owning thread
- When a thread exits, the list is walked, each mutex is flagged as abandoned and released
- For detecting process abruptly terminating, pthread robust mutexes give us that. When using file locks, the file lock is automatically released by the system. Upon acquiring a lock, the lock owner info in the shared memory is checked to see if the mutex was abandoned.

Miscellaneous
- CreateMutex and OpenMutex both create new handles for each mutex opened. Each handle just refers to the process data header for the mutex name.
- Some of the above features are already available in the PAL, but not quite in a way that I can use for this purpose. The existing shared memory, naming, and waiting infrastructure is not suitable for this purpose, and is not used.

kouvel added a commit to kouvel/coreclr that referenced this issue May 19, 2016

Add named mutex for cross-process synchronization
Fixes #3422

- On systems that support pthread process-shared robust recursive mutexes, they will be used
- On other systems, file locks are used. File locks unfortunately don't have a timeout in the blocking wait call, and I didn't find any other sync object with a timed wait with the necessary properties, so polling is done for timed waits.

Shared memory files:
- Session-scoped mutexes (name not prefixed, or prefixed with Local\) go in /tmp/coreclr/shm/session<sessionId>/<mutexName>
- Globally-scoped mutexes (name prefixed with Global\) go in /tmp/coreclr/shm/global/<mutexName>
- Contains shared state, and is mmap'ped into the process, see SharedMemorySharedDataHeader and NamedMutexSharedData for data stored
- Creation and deletion is synchronized using an exclusive file lock on the shm directory
- Any process using the shared memory file holds a shared file lock on the shared memory file
- Upon creation, if the shared memory file already exists, an exclusive file lock is attempted on it, to see if the file data is valid. If no other processes have the mutex open, the file is reinitialized.
- Upon releasing the last reference to a mutex in a process, it will try to get an exclusive lock on the shared memory file to see if any other processes have the mutex opened. If not, the file is deleted, along with the session directory if it's empty. The coreclr and shm directories are not deleted.
- This allows managing the lifetime of mutex state based on active processes that have the mutex open. Depending on how the process terminated, the file may still be left over in the tmp directory, I haven't found anything that can be done about that.

Lock files when using file locks:
- In addition to the shared memory file, we need another file for the actual synchronization file lock, since a file lock on the shared memory file is used for lifetime purposes.
- These files go in /tmp/coreclr/lockfiles/session<sessionId>|global/<mutexName>
- The file is empty, and is only used for file locks

Process data
- See SharedMemoryProcessDataHeader and NamedMutexProcessData for data stored
- Per mutex name, there is only one instance of process data that is ref-counted. They are currently stored in a linked list in SharedMemoryManager. It should use a hash table, but of the many hash table implementations that are already there, none seem to be easily usable in the PAL. I'll look into that and will fix later.
- Refers to the associated shared memory, and knows how to clean up both the process data and shared data
- When using file locks for synchronization, a process-local mutex is also created for synchronizing threads, since file locks are owned at the file descriptor level and there is only one open file descriptor in the process per mutex name. The process-local mutex is locked around the file lock, so that only one thread per process is ever trying to flock on a given file descriptor.

Abandon detection
- When a lock is acquired, the process data is added to a linked list on the owning thread
- When a thread exits, the list is walked, each mutex is flagged as abandoned and released
- For detecting process abruptly terminating, pthread robust mutexes give us that. When using file locks, the file lock is automatically released by the system. Upon acquiring a lock, the lock owner info in the shared memory is checked to see if the mutex was abandoned.

Miscellaneous
- CreateMutex and OpenMutex both create new handles for each mutex opened. Each handle just refers to the process data header for the mutex name.
- Some of the above features are already available in the PAL, but not quite in a way that I can use for this purpose. The existing shared memory, naming, and waiting infrastructure is not suitable for this purpose, and is not used.

kouvel added a commit to kouvel/coreclr that referenced this issue May 19, 2016

Add named mutex for cross-process synchronization
Fixes #3422

- On systems that support pthread process-shared robust recursive mutexes, they will be used
- On other systems, file locks are used. File locks unfortunately don't have a timeout in the blocking wait call, and I didn't find any other sync object with a timed wait with the necessary properties, so polling is done for timed waits.

Shared memory files:
- Session-scoped mutexes (name not prefixed, or prefixed with Local\) go in /tmp/coreclr/shm/session<sessionId>/<mutexName>
- Globally-scoped mutexes (name prefixed with Global\) go in /tmp/coreclr/shm/global/<mutexName>
- Contains shared state, and is mmap'ped into the process, see SharedMemorySharedDataHeader and NamedMutexSharedData for data stored
- Creation and deletion is synchronized using an exclusive file lock on the shm directory
- Any process using the shared memory file holds a shared file lock on the shared memory file
- Upon creation, if the shared memory file already exists, an exclusive file lock is attempted on it, to see if the file data is valid. If no other processes have the mutex open, the file is reinitialized.
- Upon releasing the last reference to a mutex in a process, it will try to get an exclusive lock on the shared memory file to see if any other processes have the mutex opened. If not, the file is deleted, along with the session directory if it's empty. The coreclr and shm directories are not deleted.
- This allows managing the lifetime of mutex state based on active processes that have the mutex open. Depending on how the process terminated, the file may still be left over in the tmp directory, I haven't found anything that can be done about that.

Lock files when using file locks:
- In addition to the shared memory file, we need another file for the actual synchronization file lock, since a file lock on the shared memory file is used for lifetime purposes.
- These files go in /tmp/coreclr/lockfiles/session<sessionId>|global/<mutexName>
- The file is empty, and is only used for file locks

Process data
- See SharedMemoryProcessDataHeader and NamedMutexProcessData for data stored
- Per mutex name, there is only one instance of process data that is ref-counted. They are currently stored in a linked list in SharedMemoryManager. It should use a hash table, but of the many hash table implementations that are already there, none seem to be easily usable in the PAL. I'll look into that and will fix later.
- Refers to the associated shared memory, and knows how to clean up both the process data and shared data
- When using file locks for synchronization, a process-local mutex is also created for synchronizing threads, since file locks are owned at the file descriptor level and there is only one open file descriptor in the process per mutex name. The process-local mutex is locked around the file lock, so that only one thread per process is ever trying to flock on a given file descriptor.

Abandon detection
- When a lock is acquired, the process data is added to a linked list on the owning thread
- When a thread exits, the list is walked, each mutex is flagged as abandoned and released
- For detecting process abruptly terminating, pthread robust mutexes give us that. When using file locks, the file lock is automatically released by the system. Upon acquiring a lock, the lock owner info in the shared memory is checked to see if the mutex was abandoned.

Miscellaneous
- CreateMutex and OpenMutex both create new handles for each mutex opened. Each handle just refers to the process data header for the mutex name.
- Some of the above features are already available in the PAL, but not quite in a way that I can use for this purpose. The existing shared memory, naming, and waiting infrastructure is not suitable for this purpose, and is not used.

@kouvel kouvel closed this in #5030 May 20, 2016

kouvel added a commit to kouvel/corefx that referenced this issue May 20, 2016

kouvel added a commit to kouvel/corefx that referenced this issue May 20, 2016

kouvel added a commit to kouvel/corefx that referenced this issue May 20, 2016

kouvel added a commit to kouvel/corefx that referenced this issue May 20, 2016

kouvel added a commit to kouvel/corefx that referenced this issue May 20, 2016

kouvel added a commit to kouvel/corefx that referenced this issue May 20, 2016

@ghost

This comment has been minimized.

Show comment
Hide comment
@ghost

ghost Jun 5, 2016

Mono's create named semaphore implementation is situated at: https://github.com/mono/mono/blob/ef40790/mono/io-layer/semaphores.c#L193-L289.

ghost commented Jun 5, 2016

Mono's create named semaphore implementation is situated at: https://github.com/mono/mono/blob/ef40790/mono/io-layer/semaphores.c#L193-L289.

@stephentoub

This comment has been minimized.

Show comment
Hide comment
@stephentoub

stephentoub Jun 5, 2016

Member

Mono's create named semaphore

Mono's implementation is not cross-process.

Member

stephentoub commented Jun 5, 2016

Mono's create named semaphore

Mono's implementation is not cross-process.

@ChangePlaces

This comment has been minimized.

Show comment
Hide comment
@ChangePlaces

ChangePlaces Feb 17, 2017

I'm confused, what's the current state of named Mutexes in core? Will they work on windows and linux?

I'm confused, what's the current state of named Mutexes in core? Will they work on windows and linux?

@krytarowski

This comment has been minimized.

Show comment
Hide comment
@krytarowski

krytarowski Feb 17, 2017

Contributor

There was an implementation with POSIX robust mutexes on Linux.

Contributor

krytarowski commented Feb 17, 2017

There was an implementation with POSIX robust mutexes on Linux.

@ChangePlaces

This comment has been minimized.

Show comment
Hide comment
@ChangePlaces

ChangePlaces Feb 17, 2017

so that's a yes? It'd be really nice if yes as I can get rid of the file locking hacks!

ChangePlaces commented Feb 17, 2017

so that's a yes? It'd be really nice if yes as I can get rid of the file locking hacks!

@krytarowski

This comment has been minimized.

Show comment
Hide comment
@krytarowski

krytarowski Feb 18, 2017

Contributor

I guess so, this is why I need to implement POSIX robust mutexes on my platform (NetBSD).

Contributor

krytarowski commented Feb 18, 2017

I guess so, this is why I need to implement POSIX robust mutexes on my platform (NetBSD).

@ChangePlaces

This comment has been minimized.

Show comment
Hide comment
@ChangePlaces

ChangePlaces Feb 18, 2017

I've just tried on Ubuntu 14.04 and the answer is a no

I've just tried on Ubuntu 14.04 and the answer is a no

@stephentoub

This comment has been minimized.

Show comment
Hide comment
@stephentoub

stephentoub Feb 18, 2017

Member

I've just tried on Ubuntu 14.04 and the answer is a no

What code did you try? Named mutexes should work fine on in Ubuntu.

Member

stephentoub commented Feb 18, 2017

I've just tried on Ubuntu 14.04 and the answer is a no

What code did you try? Named mutexes should work fine on in Ubuntu.

@ChangePlaces

This comment has been minimized.

Show comment
Hide comment
@ChangePlaces

ChangePlaces Feb 18, 2017

Interesting, this is my code, other processes are not honouring the mutex and are acquiring the mutex and repeating the same work. It works in Windows as expected.

using (var m = new Mutex(false, mutexName))
{
    var hasMutex = false;
    try
    {
	if (m.WaitOne(2))
	{
	    hasMutex = true;
	    ProcessIterationAsync(currentOutputFileNameBase, c, _config.CountryCode).Wait();
	}
    }
    catch (AbandonedMutexException e)
    {
	continue;
    }
    catch (Exception e)
    {
        continue;
    }
    finally
    {
	if (hasMutex)
	{
	    m.ReleaseMutex();
	}
    }
}

ChangePlaces commented Feb 18, 2017

Interesting, this is my code, other processes are not honouring the mutex and are acquiring the mutex and repeating the same work. It works in Windows as expected.

using (var m = new Mutex(false, mutexName))
{
    var hasMutex = false;
    try
    {
	if (m.WaitOne(2))
	{
	    hasMutex = true;
	    ProcessIterationAsync(currentOutputFileNameBase, c, _config.CountryCode).Wait();
	}
    }
    catch (AbandonedMutexException e)
    {
	continue;
    }
    catch (Exception e)
    {
        continue;
    }
    finally
    {
	if (hasMutex)
	{
	    m.ReleaseMutex();
	}
    }
}
@kouvel

This comment has been minimized.

Show comment
Hide comment
@kouvel

kouvel Feb 18, 2017

Member

What name did you use? By default, names have session scope and sessions are more granular on Unix (each terminal gets its own session). If you haven't already, please try adding a "Global" prefix to the name minus the quotes.

There is build-time code to detect if the system supports POSIX robust mutexes, see https://github.com/dotnet/coreclr/blob/master/src/pal/src/configure.cmake#L997 and https://github.com/dotnet/coreclr/blob/master/src/pal/src/configure.cmake#L1020. Based on that, it would use POSIX robust mutexes or file locks, but it should work either way.

Member

kouvel commented Feb 18, 2017

What name did you use? By default, names have session scope and sessions are more granular on Unix (each terminal gets its own session). If you haven't already, please try adding a "Global" prefix to the name minus the quotes.

There is build-time code to detect if the system supports POSIX robust mutexes, see https://github.com/dotnet/coreclr/blob/master/src/pal/src/configure.cmake#L997 and https://github.com/dotnet/coreclr/blob/master/src/pal/src/configure.cmake#L1020. Based on that, it would use POSIX robust mutexes or file locks, but it should work either way.

@kouvel

This comment has been minimized.

Show comment
Hide comment
@kouvel

kouvel Feb 18, 2017

Member

Although unrelated, in the above code, when an AbandonedMutexException is thrown, the thread is still given ownership of the lock, so you'd have to ReleaseMutex on that path as well

Member

kouvel commented Feb 18, 2017

Although unrelated, in the above code, when an AbandonedMutexException is thrown, the thread is still given ownership of the lock, so you'd have to ReleaseMutex on that path as well

@ChangePlaces

This comment has been minimized.

Show comment
Hide comment
@ChangePlaces

ChangePlaces Feb 18, 2017

I prefixed the name with "Global\"as suggested here to no avail. I am running the processes in multiple screen instances. This is with RC4 (installed with apt-get)

ChangePlaces commented Feb 18, 2017

I prefixed the name with "Global\"as suggested here to no avail. I am running the processes in multiple screen instances. This is with RC4 (installed with apt-get)

@kouvel

This comment has been minimized.

Show comment
Hide comment
@kouvel

kouvel Feb 19, 2017

Member

Would it be possible to provide a small runnable repro? I can give it a try and see what's going on.

Member

kouvel commented Feb 19, 2017

Would it be possible to provide a small runnable repro? I can give it a try and see what's going on.

@ChangePlaces

This comment has been minimized.

Show comment
Hide comment
@ChangePlaces

ChangePlaces Feb 21, 2017

I took this down to the most basic example I could make and it worked... so I worked backwards and fixed. It was the missing "Global\" along with a few other things - User error - thanks and sorry!

ChangePlaces commented Feb 21, 2017

I took this down to the most basic example I could make and it worked... so I worked backwards and fixed. It was the missing "Global\" along with a few other things - User error - thanks and sorry!

@kouvel

This comment has been minimized.

Show comment
Hide comment
@kouvel

kouvel Feb 21, 2017

Member

Ok great

Member

kouvel commented Feb 21, 2017

Ok great

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment