Sharedfp sm component doesn't work on MacOS

[edgargabriel]

@hjelmn pointed out a couple of days back that the sharedfp sm component raises on error on MacOS. After finally getting my hand on a MAC, the problem has been identified. Semaphore names on MacOS are restricted to 31 bytes, and we are exceeding that. Solution is mostly coded up.

[edgargabriel]

@hjelmn and @jsquyres a question to the two of you as more experienced MAC programmers. So the root problem of the bug that Nathan saw on his mac is that MacOS restricts the name lengths of a named semaphore to 31 characters, while linux (at least on the system that I tested) allows up to 252 characters. I could not come with a scheme that would allow multiple semaphores simultaneously to co-exist that does not use the filename (because of scenarios such as subgroups of processes operate on separate files, which would requires different semaphores to manage their shared file pointers - this basically eliminates simple counter based approaches).

So the options that I see right now are:

1. I just shorten the semaphore name, e.g. instead of "OMPIO_sharedfp_sem_{filename}" to OMPIO_{filename}. This makes the testsuite pass, but it depends of course on the filename lengths and might fail in real life.
2. I try to a bit more 'correct' and shorten the semaphore name to 31 characters in MAC and 253 characters in linux (i.e. snprintf instead of sprintf). This would requires an architecture dependent ifdef.
3. WE could disable the sm sharedfp component on Mac, or at least decrease its priority. The testsuite passes with the lockedfile component as well.

What's your opinion on that? Did I overlook another solution?

[hppritcha]

option 2 sounds better to me.

---

sent from my smart phonr so no good type.

Howard
On Oct 6, 2015 1:03 PM, "Edgar Gabriel" notifications@github.com wrote:

@hjelmn https://github.com/hjelmn and @jsquyres
https://github.com/jsquyres a question to the two of you as more
experienced MAC programmers. So the root problem of the bug that Nathan saw
on his mac is that MacOS restricts the name lengths of a named semaphore to
31 characters, while linux (at least on the system that I tested) allows up
to 252 characters. I could not come with a scheme that would allow multiple
semaphores simultaneously to co-exist that does not use the filename
(because of scenarios such as subgroups of processes operate on separate
files, which would requires different semaphores to manage their shared
file pointers - this basically eliminates simple counter based approaches).

So the options that I see right now are:

1. I just shorten the semaphore name, e.g. instead of
   "OMPIO_sharedfp_sem_" to OMPIO_. This makes the testsuite pass, but it
   depends of course on the filename lengths and might fail in real life.

   I try to a bit more 'correct' and shorten the semaphore name to 31
   characters in MAC and 253 characters in linux (i.e. snprintf instead of
   sprintf). This would requires an architecture dependent ifdef.
   2.

   WE could disable the sm sharedfp component on Mac, or at least
   decrease its priority. The testsuite passes with the lockedfile component
   as well.

What's your opinion on that? Did I overlook another solution?

—
Reply to this email directly or view it on GitHub
#980 (comment).

[jsquyres]

What is the format of the filename that is causing the name to be >31 characters? Can you use some other randomizing / safe prefix? I ask because it seems like 31 characters is a lot of space to play with to get enough randomization to guarantee uniqueness. Additionally, can the file go in the session directory somewhere? There's subdirectories in there that may also provide additional guarantees of uniqueness...?

[edgargabriel]

So let me describe a scenario. One node, 32 processses, 16 processes work on file1 the other 16 processes work on file2. What we need to establish is a named semaphore that all that uniquely allows each of the 16 processes work on file1 to access the same mmaped file, and the other 16 the other mmaped file. Right now, the scheme is that the name of the semaphore is 'OMPIO_sharedfp_sem_file1' and 'OMPIO_sharedfp_sem_file2'. I did not find a solution that would allow to not use the filename to uniquely identify the file that they are working on. A filename alone can however be larger than 31 bytes by POSIX rules.

I tried a counter approach, e.g. OMPIO_sem_1, OMPIO_sem_2 etc. but the problem with that is that all processes can access all named semaphores on the same shared memory node, its not like the second 16 processes will get an error message when they try to use OMPIO_sem_1 in order for them to determine that they have to increase the counter value.

And yes, the mmaped file can go into session directory, but that would not solve the problem in itself.

[edgargabriel]

To be clear for my previous comment, file1 and file2 are assumed to be the filenames passed by the application to MPI_File_open()

[edgargabriel]

I will implement solution 2 and we can discuss a revised/better solution later if we want to revisit it.

[edgargabriel]

fixed by commit 2c7265d