[3.1] Change new thread's affinity after thread starts, from the same thread, as a workaround for Snap

[kouvel]

• Port of Change new thread's affinity after thread starts, from the same thread, as a workaround for Snap runtime#40205 to 3.1
• Snap's default strict confinement doesn't allow setting the affinity of a different thread, and currently doesn't allow sched_setaffinity(<nonzeroPid>, ...) even with the calling thread's pid, which pthread implementation uses
• Switched to use sched_setaffinity(0, ...) where appropriate

Fixes dotnet/runtime#1634

Customer impact

• .NET Core doesn't work under Snap's default strict confinement. When a thread is created coreclr requests for the processor affinity of the new thread to be changed to the process' affinity, which is not allowed under Snap's strict confinement.
• There are a couple of workarounds, but both involve manual/non-default steps (inconvenient) and give the process more control than desired:
  • Specify the process-control plug and manually connect it before running the app
  • Require the classic confinement instead and specify it when installing the app
• The workarounds may also cause some difficulty, especially longer-term, for .NET Core apps to be added to the public snap store due to the extra control the process would require.
  • It is possible to specify to auto-connect the process-control plug using store processes, but it requires review and justification, and the purpose of working around this issue may be difficult to justify longer-term
  • Apps that use the classic confinement run without any sandboxing, and may not be allowed in the public snap store for the purpose of working around this issue
  • See dotnet core 3.1 self-contained snap runtime#1634 (comment) for more information

Regression?

No

Testing

• Tested a small app under Snap's strict confinement
• PAL, coreclr, and libraries tests

Risk

Low:

• Instead of the creating thread changing the affinity of the created thread, the created thread changes its own affinity after starting, the difference is mostly invisible
• Where appropriate, used a fallback to use the pthread implementation for setting the affinity when sched_setaffinity is not available, so in the theoretical possibility of a different pthread implementation for setting the affinity, there would be no change from before