Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Change new thread's affinity after thread starts, from the same thread, as a workaround for Snap #40205

Merged
merged 1 commit into from Aug 4, 2020

Conversation

kouvel
Copy link
Member

@kouvel kouvel commented Jul 31, 2020

  • Snap's default strict confinement doesn't allow setting the affinity of a different thread, and currently doesn't allow sched_setaffinity(<nonzeroPid>, ...) even with the calling thread's pid, which pthread implementation uses
  • Switched to use sched_setaffinity(0, ...) where appropriate

Fix for #1634 in master

…d, as a workaround for Snap

- Snap's default strict confinement doesn't allow setting the affinity of a different thread, and currently doesn't allow `sched_setaffinity(<nonzeroPid>, ...)`, which pthread implementation calls
- Switched to use sched_setaffinity(0, ...) where appropriate

Fix for dotnet#1634 in master
@kouvel kouvel added this to the 5.0.0 milestone Jul 31, 2020
@kouvel kouvel requested a review from janvorli July 31, 2020 15:19
@kouvel kouvel self-assigned this Jul 31, 2020
Copy link
Member

@janvorli janvorli left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thank you. I just have one nit.


CPU_ZERO(&cpuSet);

st = sched_getaffinity(gPID, sizeof(cpu_set_t), &cpuSet);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A nit - zero instead of gPID should work here too.

Copy link
Member Author

@kouvel kouvel Jul 31, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Based on the docs I'm not completely sure that they're the same thing. The previous intention was to get the affinity of the process (or perhaps the thread corresponding with gPID) and apply the same affinity to the new thread. The doc for sched_getaffinity only talks about "process" and not "thread", but I wonder if there is ambiguity there since threads have their own pid the the same way as processes.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You are actually right, my mistake. The goal of this code is to set the affinity to be the same as the affinity of the main thread (which is identified by the same PID as the process). If we used zero here, the operation would be effectively noop, as it would just get the affinity of the current thread and then set it to the same value.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was looking at https://linux.die.net/man/2/sched_getaffinity, it does mention a bit about threads:

The affinity mask is actually a per-thread attribute that can be adjusted independently for each of the threads in a thread group. The value returned from a call to gettid(2) can be passed in the argument pid. Specifying pid as 0 will set the attribute for the calling thread, and passing the value returned from a call to getpid(2) will set the attribute for the main thread of the thread group. (If you are using the POSIX threads API, then use pthread_setaffinity_np(3) instead of sched_setaffinity().)

According to https://man7.org/linux/man-pages/man2/sched_getaffinity.2.html, which talks about threads, it's pretty clear from:

If pid is zero, then the mask of the calling thread is returned.

It doesn't sound like it would mean the same thing.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jdstrand, is sched_getaffinity restricted in process-id usage, too? i.e. would the usage above fall foul of the same situation that we have with sched_setaffinity that this PR aims to fix?

Copy link
Member Author

@kouvel kouvel Aug 10, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It does not seem to be restricted, only sched_setaffinity is failing from my tests

Copy link

@jdstrand jdstrand Aug 10, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jdstrand, is sched_getaffinity restricted in process-id usage, too? i.e. would the usage above fall foul of the same situation that we have with sched_setaffinity that this PR aims to fix?

sched_getaffinity is allowed as part of the default template (since it doesn't allow interfering with other snaps and the small info leak isn't worth the developer friction of putting it in another interface).

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Awesome :-) thanks for this!

@kouvel kouvel closed this Aug 3, 2020
@kouvel kouvel reopened this Aug 3, 2020
@kouvel kouvel merged commit 6991ccd into dotnet:master Aug 4, 2020
@kouvel kouvel deleted the SnapFix branch August 4, 2020 04:17
Jacksondr5 pushed a commit to Jacksondr5/runtime that referenced this pull request Aug 10, 2020
…d, as a workaround for Snap (dotnet#40205)

- Snap's default strict confinement doesn't allow setting the affinity of a different thread, and currently doesn't allow `sched_setaffinity(<nonzeroPid>, ...)`, which pthread implementation calls
- Switched to use sched_setaffinity(0, ...) where appropriate

Fix for dotnet#1634 in master
@dotnet dotnet locked as resolved and limited conversation to collaborators Dec 8, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants