Disable alter_sys_path on background threads.

[coderanger]

This prevents issues in threaded launchers because sys.path is a global value so changing it on one thread changes it for all. This leads to race conditions.

I've been running it for a week now and things seem to be much more stable. There isn't a threaded launcher in Dagster itself, so this should have no impact for normal users.

[mgasner]

I wonder if this will interfere with code pointer loading for certain types of code pointers in a potential threaded implementation @prha

[coderanger]

It means you need to be more rigorous with module paths but I think that's a reasonable tradeoff for the overhead benefits of a threaded launcher in my case :)

[coderanger]

I could also make this use an env var instead of checking the thread ID, since this seems to be breaking some of the tests.

[coderanger]

It would appear that pytest runs things in a background thread itself, so that's triggering the new behavior :) Thoughts on making this an env var instead? That's unlike other config but there's not really a good way to get config data down this deep in the system without big changes.

Alternatively I could leave the current thread ID check and add an env var to override it, and set that in the test env.

[prha]

I think having the alter_sys_path behavior disable by a very descriptive environment variable seems like the way to go.

I'm curious about the threaded launcher you have. Are you loading repository code to execute in the run launcher process?

Should also note that we use alter_sys_path to load repositories in the workspace that specify a working_directory argument (https://docs.dagster.io/overview/repositories-workspaces/workspaces), which means that your run launcher would not be able to load certain workspaces.

[sryza]

I'm going to close this, because it hasn't made progress in a few months. @coderanger - feel free to reopen if you'd like to pick this back up.

[coderanger]

Sorry about this, got pulled onto product stuff with the new year :-( Will see how things go with our next big upgrade.