-
Notifications
You must be signed in to change notification settings - Fork 1.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Change dask executor config depending on if using job #5848
Conversation
This pull request is being automatically deployed with Vercel (learn more). 🔍 Inspect: https://vercel.com/elementl/dagster/GLSke3a6kLkB8RgcHeC8mSzR76wj [Deployment for 4d3a1ae canceled] |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
plan context has IPipeline which has the definition, could use that?
also need a test
8efc6be
to
5d4dc7b
Compare
Right, but are we sure we're okay with that? Could the process where this code runs potentially be a place where we aren't okay with loading the definition? |
the host only code paths are separated - especially from this dask executor specific code. I think its fine. |
5d4dc7b
to
b3a8b0f
Compare
b3a8b0f
to
4d3a1ae
Compare
The check that I chose here is maybe not the correct one. If someone has a pipeline with a mode default, and put the dask executor on it, then we might run into a problem.