Hardcoded run config does not work with default in_process_executor

[alexknorr]

Dagster version

1.3.4

What's the issue?

When using in_process_executor as default in the Definitions for a code-location, hardcoded job run-configs fail with:

dagster._core.errors.DagsterInvalidConfigError: Error in config for job XXX
Error 1: Received unexpected config entry "multiprocess" at path root:execution:config. Expected: "{ marker_to_close?: String retries?: { disabled?: { } enabled?: { } } }"

It even seems to ignore any execution run-config and always pushes "multiprocess" downstream to the executor which then fails config-validation in the in_process_executor.

What did you expect to happen?

Hardcoded run-config should not assume multiprocess_executor or multi_or_in_process_executor as default and not unintentionally specify "multiprocess" config to the default executor under the hood.

How to reproduce?

I used the Hardcoded configuration sample from the docs and just added a Definitions with in_process_executor. Run it with dagster dev -f sample.py

from dagster import (
    Config,
    Definitions,
    RunConfig,
    in_process_executor,
    job,
    op
)


class DoSomethingConfig(Config):
    config_param: str


@op
def do_something(context, config: DoSomethingConfig):
    context.log.info("config_param: " + config.config_param)


default_config = RunConfig(
    ops={"do_something": DoSomethingConfig(config_param="stuff")}
)


@ job(config=default_config)
def do_it_all_with_default_config():
    do_something()


defs = Definitions(jobs=[do_it_all_with_default_config], executor=in_process_executor)

Deployment type

Local

Deployment details

No response

Additional information

No response

Message from the maintainers

Impacted by this issue? Give it a 👍! We factor engagement into prioritization.

[jamiedemaria]

@schrockn mind taking a look at this? or @benpankow when you're back