[Bug/Question] `config_path` resolution is misleading #2274

stoney95 · 2022-06-29T16:34:32Z

🐛 Bug

Description

The config_path behaves differntly. It depends on if hydra.main is used in the executed script or it is part of a custom decorator located in another module. The issue is rather complex to describe - please take a look at the "To reproduce" section. I will describe the issue there in detail.

I actually want that the DictConfig provided by hydra.main is directly resolved to a dataclass that represents my config. I use hydra.utils.instantiate to achieve this. As I have multiple entry points ("main scripts") in my project that share the same config I want to move this step into a custom decorator that I can reuse in all entry points.

I also checked to find an other way to achieve the direct resolution, but could not find any. If there is a better solution to my problem, I will gratefully accept it as a solution.

Checklist

I checked on the latest version of Hydra
I created a minimal repro (See this for tips).

To reproduce

This issue has multiple cases. I created a small repo that displays each of these cases. I will also list the cases in the following and provide the corresponding information

1. Use `@hydra.main` directly in the script

This works as expected.

Script: here
Error message: -

Note: I use a pathlib.Path object representing the absolute path to my conf folder as config_path.

2. Move `hydra.utils.instantiate` into decorator

In the next step I moved everything into a decorator which I can use instead of @hydra.main. This decorator is placed in the script itself. With this setup it works.

The decorator looks like this:

def ingest_config(func):
    @functools.wraps(func)
    @hydra.main(config_path=ROOT_DIR / "conf", config_name="config", version_base=None)
    def inner(dict_config):
        config = hydra.utils.instantiate(dict_config)
        func(config)

    return inner

Script: here
Decorator: ingest_config
Error: -

Note: It is important that the decorator is placed in the script itself

3. Use custom decorator that's imported from a module

The next step is to move the decorator into a module as I would like to use it in multiple scripts. The module in this case is common.config. Then doing this the resolution of the config_path is not working anymore.

Script: here
Decorator: ingest_config_not_working_with_pathlib
Error:

Traceback (most recent call last):
  File "src/app_not_working_pathlib.py", line 12, in <module>
    main()
  File "/Users/simons/anaconda3/envs/ai_attack_mlops/lib/python3.8/site-packages/hydra/main.py", line 90, in decorated_main
    _run_hydra(
  File "/Users/simons/anaconda3/envs/ai_attack_mlops/lib/python3.8/site-packages/hydra/_internal/utils.py", line 332, in _run_hydra
    search_path = create_automatic_config_search_path(
  File "/Users/simons/anaconda3/envs/ai_attack_mlops/lib/python3.8/site-packages/hydra/_internal/utils.py", line 182, in create_automatic_config_search_path
    search_path_dir = compute_search_path_dir(calling_file, calling_module, config_path)
  File "/Users/simons/anaconda3/envs/ai_attack_mlops/lib/python3.8/site-packages/hydra/_internal/utils.py", line 139, in compute_search_path_dir
    config_path = config_path.replace(os.path.sep, "/")
TypeError: replace() takes 2 positional arguments but 3 were given

Note: To me it looks like hydra has problems handling the pathlib.Path. In the next section I cast the pathlib.Path to a str

4. Use custom decorator with `str` object representing the absolute path to conf folder as `config_path`

As the previous error looks like hydra is struggling with pathlib.Path as config_path I cast it to str. The path itself stays the same. This is not working. The error message changes.

Script: here
Decorator: ingest_config_not_working_absolute_path
Error:

Traceback (most recent call last):
  File "app_not_working_absolute_path.py", line 12, in <module>
    main()
  File "/Users/simons/anaconda3/envs/ai_attack_mlops/lib/python3.8/site-packages/hydra/main.py", line 90, in decorated_main
    _run_hydra(
  File "/Users/simons/anaconda3/envs/ai_attack_mlops/lib/python3.8/site-packages/hydra/_internal/utils.py", line 389, in _run_hydra
    _run_app(
  File "/Users/simons/anaconda3/envs/ai_attack_mlops/lib/python3.8/site-packages/hydra/_internal/utils.py", line 452, in _run_app
    run_and_report(
  File "/Users/simons/anaconda3/envs/ai_attack_mlops/lib/python3.8/site-packages/hydra/_internal/utils.py", line 216, in run_and_report
    raise ex
  File "/Users/simons/anaconda3/envs/ai_attack_mlops/lib/python3.8/site-packages/hydra/_internal/utils.py", line 213, in run_and_report
    return func()
  File "/Users/simons/anaconda3/envs/ai_attack_mlops/lib/python3.8/site-packages/hydra/_internal/utils.py", line 453, in <lambda>
    lambda: hydra.run(
  File "/Users/simons/anaconda3/envs/ai_attack_mlops/lib/python3.8/site-packages/hydra/_internal/hydra.py", line 105, in run
    cfg = self.compose_config(
  File "/Users/simons/anaconda3/envs/ai_attack_mlops/lib/python3.8/site-packages/hydra/_internal/hydra.py", line 594, in compose_config
    cfg = self.config_loader.load_configuration(
  File "/Users/simons/anaconda3/envs/ai_attack_mlops/lib/python3.8/site-packages/hydra/_internal/config_loader_impl.py", line 141, in load_configuration
    return self._load_configuration_impl(
  File "/Users/simons/anaconda3/envs/ai_attack_mlops/lib/python3.8/site-packages/hydra/_internal/config_loader_impl.py", line 229, in _load_configuration_impl
    self.ensure_main_config_source_available()
  File "/Users/simons/anaconda3/envs/ai_attack_mlops/lib/python3.8/site-packages/hydra/_internal/config_loader_impl.py", line 128, in ensure_main_config_source_available
    self._missing_config_error(
  File "/Users/simons/anaconda3/envs/ai_attack_mlops/lib/python3.8/site-packages/hydra/_internal/config_loader_impl.py", line 102, in _missing_config_error
    raise MissingConfigException(
hydra.errors.MissingConfigException: Primary config module 'common..Users.simons.dev.hydra-reproduce-error.conf' not found.
Check that it's correct and contains an __init__.py file

Note: Hydra struggles to resolve the absolute path. The path hydra created from the absolute path is common..Users.simons.dev.hydra-reproduce-error.conf. The decorator where @hydra.main is used is located in src/common/config.py. Users.simons.dev.hydra-reproduce-error.conf is the location where the configuration is actually located.

5. Custom decorator with relative path pointing to the conf folder

The next step was to replace the absolute path with a relative path to the conf folder. This failed as well

Script: here
Decorator: ingest_config_not_working_relative_path
Error:

Traceback (most recent call last):
  File "src/app_not_working_relative_path.py", line 12, in <module>
    main()
  File "/Users/simons/anaconda3/envs/ai_attack_mlops/lib/python3.8/site-packages/hydra/main.py", line 90, in decorated_main
    _run_hydra(
  File "/Users/simons/anaconda3/envs/ai_attack_mlops/lib/python3.8/site-packages/hydra/_internal/utils.py", line 389, in _run_hydra
    _run_app(
  File "/Users/simons/anaconda3/envs/ai_attack_mlops/lib/python3.8/site-packages/hydra/_internal/utils.py", line 452, in _run_app
    run_and_report(
  File "/Users/simons/anaconda3/envs/ai_attack_mlops/lib/python3.8/site-packages/hydra/_internal/utils.py", line 216, in run_and_report
    raise ex
  File "/Users/simons/anaconda3/envs/ai_attack_mlops/lib/python3.8/site-packages/hydra/_internal/utils.py", line 213, in run_and_report
    return func()
  File "/Users/simons/anaconda3/envs/ai_attack_mlops/lib/python3.8/site-packages/hydra/_internal/utils.py", line 453, in <lambda>
    lambda: hydra.run(
  File "/Users/simons/anaconda3/envs/ai_attack_mlops/lib/python3.8/site-packages/hydra/_internal/hydra.py", line 105, in run
    cfg = self.compose_config(
  File "/Users/simons/anaconda3/envs/ai_attack_mlops/lib/python3.8/site-packages/hydra/_internal/hydra.py", line 594, in compose_config
    cfg = self.config_loader.load_configuration(
  File "/Users/simons/anaconda3/envs/ai_attack_mlops/lib/python3.8/site-packages/hydra/_internal/config_loader_impl.py", line 141, in load_configuration
    return self._load_configuration_impl(
  File "/Users/simons/anaconda3/envs/ai_attack_mlops/lib/python3.8/site-packages/hydra/_internal/config_loader_impl.py", line 229, in _load_configuration_impl
    self.ensure_main_config_source_available()
  File "/Users/simons/anaconda3/envs/ai_attack_mlops/lib/python3.8/site-packages/hydra/_internal/config_loader_impl.py", line 128, in ensure_main_config_source_available
    self._missing_config_error(
  File "/Users/simons/anaconda3/envs/ai_attack_mlops/lib/python3.8/site-packages/hydra/_internal/config_loader_impl.py", line 102, in _missing_config_error
    raise MissingConfigException(
hydra.errors.MissingConfigException: Primary config module 'conf' not found.
Check that it's correct and contains an __init__.py file

Note: Hydra still struggles to find the conf folder. Eventhough the path is now resolved differently compared to what happend when using the absolute path

6. Custom decorator with relative path pointing to the conf folder inside the src directory

As a last step I copied the conf folder into src. I also renamed it to conf_inside_src to be sure that this folder is referenced. I used a custom decorator that points to this folder. This made it work!

Script: here
Decorator: ingest_config_working_relative_path
Error: -

Note: I don't understand why hydra works differently depending on the location of the conf folder. I also would like to keep conf outside the src folder.

Expected Behavior

I expect that the usage of an absolute path works. I would also expect that case 5 (Using a relative path to "conf" folder which is not in "src") works as well.

System information

Hydra Version : 1.2.0
Python version : 3.8.13
Virtual environment type and version : conda 4.5.12
Operating system : macOS Monterey 12.1

Additional context

As stated in the beginning if there is an other way to achieve resolution of yaml files to dataclass objects, I would be happy to use it.

The text was updated successfully, but these errors were encountered:

Jasha10 · 2022-07-02T00:06:57Z

Thanks very much for taking the time to organize these examples.
I'll take a look...

stoney95 · 2022-07-02T10:01:25Z

@Jasha10 , thanks for taking a look. I prefixed the script names with the number of the cases from the issue. I hope this improves the "readability" of the examples.

stoney95 · 2022-08-22T11:19:19Z

@Jasha10 any updates?

Jasha10 · 2022-09-07T18:05:05Z

Apologies for the slow reply @stoney95. I'll get back to you on this asap.

omry · 2022-09-10T13:36:24Z

Why are you not using the compose API to create your config?
hydra.main is definitely not designed to be used like this.

stoney95 · 2022-10-08T13:21:54Z

Is the compose API also providing the possibility to use overrides as runtime arguments? As far as I understand the compose API I would need to parse the overrides manually and provide them to compose.

Is there a reason why main should not be used in the way described above?

stoney95 added the bug Something isn't working label Jun 29, 2022

Jasha10 added this to the Hydra 1.3.0 milestone Nov 10, 2022

Jasha10 mentioned this issue Nov 10, 2022

[Feature Request] Option to specify the arg --config-path, -cp as an absolute path #2368

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Bug/Question] `config_path` resolution is misleading #2274

[Bug/Question] `config_path` resolution is misleading #2274

stoney95 commented Jun 29, 2022 •

edited

Jasha10 commented Jul 2, 2022

stoney95 commented Jul 2, 2022

stoney95 commented Aug 22, 2022

Jasha10 commented Sep 7, 2022

omry commented Sep 10, 2022

stoney95 commented Oct 8, 2022

[Bug/Question] config_path resolution is misleading #2274

[Bug/Question] config_path resolution is misleading #2274

Comments

stoney95 commented Jun 29, 2022 • edited

🐛 Bug

Description

Checklist

To reproduce

1. Use @hydra.main directly in the script

2. Move hydra.utils.instantiate into decorator

3. Use custom decorator that's imported from a module

4. Use custom decorator with str object representing the absolute path to conf folder as config_path

5. Custom decorator with relative path pointing to the conf folder

6. Custom decorator with relative path pointing to the conf folder inside the src directory

Expected Behavior

System information

Additional context

Jasha10 commented Jul 2, 2022

stoney95 commented Jul 2, 2022

stoney95 commented Aug 22, 2022

Jasha10 commented Sep 7, 2022

omry commented Sep 10, 2022

stoney95 commented Oct 8, 2022

[Bug/Question] `config_path` resolution is misleading #2274

[Bug/Question] `config_path` resolution is misleading #2274

stoney95 commented Jun 29, 2022 •

edited

1. Use `@hydra.main` directly in the script

2. Move `hydra.utils.instantiate` into decorator

4. Use custom decorator with `str` object representing the absolute path to conf folder as `config_path`