[Bug] Reproducing a Hydra run from previous Hydra job configs - WITH SCHEMA

[noamgot]

🐛 Bug

Description

This is a particular case of the issue raised in #1805 - except in this case, my yaml config file has a defined schema.

The problem is that the saved config yaml file does not contain the schema, and when using Omegacong.to_object on the DictConfig object, it converts it into a vanilla dict, instead of the desired dataclass.

Checklist

• [v] I checked on the latest version of Hydra
• [v] I created a minimal repro (See this for tips).

To reproduce

Consider this minimal example:

yaml file - config.yaml

# config.yaml
defaults:
  - config_schema
  - _self_

x: 123
y: 456

code - main.py

from dataclasses import dataclass

import hydra
from hydra.core.config_store import ConfigStore
from omegaconf import DictConfig, OmegaConf


@dataclass
class Foo:
    x: int = 0
    y: int = 1


cs = ConfigStore.instance()
cs.store(name="config_schema", node=Foo)


@hydra.main(config_name="config", config_path=".", version_base=None)
def main(config: DictConfig):
    config_obj: Foo = OmegaConf.to_object(config)
    print(config_obj.x)
    print(config_obj.y)


if __name__ == '__main__':
    main()

Assume that main.py and config.yaml are located in the same directory.

When running:

python main.py 

There's no problem, and it prints:

123
456

But when I try to reproduce this file (naively) with this command:

python main.py --config-path /..../outputs/.../.hydra/ --config-name=config

** Stack trace/error message **

AttributeError: 'dict' object has no attribute 'x'

Expected Behavior

From my point of view, I believe that the config.yaml should be saved with the original schema. If it's rather a feature than a bug - I'd love to hear what's the correct way to reproduce a hydra run in such cases.

System information

• Hydra Version : 1.3.2
• Python version : 3.7
• Virtual environment type and version : conda env
• Operating system : Linux

[Jasha10]

@noamgot this is a know limitation.
Have you seen the Re-run feature?
(Note there is an error on that webpage:
it should be _target_: hydra.experimental.callbacks.PickleJobInfoCallback instead of _target_: hydra.experimental.pickle_job_info_callback.PickleJobInfoCallback)

$ cat config.yaml
# config.yaml
defaults:
  - config_schema
  - _self_

x: 123
y: 456

hydra:
  callbacks:
    save_job_info:
      _target_: hydra.experimental.callbacks.PickleJobInfoCallback

$ python main.py
[2023-07-20 09:32:45,374][hydra.experimental.callbacks.PickleJobInfoCallback][INFO] - Saving job configs in /home/homestar/dev/hydra/outputs/2023-07-20/09-32-45/.hydra/config.pickle
123
456
[2023-07-20 09:32:45,375][hydra.experimental.callbacks.PickleJobInfoCallback][INFO] - Saving job_return in /home/homestar/dev/hydra/outputs/2023-07-20/09-32-45/.hydra/job_return.pickle

$ python main.py --experimental-rerun outputs/2023-07-20/09-33-17/.hydra/config.pickle
/home/homestar/dev/hydra/hydra/main.py:24: UserWarning: Experimental rerun CLI option, other command line args are ignored.
  warnings.warn(msg, UserWarning)
123
456

[noamgot]

@Jasha10 No, I wasn't aware of that. thanks!
You said that it's a known limitation, but is there a plan to fix this in the future? Or do you plan to use the (currently experimental) rerun method you demonstrated?

[omry]

This is not a limitation.
You can use the config object as is without converting it and it would allow you to access cfg.x.

[noamgot]

@omry Maybe I should clarify myself: my goal here is to reproduce an existing run (e.g., for debugging), using the auto-generated config file (within the .hydra directory). But using it directly (as demonstrated above) causes this behavior (as I pointed out, it does not contain the defaults section, and that's why the to_object method returns a dict rather than the desired object). Does it make sense now?