[Feature Request] Mixing _global_ / _here_ / _package_

[YannDubs]

🚀 Feature Request

Allow the use of packages inise of the yaml. E.g.

# this is local
is_local: True

_global_: # Special keyword that says that it's global
  outside:
    is_global: True

or equivalently

# @package _global_

# this is local
_here_: # Special keyword that says that it's local
  is_local: True

# this is global because it's the default package
outside:
  is_global: True

Motivation

I often reuse a lot of my configs for different parts of the programs to avoid duplicating code, so I often use something like X@Y: Z but currently I don't really know how to deal with cases where Z has to make global and local changes. Because if I use # @package _global_ then I can't change locally Y but if I use the default package I can't change global configs.

I think it would be nice to have a way of mixing _here_ and _global_ in a single config. Another way of achieving this would be to be able to use multiple packages in one yaml e.g.,

# @package _here_
is_local: True

# @package _global_

outside:
  is_global: True

Although that seems less natural.

My current workarounds to split the yaml into the global and the local ones, but that's less readable and you end up with a lot of different configs.

[Jasha10]

Hi @YannDubs,

Thanks for sharing this idea.

Introducing a new keyword to Hydra's config composition would be a big step, so we'd have to consider carefully whether it's the right direction for Hydra long-term. I'm also concerned that the implementation would be challenging.

I'll put this on the wishlist for now.

[dhruvdcoder]

I often end up with lots of small configs and nested folders. This feature would be very helpful in improving the readability and maintainability.

[max-sixty]

Old issue, but FWIW, I would have found this much easier to understand by using an actual value in the YAML for the equivalent of # @package foo.bar.

Giving YAmL comments semantic meaning isn't what I would have expected and isn't consistent with the rest of Hydra — there's plenty of prior art for "special" keys in Hydra, such as _self_ etc. So the original suggestion of something like:

_global_: # Special keyword that says that it's global
  outside:

...mades a lot of sense to me.