[Bug] Paths not properly configured when packaged

[jbohnslav]

🐛 Bug

I'm releasing my app as a pip package. I want users to be able to type "my_app" at the command line. I want this main script to take a Hydra configuration as input.

What works:
python -m my_app.gui.main
What doesn't work:
my_app

In my_app\gui\main:

@hydra.main(config_path='../conf/gui.yaml')
def run(cfg: DictConfig) -> None:
    log.info('CWD: {}'.format(os.getcwd()))

In my setup.py:

entry_points={
        'console_scripts':[
            'my_app= my_app.gui.main:run']
    },

** Stack trace/error message **

Traceback (most recent call last):
  File "C:\ProgramData\Anaconda3\envs\deg\Scripts\my_app-script.py", line 11, in <module>
    load_entry_point('my_app==0.0.1', 'console_scripts', 'my_app')()
  File "C:\ProgramData\Anaconda3\envs\deg\lib\site-packages\hydra\main.py", line 24, in decorated_main
    strict=strict,
  File "C:\ProgramData\Anaconda3\envs\deg\lib\site-packages\hydra\_internal\utils.py", line 147, in run_hydra
    calling_file, calling_module, config_dir
  File "C:\ProgramData\Anaconda3\envs\deg\lib\site-packages\hydra\_internal\utils.py", line 120, in create_automatic_config_search_path
    message="Primary config dir not found: {}".format(search_path_dir),
hydra.errors.MissingConfigException: Primary config dir not found: C:\ProgramData\Anaconda3\envs\deg\Scripts\conf

System information

• Hydra Version : hydra-core 0.11.3
• Python version : 3.7.6
• Virtual environment type and version : conda 4.8.2
• Operating system : Windows

Additional context

I tried to follow the formula outlined in this issue here

[jbohnslav]

Edit: I think it was just a versioning issue. Switching to the "old" version of issue #92 works fine.

[omry]

@jbohnslav, this is the doc for the app packaging.
There is a link there to a working example you should base your thing on.

When 1.0.0 is out, the example in master will be the correct one to follow.

[jbrry]

I am having the same issue. I followed the working example and can confirm that I can run hydra_app as an executable after pip installing in that directory.

I tried the same in my own package, but Hydra is unable to find my default config file despite it being located in mypackage/conf/config.yaml: hydra.errors.MissingConfigException: Cannot find primary config 'config'. Check that it's in your config search path.. Out of curiosity, I copied the working example, and changed the name of hydra_app to foo_app which seems to cause something to break:

cp -r hydra/examples/advanced/hydra_app_example/ .
cd hydra_app_example

Change the name of hydra_app to foo_app:

mv hydra_app/ foo_app/

Change the contents of setup.py to the new path:

from setuptools import find_packages, setup

setup(
    name="foo-app",
    version="0.1",
    packages=find_packages(include=["foo_app"]),
    entry_points={"console_scripts": ["foo_app = foo_app.main:main"]},
    author="you!",
    author_email="your_email@example.com",
    url="http://hydra-app.example.com",
    include_package_data=True,
    install_requires=["hydra-core>=1.0.0"],
)

pip install .

# run the executable:
foo_app

Throws an error:

(hydra392) [jbarry@mycomputer hydra_app_example]$ foo_app
Traceback (most recent call last):
  File "/u/jbarry/miniconda3/envs/hydra392/bin/foo_app", line 8, in <module>
    sys.exit(main())
  File "/u/jbarry/miniconda3/envs/hydra392/lib/python3.9/site-packages/hydra/main.py", line 90, in decorated_main
    _run_hydra(
  File "/u/jbarry/miniconda3/envs/hydra392/lib/python3.9/site-packages/hydra/_internal/utils.py", line 394, in _run_hydra
    _run_app(
  File "/u/jbarry/miniconda3/envs/hydra392/lib/python3.9/site-packages/hydra/_internal/utils.py", line 457, in _run_app
    run_and_report(
  File "/u/jbarry/miniconda3/envs/hydra392/lib/python3.9/site-packages/hydra/_internal/utils.py", line 222, in run_and_report
    raise ex
  File "/u/jbarry/miniconda3/envs/hydra392/lib/python3.9/site-packages/hydra/_internal/utils.py", line 219, in run_and_report
    return func()
  File "/u/jbarry/miniconda3/envs/hydra392/lib/python3.9/site-packages/hydra/_internal/utils.py", line 458, in <lambda>
    lambda: hydra.run(
  File "/u/jbarry/miniconda3/envs/hydra392/lib/python3.9/site-packages/hydra/_internal/hydra.py", line 105, in run
    cfg = self.compose_config(
  File "/u/jbarry/miniconda3/envs/hydra392/lib/python3.9/site-packages/hydra/_internal/hydra.py", line 594, in compose_config
    cfg = self.config_loader.load_configuration(
  File "/u/jbarry/miniconda3/envs/hydra392/lib/python3.9/site-packages/hydra/_internal/config_loader_impl.py", line 142, in load_configuration
    return self._load_configuration_impl(
  File "/u/jbarry/miniconda3/envs/hydra392/lib/python3.9/site-packages/hydra/_internal/config_loader_impl.py", line 243, in _load_configuration_impl
    self.ensure_main_config_source_available()
  File "/u/jbarry/miniconda3/envs/hydra392/lib/python3.9/site-packages/hydra/_internal/config_loader_impl.py", line 129, in ensure_main_config_source_available
    self._missing_config_error(
  File "/u/jbarry/miniconda3/envs/hydra392/lib/python3.9/site-packages/hydra/_internal/config_loader_impl.py", line 102, in _missing_config_error
    raise MissingConfigException(
hydra.errors.MissingConfigException: Primary config module 'foo_app.conf' not found.
Check that it's correct and contains an __init__.py file

So perhaps there is some unexpected behaviour when the package is not called hydra_app?

[omry]

hydra_app shows up in the MANIFEST (which tells the packager what files to put in the package) and in setup.py.
try to update it everywhere.