Packaging hy code with zipapp does not work

[AlexanderWingard]

I'm writing this ticket in response to this How to load Hy package in zipapp? stackoverflow question.

I have tried reproducing this on the latest master a38cdb7 using the files in this gist

It works fine when not using zipapp:

$ python3 hyz
hello hy

But after zipping:

$ python3 -m zipapp hyz
$ python3 hyz.pyz 
Traceback (most recent call last):
  File "/home/lex/anaconda3/lib/python3.7/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/home/lex/anaconda3/lib/python3.7/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "hyz.pyz/__main__.py", line 2, in <module>
ModuleNotFoundError: No module named 'hyz'

[Kodiologist]

As a workaround, you can translate Hy files to Python first with hy2py.

[AlexanderWingard]

Ah, I was experimenting with using hyc to produce pyc files and put them inside the zip but that is not the right approach?

[Kodiologist]

I don't know. You can try it. Python has had ambivalent support for running bytecode without a corresponding source file.

[AlexanderWingard]

Using hy2py worked for my code, however, I really wanted to use the same trick to bring in hy.contrib.hy-repr to be able to print hy data structures nicely but did not work well for those modules.

[Kodiologist]

What do you mean, "did not work well"?

[AlexanderWingard]

Sorry, actually the problem was not in hy-repr but actually from it importing hy.core.language, so when trying to hy2py that file it ends up in some import loop:

Traceback (most recent call last):
  File "/home/lex/anaconda3/lib/python3.7/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/home/lex/anaconda3/lib/python3.7/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "./tmp2.pyz/__main__.py", line 2, in <module>
  File "./tmp2.pyz/hello.py", line 3, in <module>
  File "./tmp2.pyz/hy/contrib/hy_repr.py", line 1, in <module>
  File "./tmp2.pyz/hy/core/language.py", line 2, in <module>
ImportError: cannot import name 'accumulate' from 'hy.core.language' (./tmp2.pyz/hy/core/language.py)

[allison-casey]

zipapp doesn't package dependencies (which Hy is one of) along with the code being packaged. I believe in this use case you'd have to use something like shiv. This could be a good thing to add to the documentation.

[Kodiologist]

Here's a minimal example that follows zipapp's instructions on how to include dependencies. But it still doesn't work, so there's still a bug, presumably.

$ mkdir myapp
$ echo '(print "Hi from Hy.")' > myapp/foo.hy
$ echo 'import hy, foo' > myapp/__main__.py
$ python3 -m pip install hy --target myapp
…
$ python3 myapp
Hi from Hy.
$ python3 -m zipapp myapp
$ python3 myapp.pyz
Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "/tmp/myapp.pyz/__main__.py", line 1, in <module>
  File "/tmp/myapp.pyz/hy/__init__.py", line 15, in <module>
  File "/tmp/myapp.pyz/hy/importer.py", line 175, in _inject_builtins
  File "/tmp/myapp.pyz/hy/macros.py", line 310, in load_macros
  File "/usr/lib/python3.11/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ModuleNotFoundError: No module named 'hy.core.macros'

[Kodiologist]

#2354 partly provides this: hy myapp.pyz with the above example works, but python3 myapp.pyz still doesn't, now with a much longer and more cryptic stack trace.