require with relative path crashes the compiler

[daehwannam]

I wrote the following scripts where file_b.hy tries to get macro do-twice in file_a.hy via "require" with relative paths.

relative_test.zip

• directories & files
  relative_test
  ├── dir1
  │ ├── dir2
  │ │ └── file_a.hy
  │ └── dir3
  │ └── file_b.hy
  └── main.hy

• main.hy
  (import dir1.dir3.file_b)

• file_a.hy
  (defmacro do-twice [x] `(do ~x ~x))
  (defn call-twice [f] (f) (f))

• file_b.hy
  (import [..dir2.file_a [call-twice]])
  (call-twice (fn [] (print "hello")))
  ;; the code below raise error
  (require [..dir2.file_a [do-twice]]) ; it works when "..dir2.file_a" is replaced with "dir1.dir2.file_a"
  (do-twice (print "hello"))

When I run main.hy, the import statement in file_b.hy works to get function call-twice (if require statement is commented), but the require statement doesn't work. I think require statement works with absolute path but it doesn't work with relative path. Is it bug or unintended use of require statement?

Here is the execution result of the scripts:
----------------------------------------------shell-begin----------------------------------------------
user@ubuntu:/home/data/user/exercise/hy/relative_test$ hy main.hy
Traceback (most recent call last):
File "/home/user/bin/anaconda3/envs/test/bin/hy", line 10, in
sys.exit(hy_main())
File "/home/user/bin/anaconda3/envs/test/lib/python3.6/runpy.py", line 263, in run_path
pkg_name=pkg_name, script_name=fname)
File "/home/user/bin/anaconda3/envs/test/lib/python3.6/runpy.py", line 96, in _run_module_code
mod_name, mod_spec, pkg_name, script_name)
File "/home/user/bin/anaconda3/envs/test/lib/python3.6/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "/home/data/user/exercise/hy/relative_test/main.hy", line 2, in
(import dir1.dir3.file_b)
File "", line 971, in _find_and_load
File "", line 955, in _find_and_load_unlocked
File "", line 665, in _load_unlocked
File "", line 674, in exec_module
File "", line 781, in get_code
File "/home/user/bin/anaconda3/envs/test/lib/python3.6/importlib/init.py", line 121, in import_module
raise TypeError(msg.format(name))
hy.errors.HyCompileError: Internal Compiler Bug 😱
⤷ Traceback (most recent call last):
File "/home/user/bin/anaconda3/envs/test/lib/python3.6/site-packages/hy/compiler.py", line 433, in compile
ret = self.compile_atom(tree)
File "/home/user/bin/anaconda3/envs/test/lib/python3.6/site-packages/hy/compiler.py", line 427, in compile_atom
return Result() + _model_compilers[type(atom)](self, atom)
File "/home/user/bin/anaconda3/envs/test/lib/python3.6/site-packages/hy/compiler.py", line 1777, in compile_expression
self, expr, unmangle(sroot), *parse_tree)
File "/home/user/bin/anaconda3/envs/test/lib/python3.6/site-packages/hy/compiler.py", line 1196, in compile_import_or_require
prefix=prefix):
File "/home/user/bin/anaconda3/envs/test/lib/python3.6/site-packages/hy/macros.py", line 175, in require
source_module = importlib.import_module(source_module)
File "/home/user/bin/anaconda3/envs/test/lib/python3.6/importlib/init.py", line 121, in import_module
raise TypeError(msg.format(name))
TypeError: the 'package' argument is required to perform a relative import for '..dir2.file_a'

user@ubuntu:/home/data/user/exercise/hy/relative_test$
----------------------------------------------shell-end----------------------------------------------

[Kodiologist]

This is a bug. Relative paths should work the same for require as for import.

[jams2]

I've been looking into this, and there's something odd going on. This patch solves importlib.import_module requiring a package argument:

diff --git a/hy/macros.py b/hy/macros.py
index 3bef40ce..f4cacd4a 100644
--- a/hy/macros.py
+++ b/hy/macros.py
@@ -171,8 +171,15 @@ def require(source_module, target_module, assignments, prefix=""):
         return False
 
     if not inspect.ismodule(source_module):
+        # import pdb; pdb.set_trace()
         try:
-            source_module = importlib.import_module(source_module)
+            if source_module.startswith(".."):
+                package = ".".join(
+                    getattr(target_module, "__name__", target_module).split(".")[:-2]
+                    + source_module.split(".")[2:3])
+            else:
+                package = None
+            source_module = importlib.import_module(source_module, package)
         except ImportError as e:
             reraise(HyRequireError, HyRequireError(e.args[0]), None)
 
diff --git a/tests/native_tests/language.hy b/tests/native_tests/language.hy
index 20689c5b..29946c19 100644
--- a/tests/native_tests/language.hy
+++ b/tests/native_tests/language.hy
@@ -1480,6 +1480,11 @@ cee\"} dee" "ey bee\ncee dee"))
                   "success")
               (except [NameError] "failure")))))
 
+(defn test-relative-require []
+  ""
+  (require [..resources.macros [nonlocal-test-macro]])
+  (assert (in "nonlocal_test_macro" __macros__)))
+
 
 (defn test-encoding-nightmares []
   "NATIVE: test unicode encoding escaping crazybits"

But it intermittently causes some tests to fail. In particular:

    
    (defn test-doc [capsys]
      (defmacro <-mangle-> []
        "a fancy docstring"
        '(+ 2 2))
      (doc <-mangle->)
      (setv [out err] (.readouterr capsys))
      ;; https://github.com/hylang/hy/issues/1946
      (assert (.startswith (.strip out)
                f"Help on function {(mangle '<-mangle->)} in module "))
      (assert (in "a fancy docstring" out))
      (assert (empty? err))
>     #doc @
E     NameError: name '__tags__' is not defined

tests/native_tests/core.hy:713: NameError
================================== short test summary info ==================================
FAILED tests/native_tests/core.hy::test_doc - NameError: name '__tags__' is not defined
============================== 1 failed, 636 passed in 16.15s ===============================

This doesn't always fail. If I insert a breakpoint on the line before the #doc @ call, and continue immediately, the tests pass on that run and one subsequent run. Any ideas?

[Kodiologist]

That test failure predates your patch; see #1970.