Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve error presentation in IPython magic #3196

Merged
merged 2 commits into from Oct 21, 2019
Merged

Conversation

mje-nz
Copy link
Contributor

@mje-nz mje-nz commented Oct 20, 2019

When using the IPython magic, if the compilation or build fails then the appropriate errors are printed followed by long tracebacks into Cython internals. These changes hide the tracebacks so only the actual errors are shown.

Compilation failure

Before:

Error compiling Cython file:
------------------------------------------------------------
...
        (void)expander{0, (void(out << ',' << std::forward<Args>(args)), 0)...};
    }
    """
    void doPrint[T1, T2](T1 a, T2 b)

def cprint(a, b)
               ^
------------------------------------------------------------

/Users/matthew/.ipython/cython/_cython_magic_1976d2dff2a61ed3a6c96c26bc6f78aa75443df0.pyx:13:16: Expected ':', found 'NEWLINE'
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-3-d98d1dc3b2a3> in <module>
----> 1 get_ipython().run_cell_magic('cython', '-+', 'cdef extern from *:\n    """\n    template <typename Arg, typename... Args>\n    void doPrint(std::ostream& out, Arg&& arg, Args&&... args)\n    {\n        out << std::forward<Arg>(arg);\n        using expander = int[];\n        (void)expander{0, (void(out << \',\' << std::forward<Args>(args)), 0)...};\n    }\n    """\n    void doPrint[T1, T2](T1 a, T2 b)\n\ndef cprint(a, b)\n    doPrint(a, b)\n    \n')

/usr/local/lib/python3.7/site-packages/IPython/core/interactiveshell.py in run_cell_magic(self, magic_name, line, cell)
   2356             with self.builtin_trap:
   2357                 args = (magic_arg_s, cell)
-> 2358                 result = fn(*args, **kwargs)
   2359             return result
   2360 

</usr/local/lib/python3.7/site-packages/decorator.py:decorator-gen-119> in cython(self, line, cell)

/usr/local/lib/python3.7/site-packages/IPython/core/magic.py in <lambda>(f, *a, **k)
    185     # but it's overkill for just that one bit of state.
    186     def magic_deco(arg):
--> 187         call = lambda f, *a, **k: f(*a, **k)
    188 
    189         if callable(arg):

~/Code/ThirdParty/cython/Cython/Build/IpythonMagic.py in cython(self, line, cell)
    324         if need_cythonize:
    325             extensions = self._cythonize(module_name, code, lib_dir, args, quiet=args.quiet)
--> 326             assert len(extensions) == 1
    327             extension = extensions[0]
    328             self._code_cache[key] = module_name

TypeError: object of type 'NoneType' has no len()

After:

Error compiling Cython file:
------------------------------------------------------------
...
        (void)expander{0, (void(out << ',' << std::forward<Args>(args)), 0)...};
    }
    """
    void doPrint[T1, T2](T1 a, T2 b)

def cprint(a, b)
               ^
------------------------------------------------------------

/Users/matthew/.ipython/cython/_cython_magic_1976d2dff2a61ed3a6c96c26bc6f78aa75443df0.pyx:13:16: Expected ':', found 'NEWLINE'

Build failure

Before:

/Users/matthew/.ipython/cython/_cython_magic_c02b46bbac09e13a84a662a3df7b045249572696.cpp:1281:3: error: no matching function for call to 'doPrint'
  doPrint<int,int>(__pyx_v_a, __pyx_v_b);
  ^~~~~~~~~~~~~~~~
/Users/matthew/.ipython/cython/_cython_magic_c02b46bbac09e13a84a662a3df7b045249572696.cpp:678:10: note: candidate function not viable: no known conversion from 'int' to 'int &&' for
      1st argument
    void doPrint(Arg&& arg, Args&&... args)
         ^
1 error generated.
---------------------------------------------------------------------------
DistutilsExecError                        Traceback (most recent call last)
/usr/local/Cellar/python/3.7.4_1/Frameworks/Python.framework/Versions/3.7/lib/python3.7/distutils/unixccompiler.py in _compile(self, obj, src, ext, cc_args, extra_postargs, pp_opts)
    117             self.spawn(compiler_so + cc_args + [src, '-o', obj] +
--> 118                        extra_postargs)
    119         except DistutilsExecError as msg:

/usr/local/Cellar/python/3.7.4_1/Frameworks/Python.framework/Versions/3.7/lib/python3.7/distutils/ccompiler.py in spawn(self, cmd)
    908     def spawn(self, cmd):
--> 909         spawn(cmd, dry_run=self.dry_run)
    910 

/usr/local/Cellar/python/3.7.4_1/Frameworks/Python.framework/Versions/3.7/lib/python3.7/distutils/spawn.py in spawn(cmd, search_path, verbose, dry_run)
     35     if os.name == 'posix':
---> 36         _spawn_posix(cmd, search_path, dry_run=dry_run)
     37     elif os.name == 'nt':

/usr/local/Cellar/python/3.7.4_1/Frameworks/Python.framework/Versions/3.7/lib/python3.7/distutils/spawn.py in _spawn_posix(cmd, search_path, verbose, dry_run)
    158                           "command %r failed with exit status %d"
--> 159                           % (cmd, exit_status))
    160             elif os.WIFSTOPPED(status):

DistutilsExecError: command 'clang' failed with exit status 1

During handling of the above exception, another exception occurred:

CompileError                              Traceback (most recent call last)
<ipython-input-10-2d51d255771d> in <module>
----> 1 get_ipython().run_cell_magic('cython', '-+ --compile-args=-std=c++11', 'cdef extern from *:\n    """\n    #include <iostream>\n    template <typename Arg, typename... Args>\n    void doPrint(Arg&& arg, Args&&... args)\n    {\n        std::cout << std::forward<Arg>(arg);\n        using expander = int[];\n        (void)expander{0, (void(std::cout << \',\' << std::forward<Args>(args)), 0)...};\n    }\n    """\n    void doPrint[T1, T2](T1 a, T2 b)\n\ndef cprint(int a, int b):\n    doPrint(a, b)\n    \n')

/usr/local/lib/python3.7/site-packages/IPython/core/interactiveshell.py in run_cell_magic(self, magic_name, line, cell)
   2356             with self.builtin_trap:
   2357                 args = (magic_arg_s, cell)
-> 2358                 result = fn(*args, **kwargs)
   2359             return result
   2360 

</usr/local/lib/python3.7/site-packages/decorator.py:decorator-gen-119> in cython(self, line, cell)

/usr/local/lib/python3.7/site-packages/IPython/core/magic.py in <lambda>(f, *a, **k)
    185     # but it's overkill for just that one bit of state.
    186     def magic_deco(arg):
--> 187         call = lambda f, *a, **k: f(*a, **k)
    188 
    189         if callable(arg):

~/Code/ThirdParty/cython/Cython/Build/IpythonMagic.py in cython(self, line, cell)
    332 
    333         self._build_extension(extension, lib_dir, pgo_step_name='use' if args.pgo else None,
--> 334                               quiet=args.quiet)
    335 
    336         module = imp.load_dynamic(module_name, module_path)

~/Code/ThirdParty/cython/Cython/Build/IpythonMagic.py in _build_extension(self, extension, lib_dir, temp_dir, pgo_step_name, quiet)
    441             if not quiet:
    442                 old_threshold = distutils.log.set_threshold(distutils.log.DEBUG)
--> 443             build_extension.run()
    444         finally:
    445             if not quiet and old_threshold is not None:

/usr/local/Cellar/python/3.7.4_1/Frameworks/Python.framework/Versions/3.7/lib/python3.7/distutils/command/build_ext.py in run(self)
    338 
    339         # Now actually compile and link everything.
--> 340         self.build_extensions()
    341 
    342     def check_extensions_list(self, extensions):

/usr/local/Cellar/python/3.7.4_1/Frameworks/Python.framework/Versions/3.7/lib/python3.7/distutils/command/build_ext.py in build_extensions(self)
    447             self._build_extensions_parallel()
    448         else:
--> 449             self._build_extensions_serial()
    450 
    451     def _build_extensions_parallel(self):

/usr/local/Cellar/python/3.7.4_1/Frameworks/Python.framework/Versions/3.7/lib/python3.7/distutils/command/build_ext.py in _build_extensions_serial(self)
    472         for ext in self.extensions:
    473             with self._filter_build_errors(ext):
--> 474                 self.build_extension(ext)
    475 
    476     @contextlib.contextmanager

/usr/local/Cellar/python/3.7.4_1/Frameworks/Python.framework/Versions/3.7/lib/python3.7/distutils/command/build_ext.py in build_extension(self, ext)
    532                                          debug=self.debug,
    533                                          extra_postargs=extra_args,
--> 534                                          depends=ext.depends)
    535 
    536         # XXX outdated variable, kept here in case third-part code

/usr/local/Cellar/python/3.7.4_1/Frameworks/Python.framework/Versions/3.7/lib/python3.7/distutils/ccompiler.py in compile(self, sources, output_dir, macros, include_dirs, debug, extra_preargs, extra_postargs, depends)
    572             except KeyError:
    573                 continue
--> 574             self._compile(obj, src, ext, cc_args, extra_postargs, pp_opts)
    575 
    576         # Return *all* object filenames, not just the ones we just built.

/usr/local/Cellar/python/3.7.4_1/Frameworks/Python.framework/Versions/3.7/lib/python3.7/distutils/unixccompiler.py in _compile(self, obj, src, ext, cc_args, extra_postargs, pp_opts)
    118                        extra_postargs)
    119         except DistutilsExecError as msg:
--> 120             raise CompileError(msg)
    121 
    122     def create_static_lib(self, objects, output_libname,

CompileError: command 'clang' failed with exit status 1

After:

/Users/matthew/.ipython/cython/_cython_magic_c02b46bbac09e13a84a662a3df7b045249572696.cpp:1281:3: error: no matching function for call to 'doPrint'
  doPrint<int,int>(__pyx_v_a, __pyx_v_b);
  ^~~~~~~~~~~~~~~~
/Users/matthew/.ipython/cython/_cython_magic_c02b46bbac09e13a84a662a3df7b045249572696.cpp:678:10: note: candidate function not viable: no known conversion from 'int' to 'int &&' for
      1st argument
    void doPrint(Arg&& arg, Args&&... args)
         ^
1 error generated.

@mje-nz
Copy link
Contributor Author

mje-nz commented Oct 20, 2019

The test failure is the one I fixed in #3188.

@scoder scoder added this to the 3.0 milestone Oct 21, 2019
@scoder scoder merged commit a32a29e into cython:master Oct 21, 2019
scoder pushed a commit that referenced this pull request Nov 1, 2019
* IPython magic: hide internal traceback when compilation fails
* IPython magic: hide internal traceback when build fails
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants