Add link to C file in Cython failures and -a for annotated html

[fperez]

In the cython magic extension, it would be nice to return a link to the generated C file that can be opened in the browser as well as supporting the -a flag to produce annotated HTML. The annotation flag can be activated by setting

Cython.Compiler.Options.annotate = True

These two simple enhancements would make debugging and optimization of Cython code much more user friendly.

[minrk]

That should be pretty easy, just by putting the compiled file and generated html inside the notebook-dir, since everything there is already available.

[fperez]

Yes, I think it's very straightforward. I was tempted to do it just now but I'm in the middle of other stuff, so I decided to at least record it here...

[bfroehle]

Unfortunately this is a not a completely straightforward task as our call to cythonize passes the options as part of the ctx parameter which has been ignored since 0.14.

Since our %%cython magic is based upon cython.inline, I filed a bug report upstream. I'm not sure quite what the best way to fix this yet is, but regardless I expect that it shouldn't be very difficult.

[fperez]

@bfroehle, what about setting the global Cython flag as I indicated above? I took that tip from a thread on the cython list just a few days ago, so I assumed it would work fine...

[bfroehle]

I just closed #2143 as a duplicate. There is a small bit of code in the issue which may be useful.

[dhirschfeld]

There's an implementation in #PR2147 that works for me on win32. Whilst it seems to work perfectly in the Notebook, in the QtConsole the underlying c code which is usually collapsed is visible.

Having done this I now understand the point Fernando was making in his initial comment.
Cython.Compiler.Options.annotate = True
...is probably a much cleaner solution. I'll try to find time to look into it further.

[dhirschfeld]

It appears that the cythonize command could automatically annotate the source by passing annotate = True as an option. The cythonize command is however within an if statement if force or not os.path.isfile(module_path) which I believe prevents re-cythonizing the same source?

It appears that unconditionally re-cythonizing leads to a link error due to a permission error on the .pyd which I assume means your session is holding onto a reference to it because it has already been imported? It seems strange then that the cython -a command would work in the subprocess command at the end...

I'm a bit stumped there - I haven't been able to get it to work correctly other than the code which is in my pull-request.

[bfroehle]

--annotate was merged in #2225.

I'm not sure what @fperez meant by failures in the title. If the cython fails to compile the best we can do is output the error message (which we already do). I suppose it could fail to compile?

[fperez]

Yes, sometimes cython will sucessfully emit C code that then gcc can't compile. Having a link to the auto-generated C file is very useful, since often looking directly at the raw (and hyper-ugly) C is the only way to figure out what went wrong. Since these filenames look random to the human eye, manually rooting through your cache is kind of annoying, and a direct clickable link would help a lot.

I've had to do this very kind of manual debugging in the past, whence my opening of this issue. I'm pretty sure sage does give a link to the C file for this very reason.

[takluyver]

Any progress or thoughts on putting the C file somewhere accessible, and/or printing its location?