cython magic fail if cache dir is deleted while in session

[juliantaylor]

In [1]: %load_ext cythonmagic

In [2]: %%cython
cimport cython
   ...: 

In [3]: rm -rf /home/jtaylor/.config/ipython/cython/

In [4]: %%cython                                    
cimport cython

CompileError: command 'x86_64-linux-gnu-gcc' failed with exit status 1

the command issued is:

x86_64-linux-gnu-gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -I/usr/include/python2.7 -I/usr/include/x86_64-linux-gnu/python2.7 -c /home/jtaylor/.config/ipython/cython/_cython_magic_d37c0b24f28dfb69270e82105a4c2e8e.c -o /home/jtaylor/.config/ipython/cython/home/jtaylor/.config/ipython/cython/_cython_magic_d37c0b24f28dfb69270e82105a4c2e8e.o
Fatal error: can't create /home/jtaylor/.config/ipython/cython/home/jtaylor/.config/ipython/cython/_cython_magic_d37c0b24f28dfb69270e82105a4c2e8e.o: No such file or directory

the problematic part is this long path which is not recreated before trying to compile into it:

/home/jtaylor/.config/ipython/cython/home/jtaylor/.config/ipython/cython/

[minrk]

closed by e945888

[juliantaylor]

I can still reproduce it with current git head, note that with that commit you have to delete ~/.cache/ipython/cython/

[minrk]

Gotcha, I must have done something wrong in testing. Thanks for checking.

I tracked this down to a bug in distutils, where its mkpath utility will only ever create a particular path once,
and assume that it still exists forever after for the lifetime of the process. I can make a workaround, but it might be gross.