Enable cython caching

[robertwb]

The entire Sage library fits into about 25MB (compressed) cache.

Component: build

Author: Robert Bradshaw

Branch/Commit: u/vbraun/ticket/15430 @ 064791b

Reviewer: Volker Braun

Issue created by migration from https://trac.sagemath.org/ticket/15430

[robertwb]

Branch: u/robertwb/ticket/15430

[robertwb]

Author: Robert Bradshaw

[robertwb]

New commits:

[c93d084](https://github.com/sagemath/sagetrac-mirror/commit/c93d084)

Enable cython cache.

[robertwb]

Commit: c93d084

[vbraun]

Reviewer: Volker Braun

[vbraun]

comment:3

Dependency checking seems to work for me. If there is any problem left then I'm pretty sure we'll only be able to find it if we use this seriously. So I'm in favor of pushing this into the sage-git release....

[vbraun]

comment:4

Building Sage with a non-existing DOT_SAGE breaks because of this ticket. From the buildbot:

Compiling sage/ext/interpreters/wrapper_rdf.pyx because it changed.
Compiling sage/ext/interpreters/wrapper_cdf.pyx because it changed.
Compiling sage/ext/interpreters/wrapper_rr.pyx because it changed.
Compiling sage/ext/interpreters/wrapper_py.pyx because it changed.
Compiling sage/ext/interpreters/wrapper_el.pyx because it changed.
Traceback (most recent call last):
  File "setup.py", line 531, in <module>
    force=force)
  File "/home/buildbot/build/sage/plantain/sage_git/build/local/lib/python2.7/site-packages/Cython/Build/Dependencies.py", line 740, in cythonize
    os.mkdir(options.cache)
OSError: [Errno 2] No such file or directory: '/home/buildbot/build/sage/plantain/sage_git/dot_sage/cycache'

[vbraun]

Changed branch from u/robertwb/ticket/15430 to u/vbraun/ticket/15430

[vbraun]

Changed commit from c93d084 to 064791b

[vbraun]

comment:6

Hmm the commit field was not automatically updated?

---

New commits:

[064791b](https://github.com/sagemath/sagetrac-mirror/commit/064791b)

only enable cycache if DOT_SAGE exists

[vbraun]

comment:7

please review my change...

[ohanar]

comment:8

Why use a sage specific cycache directory? Why not something like ~/.cycache? (IMHO, Cython should use a default directory if Cython.Compiler.Main.default_options['cache'] is True).

Also, I think it should be togglable by an environment variable, say DISABLE_CYCACHE, in case someone doesn't want to use (e.g. for testing purposes/cython development purposes).

[vbraun]

comment:9

The ccache spkg also uses $DOT_SAGE/ccache.

With this patch you can disable caching by passing an invalid directory. Ugly but yet another undocumented environment variable isn't that great either.

Supporting boolean Cython.Compiler.Main.default_options['cache'] and/or using os.makedirs() instead of mkdir are just Cython enhancement requests, maybe you want to post that to the Cython trac?

[robertwb]

comment:10

This is quite the ugly hack, but should work. We can roll this back once cython/cython@b24ea41 is released.

[jdemeyer]

comment:12

I assume the intent of this patch was to always enable cycache by default? Because that's not what this patch does. It only works if the cycache directory already exists. Testing for the existence of X/.. is (almost) equivalent to testing for the existence of X.

[jdemeyer]

comment:13

Replying to @vbraun:

The ccache spkg also uses $DOT_SAGE/ccache.

That's not actually true. Sage doesn't set the ccache directory.

[jdemeyer]

comment:14

Replying to @robertwb:

This is quite the ugly hack, but should work. We can roll this back once cython/cython@b24ea41 is released.

I guess this is in Sage now, right?

[vbraun]

comment:15

Yes, feel free to open a new ticket.

[jdemeyer]

comment:16

See #16148.