Trac #16814: Add SAGE_PROFILE to enable profiler

The new `SAGE_PROFILE` environment variable controls whether to build
with profiling support. It defaults to off since this potentially is a
big performance impact. Right now it only controls Cython profiling, but
third-party packages should support it in the future if they can.

With `SAGE_PROFILE=no`:
{{{
 sage: %prun 1 + 1
         2 function calls in 0.000 seconds

   Ordered by: internal time

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
        1    0.000    0.000    0.000    0.000 <string>:1(<module>)
        1    0.000    0.000    0.000    0.000 {method 'disable' of
'_lsprof.Profiler' objects}
}}}
With `SAGE_PROFILE=yes`:
{{{
 sage: %prun 1 + 1
         13 function calls in 0.000 seconds

   Ordered by: internal time

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
        1    0.000    0.000    0.000    0.000 <string>:1(<module>)
        1    0.000    0.000    0.000    0.000 element.pyx:1538(__add__)
        1    0.000    0.000    0.000    0.000 integer.pyx:1605(_add_)
        3    0.000    0.000    0.000    0.000
integer.pyx:6276(fast_tp_new)
        2    0.000    0.000    0.000    0.000 integer.pyx:512(__init__)
        3    0.000    0.000    0.000    0.000
integer.pyx:6362(fast_tp_dealloc)
        1    0.000    0.000    0.000    0.000
coerce.pxi:7(have_same_parent)
        1    0.000    0.000    0.000    0.000 {method 'disable' of
'_lsprof.Profiler' objects}
}}}

URL: http://trac.sagemath.org/16814
Reported by: vbraun
Ticket author(s): Volker Braun
Reviewer(s): Martin Raum

src/doc/en/installation/source.rst

@@ -860,6 +860,11 @@ Here are some of the more commonly used variables affecting the build process:
     These will be notably slower but, for example, make it much easier to
     pinpoint memory allocation problems.
 
+- :envvar:`SAGE_PROFILE` - controls profiling support. If this is set
+  to ``yes``, profiling support is enabled where possible. Note that
+  Python-level profiling is always avaliable; This option enables
+  profiling in Cython modules.
+
 - :envvar:`SAGE_SPKG_LIST_FILES` - if set to ``yes``, then enable verbose
   extraction of tar files, i.e. Sage's spkg files.
   Since some spkgs contain such a huge number of files that the log files

src/setup.py

@@ -514,27 +514,43 @@ def run_cythonize():
     # enclosing Python scope (e.g. to perform variable injection).
     Cython.Compiler.Options.old_style_globals = True
 
+    debug = False
     if os.environ.get('SAGE_DEBUG', None) != 'no':
+        print('Enabling Cython debugging support')
+        debug = True
         Cython.Compiler.Main.default_options['gdb_debug'] = True
         Cython.Compiler.Main.default_options['output_dir'] = 'build'
 
+    profile = False
+    if os.environ.get('SAGE_PROFILE', None) == 'yes':
+        print('Enabling Cython profiling support')
+        profile = True
+
     # Enable Cython caching (the cache is stored in ~/.cycache which is
     # Cython's default).
     Cython.Compiler.Main.default_options['cache'] = True
 
     force = True
     version_file = os.path.join(os.path.dirname(__file__), '.cython_version')
-    if os.path.exists(version_file) and open(version_file).read() == Cython.__version__:
+    version_stamp = '\n'.join([
+        'cython version: ' + str(Cython.__version__),
+        'debug: ' + str(debug),
+        'profile: ' + str(profile),
+    ])
+    if os.path.exists(version_file) and open(version_file).read() == version_stamp:
         force = False
 
     global ext_modules
     ext_modules = cythonize(
         ext_modules,
-        nthreads = int(os.environ.get('SAGE_NUM_THREADS', 0)),
-        build_dir = 'build/cythonized',
-        force=force)
-
-    open(version_file, 'w').write(Cython.__version__)
+        nthreads=int(os.environ.get('SAGE_NUM_THREADS', 0)),
+        build_dir='build/cythonized',
+        force=force,
+        compiler_directives={
+            'profile': profile,
+        })
+
+    open(version_file, 'w').write(version_stamp)
 
 
 print "Updating Cython code...."