I've written a dependency tracker for cython. The tracker is pretty much stand alone in an extra file (including tests) and can also be easily used in other tools.
I enhanced the scons support that was already there by hooking my tracker into it. I am using this to build my scientific code. I also added a getting started README for new scons users that should get them up to speed quickly.
What do you think?
Btw, glad that you are on github now. Git makes more sense to me than hg does :).
Enhanced Scons support by adding dependency tracking
Cool. My only thought is that it would make more sense to leverage the dependency checker that's already part of Cython (https://github.com/cython/cython/blob/master/Cython/Build/Dependencies.py), rather than having two implementations. The latter correctly handles cases like
my_docstring = """
cimport this module
or pxi files that contain cimports (which are resolved relative to the including file, rather than relative to the included file). You could then also leverage inline build options "distutils: option=value" for scons.
I wasn't aware of this implementation. Sadly it is not documented (afaik) and it is not trivial for me where the entry point is. Could you help me out which function returns the dependant files?
Look in Cython/Build/Dependencies.py, in particular create_dependency_tree(). The context is primarily to set up the correct include paths, see the cythonize() function for an entry point. It could perhaps be cleaned up a bit to better fit into your model, but would be code well worth leveraging.
I invested an hour now in this. I ran into bugs with the dependency tree, for my source files dt.cimported_files(fn) recurses infinitively. I also realized that the dependency code also just uses regular expression with some preprocessing that feels unsafe, so I see no big benefits in using this and I also do not see where it is superior to my implementation. I also think my implementation is easier to work with because it comes with tests. So, I will not invest more time in turning my scons dependency tracker into using the Cython Dependency tracker. Whoever is working on this might consider taking a peek at my tracker.
I would also need a way to tell the Context that we are working from another cwd. making a chdir and back is the only way I got this working, but this is error prone.
So feel free to merge my enhancements to the scons tool or discard this pull request. I will not work on it any further for the moment.