witty 0.3.2 breaks all C extension builds: UnknownFileType: unknown file type '.h'

[yfukai]

Human summary

I found that the' spatial_graph' installation is sometimes broken. Claude found out that the error is due to a witty version update. At least I tried avoiding witty 0.3.2, and that resolved the issue.

Claude summary

Every spatial_graph C extension fails to build when witty 0.3.2 is installed. Since both spatial-graph 0.0.6 and witty 0.3.2 are the current releases on PyPI, a fresh install is broken by default.

Reproduction

import spatial_graph as sg

sg.SpatialGraph(
    ndims=2,
    node_dtype="uint64",
    node_attr_dtypes={"pos": "double[2]"},
    edge_attr_dtypes={},
    position_attr="pos",
)

distutils.compilers.C.errors.UnknownFileType: unknown file type '.h'
  (from .../site-packages/spatial_graph/_graph/src/graph_lite.h)

RTree fails the same way on _rtree/src/config.h.

Note when reproducing: clear the witty cache between attempts, otherwise a previously built module is reused and hides the failure. The cache lives at ~/Library/Caches/witty on macOS and ~/.cache/witty on Linux.

Cause

witty 0.3.2 redefined what source_files means:

0.3.1  source_files : Additional source files the PYX code depends on. Changes to
                      these files will trigger re-compilation of the module.

0.3.2  source_files : Additional source files (.c, .cpp files) that will be compiled
                      and linked against. Changes to these files will trigger
                      re-compilation of the module.
       depends_on   : List of additional files that the module depends on. Only used
                      for cache invalidation.

The old cache-invalidation meaning moved to the new depends_on parameter, and source_files is now passed to the compiler. spatial_graph still lists its headers in source_files, so .h files reach distutils as compilation units and are rejected.

Two call sites are affected:

• spatial_graph/_rtree/rtree.py:51 — rtree.h, rtree.c, config.h
• spatial_graph/_graph/graph_base.py:110 — graph_lite.h

Bisection

Tested against setuptools 83.0.0 on Python 3.11, clearing the witty cache before each run:

spatial-graph	witty	result
0.0.5	0.3.1	ok
0.0.6	0.3.1	ok
0.0.5	0.3.2	fail
0.0.6	0.3.2	fail

The break follows witty alone. setuptools is not involved — 80.9.0 and 83.0.0 behave identically.

Suggested fix

Move the headers to depends_on, keeping only real translation units in source_files.

# _rtree/rtree.py
    source_files=[
        SRC_DIR / "src" / "rtree.c",
    ],
    depends_on=[
        SRC_DIR / "src" / "rtree.h",
        SRC_DIR / "src" / "config.h",
    ],

# _graph/graph_base.py
    # graph_lite is header-only, so nothing to compile separately
    depends_on=[str(SRC_DIR / "src" / "graph_lite.h")],

depends_on only exists from witty 0.3.2, so this would need a witty>=0.3.2 floor.

Workaround

Until then, downstream projects can pin:

"witty!=0.3.2"

[tlambert03]

wow! @funkey and I estimated that surely nobody would notice a temporary break that quickly 😂. So sorry @yfukai ... I'm aware there was a breakage here and will fix asap. Indeed, please just pin <0.3.2 for now, and I'll release a fix soon

[tlambert03]

actually... i just realized @funkey already pushed 2799e02 ... so I'll just cut a new release

[tlambert03]

when you have a moment, could you try spatial-graph v0.0.7

[TeunHuijben]

@tlambert03, v0.0.7 works for me, thanks for the quick fix! 🙌

[yfukai]

@tlambert03 Thanks a lot! We're a great fan of the rtree implementation 😄