chore!: drop Darshan and Recorder support#67
Merged
Conversation
izzet
force-pushed
the
chore/drop-darshan-recorder
branch
from
July 26, 2026 01:07
51a03dc to
aaa153e
Compare
Collaborator
|
i think it is intersected with #68 right? |
hariharan-devarajan
approved these changes
Jul 26, 2026
DFAnalyzer is a DFTracer tool by identity -- it ships as dftracer-analyzer
and installs into the dftracer.analyzer namespace. Trace reading moved to
the native C++ Indexer, which darshan.py and recorder.py bypassed with
their own read_trace, so both drifted further from the core with every
release. Recorder support was already partial: the analyzer only read
parquet, and raw traces needed the bundled C++ converter first.
pydarshan also has no cp313 wheel, and its sdist builds without a bundled
libdarshan-util.so, so `import darshan` raises RuntimeError at runtime --
which the ModuleNotFoundError guard in __init__.py did not catch, taking
down the whole package import. Removing it unblocks Python 3.13.
Removed:
- darshan.py, recorder.py and their analyzer config groups
- tools/ (recorder2parquet) and the backward-cpp, cpp-logger and
recorder meson subprojects it pulled in
- the enable_tools, enable_tests and link_shared meson options, which
existed only for that tool
- the darshan extra, 4 test fixtures, and the darshan/recorder e2e cases
- CI's arrow apt source plus the arrow, MPI, ncurses, readline, parquet
and hdf5 system dependencies, needed only by the C++ tool
Darshan and Recorder analysis remains available in WisIO, which carries
its own copies of these analyzers.
izzet
force-pushed
the
chore/drop-darshan-recorder
branch
from
July 26, 2026 04:36
aaa153e to
e1758e4
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Removes the Darshan and Recorder analyzers. Darshan and Recorder analysis remains available in WisIO, which carries its own copies of both analyzers and the
recorder2parquetconverter.Why
DFAnalyzer is a DFTracer tool by identity — it ships as
dftracer-analyzerand installs into thedftracer.analyzernamespace. Trace reading has moved to the native C++Indexer, whichdarshan.pyandrecorder.pybypass entirely with their ownread_trace, so both drift further from the core with every release. Every recent feature — the fact engine, category-layer discovery, agentic presets — is DFTracer-shaped.Recorder support was already partial: the analyzer only reads parquet (
dd.read_parquet), and raw Recorder traces required the bundled C++ converter first.Darshan is also actively costing us. pydarshan has no cp313 wheel, and its sdist builds without a bundled
libdarshan-util.so, soimport darshanraisesRuntimeErrorat runtime — which theexcept ModuleNotFoundErrorguard in__init__.pydoes not catch, taking down the entiredftracer.analyzerimport rather than just darshan support. Removing it unblocks Python 3.13.Removed
darshan.py,recorder.py, and theiranalyzer=config groupstools/(recorder2parquet) and thebackward-cpp,cpp-loggerandrecordermeson subprojects it pulled inenable_tools,enable_testsandlink_sharedmeson options, which existed only for that tooldarshanextra, 4 test fixtures, and the darshan/recorder e2e matrix entries32 files, −2,164 lines.
Verification
pytest -m full— 159 passed, 22 deselected, in a clean non-editable venv.init_hydra_config_store()now exposes exactly one analyzer group:dftracer.Note
tools/meson.builddefined atest_recorder2parquetmeson test consumingrecorder-posix-raw.tar.gz; that goes away with the tool.Merge order
Please merge before the Meson removal, which builds on this branch.