-
Notifications
You must be signed in to change notification settings - Fork 0
Documentation Index & Master Reference
GRANITE v0.6.8 | ← Developer Guide | FAQ →
Version: v0.6.8 | Last updated: April 30, 2026
Purpose: Exhaustive, per-document inventory of every file in the GRANITE repository. This page is the single authoritative entry point for all documentation, code references, and engineering records. Every document is listed with its full scope, key sections, known gaps, and cross-references.
- How to Use This Index
- Document Map — Quick Reference
- Root-Level Documents
- docs/ — Technical Documentation
- .github/ — Community & Governance
- benchmarks/ — Simulation Configurations
- scripts/ — Python Tooling
- python/granite_analysis/ — Analysis Package
- containers/ — Deployment
- viz/ — Visualization
- include/granite/ — Public C++ Headers
- src/ — Physics Kernels
- tests/ — Test Suite
- Gap Analysis — What Is Missing
- Cross-Reference Matrix
This page is structured as a per-file encyclopedia. For every document in the repository you will find:
- Path — exact location in the repository
- Size / scope — approximate line count and content density
- Purpose — what the document is for and who the intended reader is
- Key sections — every major section listed and described
- Known gaps — content that is absent, outdated, or requires update
- Cross-references — other documents that cover related material
If you are a new contributor, start with §3.1 README.md → §4.8 Installation.md → §4.1 DEVELOPER_GUIDE.md.
If you are a physicist evaluating GRANITE, start with §4.3 SCIENCE.md → §4.2 BENCHMARKS.md → §4.4 COMPARISON.md.
If you are debugging a simulation, go directly to §4.6 diagnostic_handbook.md and §4.1 DEVELOPER_GUIDE.md §21.
| Document | Location | Lines | Audience | Last-Confirmed Accurate |
|---|---|---|---|---|
| README | root | ~620 | Everyone | v0.6.8 ✅ |
| CHANGELOG | root | ~330 | Developers | v0.6.8 ✅ |
| CITATION.cff | root | ~20 | Academics | v0.6.8 ✅ |
| DEVELOPER_GUIDE | docs/developer_guide/ |
~1368 | Developers | v0.6.8 ✅ |
| BENCHMARKS | docs/user_guide/ |
~313 | Scientists | v0.6.8 ✅ |
| SCIENCE | docs/theory/ |
~333 | Scientists | v0.6.8 ✅ |
| COMPARISON | docs/developer_guide/ |
~200 | Scientists | v0.6.8 ✅ |
| FAQ | docs/user_guide/ |
~200 | Everyone | v0.6.8 ✅ |
| diagnostic_handbook | docs/user_guide/ |
~133 | Simulators |
Partially outdated |
| v0.6.5_master_dictionary | docs/developer_guide/ |
~500 | Developers |
Partially outdated |
| Installation | docs/getting_started/ |
~300 | Users | v0.6.8 ✅ |
| DEPLOYMENT_AND_PERFORMANCE | docs/user_guide/ |
~200 | HPC admins | v0.6.8 ✅ |
| CONTRIBUTING | .github/ |
~100 | Contributors | v0.6.8 ✅ |
| CODE_OF_CONDUCT | .github/ |
~50 | Everyone | v0.6.8 ✅ |
| SECURITY | .github/ |
~80 | Security researchers | v0.6.8 ✅ |
| validation_tests.yaml | benchmarks/ |
~200 | CI / validators | v0.6.8 ✅ |
granite_analysis.cli.sim_tracker |
python/granite_analysis/cli/ |
~400 | Simulators | v0.6.8 ✅ |
granite_analysis.cli.dev_benchmark |
python/granite_analysis/cli/ |
~300 | Developers | v0.6.8 ✅ |
requirements.txt |
python/ |
~10 | CI / Docker users | v0.6.8 ✅ |
health_check.py |
scripts/ |
~150 | Users | v0.6.8 ✅ |
Path: README.md
Size: ~500 lines | Audience: All visitors, scientists, contributors, institutions
The primary public face of the GRANITE project. Displayed automatically by GitHub on the repository homepage. The README serves as a technical executive summary, a benchmark proof-of-concept, a quick-start guide, and a community call-to-action simultaneously.
| Section | Content |
|---|---|
| Developer Note | Personal message from LiranOG explaining the project vision and WSL-only deployment decision |
| Key Features | Bullet list of 7 core capabilities: CCZ4, GRMHD, AMR, multi-BH ID, M1 radiation, diagnostics, HDF5 I/O |
| How GRANITE Compares | 10-row feature matrix vs. Einstein Toolkit, GRChombo, SpECTRE, AthenaK |
| Benchmark Results | Two tables: Single Puncture (64³ and 128³) and Binary BBH (64³ and 96³) with exact ‖H‖₂ values |
| Quick Start Guide | 6-step guide with per-OS commands (WSL2, Linux, macOS, Windows native, Conda) |
| Repository Structure | Full annotated directory tree of every folder and key file |
| Roadmap | 5-row version roadmap table (v0.6.5 → v1.0.0) |
| Known Limitations | 8-row table of confirmed limitations with impact, status, and planned fix version |
| Versioning Policy | Explanation of why GitHub Releases are not used until v1.0.0; CHANGELOG.md is authoritative |
| To the Community | Personal letter explaining the collaborative vision |
| Institutional Partnership | Table of partner types sought (supercomputing centres, NR groups, LIGO/Virgo, departments) |
| HPC Quick-Start |
run_granite_hpc.py command with all flags shown |
| Contributing | Pointer to CONTRIBUTING.md and format command |
| Documentation Table | 8-row table linking all docs/ files with descriptions |
| Citing GRANITE | BibTeX citation block |
| Contributors | GitHub contrib.rocks image |
| License | GPL-3.0-or-later |
- The Documentation Table in README links to
docs/files but does not link to any wiki pages, making the wiki invisible to new visitors. The wiki URL should be added. - The Known Limitations table lists
alpha_centerreads from AMR level 0 — but this limitation is not documented in the diagnostic_handbook or DEVELOPER_GUIDE §21 debugging flowchart. These need to be updated. - The Benchmark Results section has no gauge_wave benchmark results, only SP and BBH. A gauge_wave row should be added once the run is documented.
- macOS is listed as supported in the Quick Start (Homebrew section exists) but listed as unsupported in the "OS Requirement" note and the Known Limitations table. This contradiction should be resolved.
- Full benchmark data →
docs/BENCHMARKS.md - Architecture details →
docs/DEVELOPER_GUIDE.md - Installation →
docs/getting_started/Installation.md - Community files →
.github/
Path: CHANGELOG.md
Size: ~1400 lines, 89.7 KB | Audience: Developers, code reviewers, the scientific community
The canonical, authoritative engineering history of GRANITE. By project policy (documented in README §Versioning Policy), all version history, phase documentation, and change logs are tracked exclusively here — not in GitHub Releases — until v1.0.0. This is the single source of truth for every bug fix, architectural decision, and milestone.
| Version / Phase | Content |
|---|---|
| v0.6.5 — The Stability Update | Master reset of CCZ4 core and AMR; memory safety fix (levels_.reserve()); MICRO_OFFSET singularity avoidance; sim_tracker procedural architecture restore; 8-layer stability summary guard |
| v0.6.0 — The Puncture Tracking Update | TwoPuncturesBBH initial data; full Berger-Oliger subcycling; block-merging algorithm; CFL synchronization; coarse-fine interpolation fixes; gauge_wave and B2_eq benchmarks added; validation_tests.yaml; 90→92 tests; DEPLOYMENT_AND_PERFORMANCE.md; sim_tracker.py |
| v0.5.0 — Repository Professionalization | Root directory cleanup; scripts/ categorization; docs/ restructuring; .github/ community files |
| Phase 7 — Selective Upwinding & Sommerfeld BCs | Chi-weighted hybrid advection stencil; Sommerfeld 1/r BCs; algebraic constraint enforcement; adaptive CFL guardian; runtime floors |
| Phase 6 — Single Puncture Stability Diagnostics | Comprehensive crash root-cause analysis: domain-size limit confirmed; evidence matrix from 6 controlled experiments; gauge wave travel time formula; dev_benchmark.py |
| Phase 5 — PPM, HPC Optimizations, GW Analysis | M5: general spin-weighted spherical harmonics (Goldberg 1967); M4: radiated momentum (Ruiz 2008); PPM reconstruction (Colella & Woodward 1984); GridBlock flat allocation (H2 fix); RHS loop order (H3 fix); OpenMP confirmed in CCZ4 main loop |
| Phase 4D — GR Metric Coupling, MP5 | C1 fix: EOS-exact sound speed; C3 fix: flat-metric HLLE → GRMetric3; H1 fix: KO dissipation single OMP region; MP5 5th-order reconstruction; 7 new tests |
| Phases 4A/4B/4C — HLLD, CT, Tabulated EOS | HLLD Riemann solver (Miyoshi & Kusano 2005); Constrained Transport (Evans & Hawley 1988); TabulatedEOS with tri-linear interpolation + NR temperature inversion + beta-equilibrium; 20 new EOS tests; TOV solver RSUN_CGS bug fixed |
| Earlier commits | Non-blocking MPI ghost zone sync; AMR+GRMHD coupling integration; CI/CD pipeline setup; CodeQL static analysis |
The CHANGELOG contains the definitive list of all confirmed fixed bugs with their IDs (C1, C3, H1, H2, H3, TOV, KO-σ, Sommerfeld+BL, Domain, alpha_center). These must never be re-introduced. Any PR touching the relevant code must cross-check this list.
- The v0.6.5 entry does not document the correct
ko_sigma: 0.1default explicitly (it is implied by the v0.5.0 → Phase 7 change but not stated in the v0.6.5 summary). - Phase labels (Early/Mid/Late Inspiral being time-based not separation-based) are listed as a known limitation in README but this change is not in the CHANGELOG as a documented decision with rationale.
- There is no CHANGELOG entry describing the
alpha_centerdiagnostic bug (reads from AMR level 0). This was discovered in code audit but not formally recorded. - The CHANGELOG does not have a "## [Unreleased]" section tracking in-flight work for v0.7.
- For physics justification of each bug fix →
docs/DEVELOPER_GUIDE.md §20 - For full simulation logs that motivated Phase 6 →
docs/internal/
Path: CITATION.cff
Size: ~20 lines | Audience: Academic users who want to cite GRANITE
Machine-readable citation metadata in the Citation File Format. GitHub natively reads this file and displays a "Cite this repository" button on the repository page. Enables automated citation generation in APA, BibTeX, and other formats.
cff-version: 1.2.0type: softwaretitle: GRANITE — General-Relativistic Adaptive N-body Integrated Tool for Extreme Astrophysicsversion: v0.6.5date-released: 2026-04-10- Author entry for LiranOG
- Repository URL
- License: GPL-3.0-or-later
- No
doifield — a Zenodo DOI should be added when the first formal release is deposited. - No
preferred-citationblock pointing to the GRANITE paper (in preparation). - The
referencesarray (upstream physics papers — Alic 2012, Miyoshi 2005, etc.) is not populated. Adding these would allow citation managers to automatically suggest all relevant papers.
- Full BibTeX block →
README.md §Citing GRANITE - Full reference list →
docs/SCIENCE.md §8 References
Path: LICENSE
Size: ~674 lines | Audience: All users, institutions, redistributors
Content: GNU General Public License v3.0 or later (GPL-3.0-or-later). GRANITE is fully free and open source under the GPL-3.0 copyleft license. Any derivative work distributed publicly must be released under the same license.
Important for institutions: The GPL-3.0 license is compatible with academic research and educational use. Commercial users who distribute GRANITE-derived software must release their modifications under GPL-3.0.
Path: CMakeLists.txt
Size: ~200 lines | Audience: Build system engineers, contributors
The master CMake build system for GRANITE. Controls compilation of all C++ targets: the main engine (granite_main), the shared library (granite_lib), and the test suite (granite_tests).
| Option | Default | Description |
|---|---|---|
GRANITE_ENABLE_MPI |
ON | Links OpenMPI or MPICH |
GRANITE_ENABLE_OMP |
ON | Enables -fopenmp / /openmp
|
GRANITE_ENABLE_HDF5 |
ON | Parallel HDF5 for I/O and checkpoints |
GRANITE_ENABLE_CUDA |
OFF | CUDA GPU kernels (roadmap) |
GRANITE_ENABLE_SANITIZERS |
OFF | ASan + UBSan for debug builds |
| Release flags (GCC/Clang) | -O3 -march=native -ffast-math |
Full optimization |
| Release flags (MSVC) | /O2 /Ob2 /arch:AVX2 /fp:fast /MT |
AVX2 vectorization |
FetchContent dependencies:
-
yaml-cpp 0.8.0— YAML parameter file parsing -
GoogleTest 1.12— Unit testing framework
Known Gaps:
- The
GRANITE_ENABLE_CUDA=ONpath exists as an option but the CUDA source files are not yet present. Enabling it will produce a clean configure but no GPU kernels. - PETSc is linked via
PkgConfig::PETSCbut is not used in any current source file — it is reserved for future elliptic solvers.
Path: pyproject.toml
Size: ~30 lines | Audience: Python developers installing granite_analysis
Purpose: Enables pip install -e . installation of the granite_analysis Python package from the repository root. Declares package metadata and dependencies.
Key dependencies declared: numpy, h5py, matplotlib, scipy, pyyaml
Path: .clang-format
Size: ~20 lines | Audience: All C++ contributors
Purpose: Enforces consistent C++ code formatting using LLVM clang-format style. Applied automatically via python3 scripts/run_granite.py format. Key settings: LLVM style base, 100-character line limit. All PRs must pass the format check.
Path: .gitignore
Size: ~40 lines | Audience: Contributors
Purpose: Prevents build artifacts, simulation output, and personal configuration files from being committed. Key excluded patterns: build/, *.h5, *.vtk, dev_logs/, runs/, output/, __pycache__/.
Path: docs/developer_guide/DEVELOPER_GUIDE.md
Size: 1368 lines, 52.7 KB | Audience: C++ contributors, physics developers, HPC engineers
The comprehensive, single-file technical bible for GRANITE. Covers every aspect of the engine from scientific motivation through architecture, physics formulations, numerical methods, data structures, coding standards, testing, and HPC deployment. This is the most important technical document in the repository.
| § | Title | Key Content |
|---|---|---|
| 1 | Project Philosophy & Scientific Motivation | Why GRANITE exists; capability gap vs. existing codes; intellectual lineage (NRCF → PRISM → SYNAPSE → AUE → GRANITE → GRAVITAS-X); core design principles table |
| 2 | Repository Structure | Annotated tree of every file in every directory, with one-line purpose for each |
| 3 | Architecture Overview | Full ASCII data-flow diagram from YAML → Initial Data → AMR → RK3 loop → Post-processing; subsystem coupling table (CCZ4↔GRMHD↔M1↔AMR via GRMetric3) |
| 4 | Core Data Structures |
GridBlock memory layout (field-major flat allocation, stride indexing, ghost zones); GRMetric3 struct with all 14 fields; complete 22-variable CCZ4 table; complete 9-variable GRMHD table |
| 5 | Physics Formulations | CCZ4 equations with κ₁=0.02, κ₂=0, η=2.0; 1+log lapse evolution; Gamma-driver shift; trumpet gauge explanation; GRMHD Valencia formulation; constrained transport; EOS modes; M1 radiation status |
| 6 | Numerical Methods | 4th-order centered FD stencil; reconstruction methods table (PLM/PPM/MP5); HLLE vs. HLLD comparison; SSP-RK3 Shu-Osher stages; KO dissipation formula and σ=0.1 warning |
| 7 | Initial Data | Two-Punctures/Bowen-York with p_t=0.0840 for d=10M; Brill-Lindquist conformal factor; TOV solver equations; compatibility table (Sommerfeld + BL = incompatible) |
| 8 | AMR Engine | Berger-Oliger subcycling; refinement criteria YAML; prolongation (trilinear); restriction (volume-weighted); ghost zone filling order; dynamic regridding status (partially implemented) |
| 9 | Boundary Conditions | Sommerfeld 1/r BCs with gauge wave speed √2; copy BCs for BL data; domain size requirement (±48M minimum) |
| 10 | Diagnostics & GW Extraction | Ψ₄ via Newman-Penrose formalism; spherical harmonic decomposition; extraction radii table; constraint monitoring with health thresholds; AH finder expansion equation; phase classification (known limitation: time-based) |
| 11 | I/O & Checkpointing | HDF5 checkpoint format (group structure, shapes); checkpoint frequency recommendation; loadCheckpoint() stub status |
| 12 | MPI & OpenMP | Hilbert curve decomposition; non-blocking MPI_Isend/Irecv pattern; OpenMP collapse(3) RHS loop; KO single OMP region (H1 fix); loop order mandate (H3 fix) |
| 13 | Development Environment Setup | Platform support table; WSL note (Linux-native filesystem only); one-command bootstrap; manual dependency table with minimum versions |
| 14 | Build System | Build targets (release/debug/tests/format); CMake options table; dev_benchmark.py pre-PR requirement |
| 15 | Coding Standards & Style Guide | C++17 mandate; no exceptions policy; Logger class requirement; naming conventions table; header guidelines with forbidden patterns; physical unit documentation requirement; clang-format enforcement |
| 16 | Testing Philosophy & Workflow | Zero-tolerance policy; test categories (unit/integration/stability/validation); adding tests template (with convergence order test pattern); pre-PR checklist |
| 17 | Adding New Features / Physics Modules | RFC → branch → implement → public interface → tests → YAML → CHANGELOG → PR step-by-step; priority areas (2026 roadmap) |
| 18 | Parameter System (YAML) | Complete canonical params.yaml with all sections: simulation, domain, AMR, initial_data, ccz4, dissipation, time_integration, boundary, diagnostics, io |
| 19 | HPC & Performance Guidelines | Scaling targets table (128³→512³→B5_star); memory estimates table; SLURM job generation; Lustre I/O tuning; profiling tools (VTune/Score-P/Nsight) |
| 20 | Known Fixed Bugs | 10-row table of all bugs (C1, C3, H1, H2, H3, TOV, KO-σ, Sommerfeld+BL, Domain, alpha_center) with descriptions and fixes |
| 21 | Simulation Health & Debugging | 6-indicator health table; ASCII debugging flowchart (NaN → CFL; ‖H‖₂ growing → domain/ko_sigma/BCs; no merger → p_t=0; no trumpet → resolution); 4 common failure modes with exact descriptions |
| 22 | Scientific Validation & Publication Policy | 6-benchmark validation table with status; requirements for publication (5 items) |
| 23 | Collaboration & Community | Communication channels; contribution recognition table; partnerships sought |
| 24 | Roadmap (v0.7 → v1.0) | 4-row version table with deliverables; B5_star scaling path description |
| 25 | Flagship Scenario: B5_star | Full YAML excerpt; expected physics sequence (3 phases); NRCF/PRISM analytic pre-validation table |
| Appendix A | Physical Constants | Geometrized unit table; km→cm conversion warning |
| Appendix B | Key References | 10-entry bibliography |
- Section 10.4 (Phase Classification): States phases are time-based "a known limitation" but does not cross-reference this to the Known Limitations table in README or provide the planned fix version. Should explicitly state: "Planned fix: v0.7 — replace with BH separation-based criterion."
-
Section 21 (Debugging Flowchart): The
alpha_centerdiagnostic bug (reads from AMR level 0, not finest level) is listed in §20 (Known Fixed Bugs) as a known-but-unfixed bug, but the debugging flowchart in §21 does not warn the user about it. Anyone usingalpha_centeras a primary health indicator will be misled. -
Section 8 (AMR): States that dynamic regridding is "partially implemented" but does not give the exact current behavior: AMR block count is fixed at 4 throughout B2_eq runs because
regrid()does not evaluate gradient criteria at runtime. -
Section 11 (I/O):
loadCheckpoint()status says "stub exists but not implemented" — this could be more explicit about what happens if a user tries to use it (no-op, not an error, so restarts silently produce wrong physics). - Section 5 (Physics): M1 radiation module section correctly states "built but not wired into the main RK3 evolution loop" but does not explain what this means for production runs: radiation transport is completely inactive even if neutrino source terms are present in the matter.
Path: docs/user_guide/BENCHMARKS.md
Size: 313 lines, 10.9 KB | Audience: Scientists, institutional partners, reviewers
The primary numerical validation document. Contains every production run result from hardware to final ‖H‖₂ value, fully reproducible. This is the document to share with HPC centers and NR groups evaluating GRANITE.
| § | Title | Key Content |
|---|---|---|
| 1 | Hardware & Software Environment | Full hardware spec (i5-8400, 16GB DDR4, GTX 1050 Ti); OS (WSL2 Ubuntu 22.04); compiler (GCC 11.4, -O3 -march=native); OpenMPI 4.1.2; HDF5 1.12.1; OpenMP 6 threads |
| 2 | Benchmark Definitions | SP: single Schwarzschild BH with BL data; B2_eq: equal-mass BBH with Two-Punctures. Full YAML key-parameter excerpts for both |
| 3 | Single Puncture — Stability Results | 64³ full result table (‖H‖₂ 0→500M: 1.083e-2 → 1.277e-4, ×84.8 reduction, 0 NaN); 128³ result table (0→120M: 1.855e-2 → 1.039e-3, ×17.9, 0 NaN) |
| 4 | Binary Black Hole Inspiral — Results | 64³ BBH (t=0→500M, ‖H‖₂ peak 8.226e-4 → 1.341e-5, ×61.3 reduction, 98.9 min); 96³ BBH (‖H‖₂ peak 2.385e-3 → 3.538e-5, ×67.4, 496 min) |
| 5 | Constraint Norm Time Series | Full step-by-step tables for 64³ BBH (16 checkpoints, steps 2–320) and 96³ BBH (12 checkpoints, steps 2–480) including α_center, ‖H‖₂, AMR block count at each step |
| 6 | Resolution Convergence Analysis | Cross-resolution ‖H‖₂ comparison table; honest interpretation of 96³ showing higher final ‖H‖₂ than 64³ with correct physical explanation |
| 7 | Throughput & Wall-Time Scaling | Desktop scaling table (64³: 0.0843 M/s, 96³: 0.0168 M/s); HPC projection table (6→512 cores; H100 post-GPU porting) |
| 8 | Reproducing These Results | Exact commands for all 4 benchmark variants with notes on which params.yaml edits are needed |
-
No gauge_wave benchmark results. The
gauge_wavebenchmark exists inbenchmarks/gauge_wave/params.yamland is described in the CHANGELOG (v0.6.0) but has no corresponding results section here. Planned addition. - No formal convergence order table. The ×2 resolution ratio runs (64³ vs 128³ for SP, and 64³ vs 96³ for BBH) cannot be used to extract a convergence order because 96³/64³ = 1.5, not 2.0. The text acknowledges this and defers to v0.7. A note explicitly stating which resolution pairs are needed (64³, 128³, 256³ at identical domain and parameters) would help contributors plan this.
- No GW waveform section. Ψ₄ extraction infrastructure exists in the code but is not yet active in production runs. A placeholder section with "planned for v0.7" and the expected extraction radius table would make the document more forward-looking.
- α_center values are misleading. The benchmark tables show α_center ≈ 0.97 at t=500M for BBH runs. This is the value read from AMR level 0 (the coarsest level), not from the finest level near the puncture — a known diagnostic bug. The tables should include a footnote explaining this caveat.
- Hardware context →
docs/DEPLOYMENT_AND_PERFORMANCE.md - Reproducibility commands →
scripts/run_granite.py dev,python -m granite_analysis.cli.dev_benchmark - Constraint norm interpretation →
docs/diagnostic_handbook.md §4
Path: docs/theory/SCIENCE.md
Size: 333 lines, 15.9 KB | Audience: Astrophysicists, cosmologists, GW scientists evaluating GRANITE
The scientific context and physics justification document. Explains what GRANITE simulates, why it matters observationally and theoretically, and provides all governing equations with full mathematical notation. Does not discuss implementation — all implementation details are in DEVELOPER_GUIDE.md.
| § | Title | Key Content |
|---|---|---|
| 1 | The Scientific Problem | Multi-body astrophysics problem description; 4 coupled physics systems; existing code domain vs. GRANITE domain |
| 2 | Why Existing Codes Are Not Enough | 5-row code capability table (Einstein Toolkit, GRChombo, SpECTRE, AthenaK, GRANITE); the unification coupling problem explained |
| 3 | The B5_star Scenario | Pentagon configuration (5 × 10⁸ M☉ SMBHs at 1pc radius + 2 × 4300 M☉ stars); Phase I/II/III physical sequence; multi-messenger signatures table (GW/X-ray/radio/neutrino with detectors); computational requirements table (12 AMR levels, ~10¹⁰ cells, ~2 TB RAM, ~5×10⁶ CPU-hours) |
| 4 | Governing Equations | Full CCZ4 equations with LaTeX notation; 22-variable table; 1+log lapse and Gamma-driver equations; Valencia GRMHD conserved variables with full D, Sⱼ, τ, Bⁱ, Yₑ definitions; CT induction equation; M1 moment closure (Minerbo/Levermore Eddington tensor) |
| 5 | Numerical Approach | 4th-order FD stencil formula; KO dissipation formula with p=3; SSP-RK3 stages; reconstruction methods table |
| 6 | Multi-Messenger Observables | Ψ₄ spherical harmonic decomposition; fixed-frequency GW integration (Reisswig & Pollney 2011); AH expansion equation; horizon mass and spin formulas; Hamiltonian and momentum constraints with explicit L2 norm definition |
| 7 | Physical Scales & Unit System | Geometrized unit table (G=c=1); conversion table for M=10⁸ M☉ and M=30 M☉ cases |
| 8 | References | 14-entry bibliography with full journal citations |
- No "Final Parsec Problem" discussion. The document references SMBH mergers but does not explain the dynamical friction / stellar loss-cone / GW emission progression that drives SMBH pairs from parsec to sub-parsec separations. This would be important context for astronomers.
- LISA band discussion is incomplete. The multi-messenger table lists LISA as a GW detector but does not give the specific frequency band (10⁻⁴–10⁻¹ Hz) or signal-to-noise ratio estimates for the B5_star scenario.
- Neutrino microphysics motivation is absent. The M1 radiation module is described but there is no discussion of why neutrino emission matters physically (e.g., neutronization, lepton number transport, r-process nucleosynthesis seeds in BNS). Since GRANITE is designed for SMBH + stellar environments, this would benefit from a paragraph on stellar disruption neutrino emission.
- Equation numbering. The LaTeX equations have no numbering system, making it impossible to cross-reference them from other documents or the code. Adding equation labels (aligned with Alic 2012 or Baumgarte & Shapiro 2010 equation numbers) would greatly help code reviewers.
- Implementation of these equations →
docs/DEVELOPER_GUIDE.md §5 - Numerical accuracy results →
docs/BENCHMARKS.md - Code comparison details →
docs/COMPARISON.md
Path: docs/developer_guide/COMPARISON.md
Size: ~200 lines | Audience: Scientists evaluating whether to use GRANITE vs. existing codes
Source-cited, per-feature comparison of GRANITE against Einstein Toolkit, GRChombo, SpECTRE, and AthenaK. Every capability claim is backed by a citation to the relevant paper or code documentation.
- Extended version of the feature matrix from README (more rows, with citations)
- Per-code qualitative analysis of architecture philosophy
- Explanation of why no existing code handles the N>2 BH + GRMHD + M1 combination
- Discussion of GRANITE's unique Python telemetry as a differentiator
- The COMPARISON.md is referenced in README but its exact content was not fetched in this audit (the file exists at
docs/COMPARISON.md). Based on README's mention of "source-cited analysis," it should contain full bibliography entries for each code claim. - No version pinning. Comparisons should specify which version of each code was examined (e.g., "Einstein Toolkit release ET_2022_05"). Codes change; the comparison should be versioned.
Path: docs/user_guide/FAQ.md
Size: ~200 lines | Audience: New users, scientists with quick questions
Frequently Asked Questions covering science, engineering, HPC, and contribution topics. Written to be the first stop for users who have a specific question but don't know which document to consult.
Based on the most common issues observed in the development history, the following FAQs are likely missing or should be verified:
- "Why does my simulation crash at t ≈ 6.25M?" (domain size issue — the most common early-stage failure)
- "What does
α_center ≈ 0.97mean for my BBH run?" (AMR level 0 diagnostic limitation) - "My BBH runs to t=500M but I see no merger — is this a bug?" (zero tangential momentum issue)
- "What is
ko_sigmaand why is 0.1 the safe value?" - "Why can't I use Sommerfeld BCs with Brill-Lindquist initial data?"
- "What resolution do I need for a real inspiral simulation?"
- "Why is the M1 radiation module not active in production runs?"
Path: docs/user_guide/diagnostic_handbook.md
Size: 133 lines, 8.32 KB | Audience: Researchers running and debugging simulations
A guide to interpreting the live simulation telemetry output from granite_analysis.cli.sim_tracker and the C++ engine. Meant to be the "what does this number mean and should I panic?" reference for anyone watching a simulation run.
| § | Title | Key Content |
|---|---|---|
| 1 | Introduction: The Pulse of the Grid | Framing: three diagnostic pillars (gauge stability α, physical validity ‖H‖₂, NaN forensics); epistemological discipline concept |
| 2 | Anatomy of the GRANITE Live Dashboard | Full telemetry field table: step, t, α_center, ‖H‖₂, speed, ETA; "speed is a physical diagnostic" explanation; AMR overhead and computational density |
| 3 | Gauge Stability: The Story of the Lapse | Lapse lifecycle: Initial (α=1) → Collapse (α→0.003) → Trumpet Stability (α→0.3); contrast of SP5 (healthy) vs. DEV_LOG_2BH (α=2.19×10⁹, catastrophic) |
| 4 | The Hamiltonian Constraint ‖H‖₂ | 4-tier threshold table (HEALTHY <0.5, WARNING 0.5–2.0, CRITICAL >10, CRASH NaN/∞); constraint explosion forensic analysis |
| 5 | NaN Forensics: Tracking Numerical Infection | "Clean Summary Paradox" — when the summary says OK but the step log shows NaN; forensic checklist (check RHS, identify source variable, locate grid point) |
| 6 | AMR and Grid Health | Tracking spheres; CFL Guard Events as "smoking gun"; speed stall + ‖H‖₂ growth = AMR failure |
| 7 | The Health Check Checklist | 6-item binary pass/fail table |
| 8 | Closing | "The grid does not lie" — epistemological message |
⚠️ This document has the most gaps of any document in the repository. It was written based on early development logs (DEV_LOG_2BH_2 era crashes) and has not been updated to reflect the stable v0.6.5 behavior.
Critical missing content:
-
Current healthy output format. The document describes the
DEV_LOG_2BH_2crash (α=2.19×10⁹) as the comparison baseline, but this crash has been fixed. The document should show the current healthy B2_eq output format:step=80 t=5.0000M α_center=0.029 [RECOVER] ‖H‖₂=0.153 [OK] Phase: Lapse recovering — trumpet stable ✓ -
α_center AMR level 0 bug is not documented. The
alpha_centerfield reads from AMR level 0 (at r ≈ 0.65M), not from the finest AMR level near r→0. This means the values shown (e.g., 0.9675 for B2_eq at t=500M) are not the lapse at the puncture. This is the most important diagnostic caveat and is completely absent from this document. -
Trumpet not resolving at dx=0.75M is expected behavior. At 128³ resolution with domain ±48M, dx=0.75M near the puncture — the trumpet geometry is not resolved. The lapse never drops below 0.02 (the trumpet phase threshold). This is not a bug but the document provides no guidance on how to interpret it.
-
Secondary gauge collapse ≠ crash. When the lapse hits the floor (α → floor) with zero NaN events and active recovery, this is a gauge pathology, not a numerical crash. Simulations should not be stopped in this state. This is a documented principle in the project but absent from the handbook.
-
Phase labels are time-based, not separation-based. The phase labels "Early/Mid/Late Inspiral" appear in
granite_analysis.cli.sim_trackeroutput and are described nowhere in this document. Users may interpret them as physics-based when they are not. -
‖H‖₂ thresholds are obsolete. The document says CRITICAL > 10.0. But production runs show ‖H‖₂ starting at 1e-3 and decaying to 1e-5. The thresholds need to be re-calibrated in terms of the growth rate γ [M⁻¹] rather than absolute value, consistent with DEVELOPER_GUIDE §10.2.
-
No discussion of domain size effect. The document doesn't mention that small domains (< ±48M) cause constraint drift regardless of code correctness. This is the most common user failure mode.
- More accurate and complete debugging guide →
docs/DEVELOPER_GUIDE.md §21 - Correct health thresholds →
docs/DEVELOPER_GUIDE.md §10.2 - Known diagnostic limitations →
README.md §Known Limitations
Path: docs/developer_guide/v0.6.5_master_dictionary.md
Size: ~500 lines | Audience: Developers needing quick-lookup of any CLI flag, parameter, constant, or patch
The exhaustive technical reference for every configurable element of GRANITE v0.6.5. Functions as a man-page and forensic record combined. Referenced in the Home wiki page as "Technical Encyclopedia."
- Every CLI flag for
run_granite.pyandrun_granite_hpc.py - Every YAML parameter with type, default, units, valid range, and warnings
- Every C++ constant in
include/granite/constants.hpp - Every CMake option
- Stability Patch forensic records (cross-referencing CHANGELOG)
-
Not yet on the wiki. This document exists as a
docs/file but has not been promoted to a wiki page. It is the ideal candidate for theParameter-Referencewiki page. -
ko_sigma warning must be prominent. Based on the development history,
ko_sigma > 0.1is the single most dangerous parameter to misset. This must be marked with a prominent warning box.
Path: docs/getting_started/Installation.md
Size: ~300 lines | Audience: New users installing GRANITE for the first time
A-to-Z installation guide for all supported platforms. Provides step-by-step instructions for every dependency, build configuration, and verification step. Includes a Q&A troubleshooting section.
- WSL2 setup guide (recommended path)
- Ubuntu/Debian, Fedora/RHEL, macOS Homebrew, Windows native (vcpkg), Conda/Miniforge dependency installation
- Build verification (
health_check.py) - Common error messages and resolutions
- Deep technical Q&A for build failures
-
WSL filesystem path warning may need strengthening. The critical requirement to run from the Linux-native filesystem (not
/mnt/c/) should be in a high-visibility warning box. -
WSL2 RAM limit. WSL2 by default uses 50% of host RAM. For 128³ simulations (~2 GB), this may be insufficient on 8 GB machines. A
.wslconfigmemory tuning section would prevent confusing out-of-memory crashes.
Path: docs/user_guide/DEPLOYMENT_AND_PERFORMANCE.md
Size: ~200 lines | Audience: HPC administrators, power users
Advanced guide for maximizing GRANITE performance on desktop and HPC hardware. Covers OpenMP thread saturation, NUMA binding, memory checks, and job scheduler templates.
-
health_check.pyoutput interpretation - OpenMP thread count tuning (
OMP_NUM_THREADS,OMP_PROC_BIND) - Memory allocation pre-checks
- NUMA topology binding for multi-socket nodes
-
run_granite_hpc.pyfull flag reference
- No performance regression baseline. The document should include a table of expected throughput (M/s) for standard configurations so users can verify they are getting expected performance.
- GPU section is a placeholder. The document likely mentions GPU deployment (vast.ai H100) but cannot provide instructions until GPU kernels are implemented in v0.7.
Path: docs/internal/
Audience: Core development team only (not user-facing)
-
GRANITE_FULL_AUDIT.md— The exhaustive Phase 0–3 physics audit report listing all 14 findings (7 Critical, 4 Moderate, 3 HPC) with exact line references and mathematical derivations -
GRANITE_CLAUDE_TASK_CATALOG.md— Internal task tracking for AI-assisted development sessions -
NAN_DEBUG_BRIEF.md— Emergency debugging brief from Phase 6 crash investigation
- These documents are valuable engineering artifacts that should be preserved permanently. They are not user-facing but should be indexed here so contributors know they exist.
-
GRANITE_FULL_AUDIT.mdcontains the definitive mathematical proofs for each bug fix — this content should eventually be promoted into the DEVELOPER_GUIDE's §20 with full equation derivations.
Path: docs/theory/
Audience: Physicists who want derivations, not just results
Physics derivations and CCZ4 notation reference documents. Provides the mathematical background behind the equations implemented in the code.
- The exact contents of this directory were not fully catalogued in this audit. It should contain at minimum: CCZ4 conformal decomposition derivation, moving-puncture gauge derivation, GRMHD Valencia formulation derivation.
- If these files are stubs or placeholders, that should be noted clearly.
Path: docs/user_guide/
Audience: End users running simulations (not developers)
End-user tutorials and worked examples. Referenced in the README repository structure as containing tutorials.
- The
docs/user_guide/installation.rstfile is referenced in the CHANGELOG (v0.5.0) as having been updated. An.rstfile in a Python-style Sphinx documentation directory suggests a documentation generation pipeline that may not be configured. If Sphinx is not being used, these should be converted to Markdown. - The user guide should contain at minimum: "Running Your First Simulation," "Understanding Simulation Output," "Tuning Parameters for Your Hardware."
Path: .github/CONTRIBUTING.md
Size: ~100 lines | Audience: External contributors
Defines the contribution workflow, code review expectations, and community standards. GitHub displays this automatically when anyone opens a PR or issue.
- Contribution types (code, physics, validation, documentation, bug reports)
- Branch naming conventions
- PR checklist (tests, format, CHANGELOG)
- Physics contribution standards (paper citation required)
- Communication etiquette
- Should include the pre-PR checklist from DEVELOPER_GUIDE §16.4 in its entirety, so contributors don't need to find it in the developer guide.
Path: .github/CODE_OF_CONDUCT.md
Size: ~50 lines | Audience: All community members
Content: Contributor Covenant Code of Conduct v2.1. Standard community behavior standards.
Path: .github/SECURITY.md
Size: ~80 lines | Audience: Security researchers
- Supported versions matrix (main branch and 1.x receive security updates)
- Vulnerability reporting via GitHub Private Vulnerability Reporting or direct email
- Response timeline: 48h acknowledgement, 7 days status update, 30 days patch
- Security best practices for users (containerized execution)
- Scope definition (GRANITE codebase, CI/CD, containers)
Path: .github/workflows/ci.yml
Audience: CI/CD engineers
GitHub Actions workflow that runs on every push and PR to main. Ensures the test suite passes before any merge.
- Install dependencies (GCC, OpenMPI, HDF5, libomp-dev, yaml-cpp)
- CMake configure with
-DGRANITE_ENABLE_OPENMP=ON - Build Release target
- Run
ctest --output-on-failure(must show 105 tests passed, 2 skipped across 20 suites) - Run
cppcheckstatic analysis
Path: .github/workflows/codeql.yml
Audience: Security / static analysis
Content: CodeQL Advanced Security Analysis for C++ and Python. Runs on push, PR, and weekly cron. C/C++ build mode is manual, matching the CI build environment exactly.
Path: .github/PULL_REQUEST_TEMPLATE.md
Audience: Contributors opening PRs
Content: Standardized PR description template. Expected to include: change summary, physics motivation, tests added, CHANGELOG updated, benchmark results.
Path: .github/ISSUE_TEMPLATE/
Audience: Users reporting bugs or requesting features
Content: Issue templates for: Bug Report, Feature Request, Physics Question. The physics question template is particularly important for a scientific project.
Path: benchmarks/single_puncture/params.yaml
Audience: Developers validating the CCZ4 gauge and trumpet formation
The standard Schwarzschild stability benchmark. Standard first validation test for any NR code: a single black hole that should remain stable indefinitely.
- Brill-Lindquist initial data, M=1.0 at origin
- Domain: ±48M, copy boundary conditions
- nx=64 (standard) or nx=128 (high-resolution variant)
- ko_sigma=0.1, kappa1=0.02, eta=2.0
- t_final=500M (64³) or t_final=120M (128³)
- α collapses from 1.0 → ~0.003 (t < 5M)
- α recovers toward trumpet value (~0.28–0.35, only resolvable at dx < 0.5M)
- ‖H‖₂ decays monotonically after initial gauge adjustment
- Zero NaN events
-
No
t_finalornxswitch mechanism. Users must manually edit the YAML to switch between the 64³/500M and 128³/120M variants. A comment in the file should document both variants explicitly.
Path: benchmarks/B2_eq/params.yaml
Audience: Scientists validating BBH inspiral
The production equal-mass binary black hole benchmark. Tests the full CCZ4 + AMR pipeline under binary inspiral conditions with Two-Punctures initial data.
- Two-Punctures/Bowen-York: m₁=m₂=0.5, positions [±5, 0, 0], momentum [0, ±0.0840, 0]
- Domain: ±200M, Sommerfeld boundary conditions
- 4 AMR levels
- ko_sigma=0.1, kappa1=0.02, eta=2.0
- The tangential momentum
p_t = ±0.0840is the quasi-circular value for d=10M equal-mass BBH from PN approximations. Zero tangential momentum produces a head-on collision, not inspiral. - The AMR block count is frozen at 4 throughout the run (dynamic regridding limitation in v0.6.8).
- At 64³ base resolution, the finest dx=0.781M is insufficient to resolve the trumpet (expected — not a bug).
Path: benchmarks/gauge_wave/params.yaml
Audience: CCZ4 code validators
The "Apples-with-Apples" CCZ4 code validation test (Alcubierre et al. 2004). Sinusoidal lapse perturbation on a flat conformal metric. Hamiltonian constraint should remain ~0 to machine precision for all time.
- 64³ on [-0.5, 0.5]³ (dx=0.015625)
- α = 1 + 0.1·sin(2πx/1.0) as initial perturbation
- CFL=0.25, t_final=100.0, ko_sigma=0.1
- No production results documented in BENCHMARKS.md. This benchmark was added in v0.6.0 but no run results have been published. This is a significant gap — the gauge wave test is the most fundamental CCZ4 validation and should have published constraint norm plots.
Path: benchmarks/B5_star/B5_star.yaml
Audience: Future HPC production teams
The flagship scientific target configuration. Five 10⁸ M☉ SMBHs in a pentagon configuration with two 4300 M☉ stars. Full physics (CCZ4 + GRMHD + M1 + 12 AMR levels).
This configuration file exists and is syntactically valid, but cannot be run to completion at current hardware scale or with current code completeness (M1 not wired in, checkpoint-restart not implemented, ~2 TB RAM required). It serves as the north star specification document for all development decisions.
Path: benchmarks/scaling_tests/
Audience: HPC administrators
Content: SLURM and PBS/Torque job script templates for strong and weak scaling benchmarks. Used to generate job submissions for institutional cluster runs.
Path: benchmarks/validation_tests.yaml
Size: ~200 lines | Audience: CI/CD systems, validation engineers
Machine-readable validation test suite definition. Specifies pass/fail criteria for all physics validation benchmarks. Designed for automated CI integration.
- Spacetime: gauge_wave (4th-order convergence, amplitude drift <1e-6), robust_stability (noise growth <10× over 10⁵M), single_puncture (horizon mass drift <1e-4), binary_equal_mass (phase error <0.01 rad vs. SXS:BBH:0001)
- GRMHD: balsara_1 shock tube (L₁ error <1e-3), magnetic_rotor (∇·B < 1e-14), bh_torus (MRI growth rate within 5% of linear theory)
- Radiation: radiative_shock_tube (M1 vs. analytic L₂ error <2%), diffusion_limit (diffusion coefficient error <1%)
- A CI runner that parses this YAML and executes the tests automatically does not yet exist. The file is a specification, not an active pipeline.
- The SXS:BBH:0001 comparison (binary_equal_mass pass criterion: phase error <0.01 rad) cannot currently be executed because Ψ₄ extraction is not active in production runs.
Path: scripts/run_granite.py
Audience: All users
Unified CLI entry point for all development operations. Abstracts platform differences (WSL vs Linux vs macOS) and provides consistent subcommands.
| Subcommand | Action |
|---|---|
build |
CMake configure + build with -O3 (Release is default) |
build --build-type Debug |
CMake configure + build with ASan/UBSan |
test |
Run C++ test suite via CTest (--filter, --timeout) |
run --benchmark <name> |
Run a named benchmark from benchmarks/
|
dev |
Build → run → stream output to granite_analysis.cli.dev_benchmark live dashboard |
dev --watch |
Like dev, but also auto-rebuilds on src/ file changes |
test |
Run Python analysis package tests via pytest tests/python -v
|
docs |
Build Sphinx HTML documentation into docs/_build/html/. Accepts --open to launch in browser |
format |
Run clang-format on all C++ files |
clean |
Remove build/ directory |
Path: python/granite_analysis/cli/dev_benchmark.py
Install: pip install -e .[dev]
Size: ~300 lines | Audience: Developers before every commit or PR
⚠️ MIGRATION NOTE:scripts/dev_benchmark.pyhas been permanently removed. The logic is fully re-implemented in thegranite_analysispackage.
Forensic diagnostic runner. Executes the single-puncture benchmark and produces detailed per-step analysis of NaN propagation, constraint growth, and lapse behavior. Must complete in under 5 minutes on development hardware.
- Per-step NaN forensics: identifies which spacetime variable (by name, e.g., A_XX = Ã_xx) goes NaN first, its grid location (i,j,k), and propagation speed in cells/step
- Lapse phase classification: Initial collapse → Trumpet forming → Lapse recovering → CRASHED
- Hamiltonian constraint exponential growth rate γ [M⁻¹] fit
- Advection CFL monitoring with upwinding threshold warning
- Options:
--verbose,--timeout <seconds>,--json <file>,--csv <file>
# Integrated pipeline (recommended)
python3 scripts/run_granite.py dev
# Direct invocation on a log file or stdin
python3 -m granite_analysis.cli.dev_benchmark [logfile] [--json out.json] [--csv out.csv]Path: python/granite_analysis/cli/ (integrated into sim_tracker runner)
Install: pip install -e ./python[dev]
⚠️ MIGRATION NOTE:scripts/dev_stability_test.pyhas been permanently removed. Extended stability runs are now managed via thedevsubcommand ofrun_granite.pywith custom YAML configurations.
- α_center stabilizes between 0.1 and 0.4 after initial collapse
- ‖H‖₂ remains < 1.0 beyond the critical t=6.25M window
- Advection CFL remains bounded by the upwinding threshold
- Zero NaN events for the full run duration
# Run dev pipeline with custom t_final via params.yaml
python3 scripts/run_granite.py dev # live build + benchmark + analysisPath: scripts/health_check.py
Size: ~150 lines | Audience: All users before running any simulation
Pre-flight validator. Checks the build configuration and system environment before simulation launch. Must be run before every B2_eq or production simulation.
- Reads
.CMakeCache.txtand verifies Release build flags are present (-O3, -march=native) - Verifies OpenMP is enabled and reports max thread count
- Checks available RAM
- Verifies HDF5 parallel mode
- Reports warning if build was not compiled in Release mode
python3 scripts/health_check.pyPath: python/granite_analysis/cli/sim_tracker.py
Install: pip install -e .[dev]
Size: ~400 lines | Audience: Simulators monitoring live runs
⚠️ MIGRATION NOTE:scripts/sim_tracker.pyhas been permanently removed. The logic is fully re-implemented in thegranite_analysispackage.
Context-aware live dashboard. Wraps the GRANITE binary and provides real-time parsed telemetry with phase classification, exponential growth rate fitting, NaN forensics, and post-run Matplotlib summaries.
- Append-only output (no cursor resets — preserves terminal history)
- Real-time phase classification (Early/Mid/Late Inspiral — note: time-based, not separation-based)
- Exponential constraint growth rate (γ) calculation
- "Zombie State" detector (frozen physics detection)
- 8-layer stability summary guard (cannot falsely report "No catastrophic events")
- Crash-proof float casting via
safe_float()(handles NaN, inf, partial log lines) - Automated Matplotlib constraint and phase summary on run completion
- JSON and CSV telemetry export
# Live log-file mode
python3 -m granite_analysis.cli.sim_tracker run.log
# Integrated pipeline (build + run + track)
python3 scripts/run_granite.py dev
# Export telemetry
python3 -m granite_analysis.cli.sim_tracker run.log --json telemetry.json --csv telemetry.csvPhase labels are time-based, not based on physical BH separation. "Early Inspiral" continues throughout t=500M even if the BBHs are not inspiral-ing (e.g., with zero tangential momentum). Do not use phase labels as physics diagnostics.
Path: scripts/run_granite_hpc.py
Size: ~200 lines | Audience: HPC users
HPC launch wrapper with NUMA binding, MPI rank specification, and AMR telemetry file output. Generates SLURM-compatible job scripts.
python3 scripts/run_granite_hpc.py build/bin/granite_main benchmarks/B2_eq/params.yaml \
--omp-threads 32 # OpenMP threads per MPI rank
--mpi-ranks 128 # Total MPI ranks
--disable-numa-bind # For single-socket systems
--amr-telemetry-file /scratch/$USER/amr.jsonlPath: python/granite_analysis/
Install: pip install -e ./python[dev] (from repository root)
Audience: Post-processing scientists and all developers (required for telemetry CLI)
| Module | Purpose | Key Functions |
|---|---|---|
gw.py |
GW strain and Ψ₄ post-processing |
Psi4Analysis.extract_strain(), compute_radiated_momentum(), spin_weighted_spherical_harmonic() (all modes, including ℓ=3,4 via Wigner d-matrices) |
| HDF5 reader | Read GRANITE checkpoint files | Parse /level_N/ccz4_vars and /level_N/grmhd_vars arrays |
constraint_plot.py |
Constraint norm time series visualization |
plot_ham_constraint(log_file), export to PDF/PNG |
| Analysis utilities | General post-processing helpers | Grid interpolation, unit conversion |
- The
validate.pyscript referenced in DEVELOPER_GUIDE §16.2 (python3 python/granite_analysis/validate.py) does not appear to be implemented yet. - No tutorial notebook (Jupyter) showing how to load a checkpoint and plot constraint norms.
Path: containers/Dockerfile
Audience: Docker users, CI systems
Content: Docker multi-stage image based on Ubuntu 22.04 LTS. Installs all dependencies (GCC, OpenMPI, HDF5, yaml-cpp), copies source, and builds GRANITE in Release mode. Provides a reproducible, containerized simulation environment.
Path: containers/granite.def
Audience: HPC users with Singularity/Apptainer
Content: Singularity/Apptainer definition file for cluster environments where Docker is unavailable due to security restrictions. Essential for running on NERSC, ARCHER2, Jülich, and other Tier-0/1 systems.
Path: viz/README.md
Audience: Post-processing scientists
Content: Documents the planned visualization scripts (plot_constraints.py, plot_gw.py, plot_amr_hierarchy.py) and their expected usage. These scripts are planned for v0.7 and are documented here as specifications.
Path: include/granite/
Audience: C++ contributors and module developers
| Subdirectory | Key Headers | Purpose |
|---|---|---|
core/ |
grid_block.hpp, types.hpp, constants.hpp
|
GridBlock data structure, type aliases, physical constants |
spacetime/ |
ccz4.hpp, gr_metric3.hpp, constraints.hpp
|
CCZ4 evolution interface; GRMetric3 (the CCZ4↔GRMHD interface) |
grmhd/ |
grmhd.hpp, tabulated_eos.hpp
|
GRMHD Valencia solver; EOS base class and tabulated implementation |
amr/ |
amr.hpp |
AMR hierarchy interface, Berger-Oliger subcycling |
initial_data/ |
initial_data.hpp |
Brill-Lindquist, Bowen-York, Two-Punctures, TOV solver interfaces |
io/ |
hdf5_io.hpp |
Parallel HDF5 checkpoint streaming |
postprocess/ |
psi4_extraction.hpp, ah_finder.hpp
|
Ψ₄ GW extraction; apparent horizon finder |
radiation/ |
m1_closure.hpp, leakage.hpp
|
M1 moment closure and neutrino leakage (built, not wired into RK3) |
Critical rule: GRMetric3 is the only permitted interface between the spacetime and GRMHD modules. Direct access to CCZ4 internal variables from GRMHD code is forbidden.
Path: src/
Audience: Physics developers, performance engineers
| Subdirectory | Key Files | Lines | Physics |
|---|---|---|---|
| (root) | main.cpp |
~68 | Thin entry point: MPI/PETSc init → parseArgs → setup → run → teardown |
core/ |
cli_parser.cpp, simulation_setup.cpp, evolution_loop.cpp, grid.cpp
|
~900 | CLI, YAML loading, SSP-RK3 loop, Sommerfeld BCs, floors, diagnostics |
spacetime/ |
ccz4.cpp |
~1158 | CCZ4 RHS (OpenMP), 1+log slicing, Gamma-driver, KO dissipation |
grmhd/ |
grmhd.cpp, hlld.cpp, tabulated_eos.cpp
|
~2200 | MP5/PPM/PLM, HLLE/HLLD, C2P Newton-Raphson, Constrained Transport, tabulated EOS |
radiation/ |
m1.cpp |
~416 | M1 moment equations (grey), Eddington closure [built, not wired into RK3] |
neutrino/ |
neutrino.cpp |
~411 | Hybrid leakage-M1: URCA, pair annihilation, optical depth [built, not wired] |
amr/ |
amr.cpp |
~722 | Berger-Oliger AMR, trilinear prolongation, volume-weighted restriction |
initial_data/ |
initial_data.cpp |
~957 | All initial data generators (BL, BY, Two-Punctures, TOV, gauge wave) |
horizon/ |
horizon_finder.cpp |
~565 | Apparent horizon finder: null expansion θ⁺, flow iteration |
postprocess/ |
postprocess.cpp |
~649 | Ψ₄ GW extraction, NP tetrad, spin-weighted harmonics, strain integration |
io/ |
hdf5_io.cpp |
~497 | Parallel HDF5 I/O: grid snapshots, checkpoint write/read |
Path: tests/
Size: 107 tests (105 passed, 2 skipped) | Audience: All contributors (must pass before every PR)
| amr/test_amr_basic.cpp | AMRSmokeTest | 5 | AMR hierarchy init, block count, subcycling smoke |
| core/test_grid.cpp | GridBlockTest, GridBlockBufferTest, GridBlockFlatLayoutTest | 22 | GridBlock memory layout, flat allocation, buffer packing, MPI neighbor metadata |
| core/test_types.cpp | TypesTest | 7 | NUM_HYDRO_VARS=9, NUM_SPACETIME_VARS=22, type sizes |
| spacetime/test_ccz4_flat.cpp | CCZ4FlatTest | 10 | Flat spacetime RHS=0; KO dissipation zero for constants, bounded for gauge wave; selective advection near/far puncture |
| spacetime/test_gauge_wave.cpp | GaugeWaveTest | 4 | CCZ4 gauge wave convergence; constraint residuals |
| initial_data/test_brill_lindquist.cpp | BrillLindquistTest | 5 | Conformal factor positivity, ψ→1 at infinity, constraint satisfaction |
| initial_data/test_polytrope.cpp | PolytropeTest | 7 | TOV solver convergence, M≈1.4 M☉, R≈10 km |
| horizon/test_schwarzschild_horizon.cpp | SchwarzschildHorizonTest | 3 | AH finder on Schwarzschild slicing |
| io/test_hdf5_roundtrip.cpp | HDF5RoundtripTest | 3 | HDF5 write + read roundtrip for all variable groups |
| radiation/test_m1_diffusion.cpp | M1SmokeTest | 4 | M1 diffusion limit, opacity scaling |
| grmhd/test_grmhd_gr.cpp | GRMHDGRTest | 5 | HLLE reduces to SR for flat metric; lapse scaling (C3 fix); MP5 accuracy |
| grmhd/test_hlld_ct.cpp | HLLDTest, CTTest | 7 | HLLD Sod/Brio-Wu/contact/symmetry; CT ∇·B preservation |
| grmhd/test_ppm.cpp | PPMTest | 5 | PPM parabolic recovery, contact sharpness, monotonicity |
| grmhd/test_tabulated_eos.cpp | IdealGasLimitTest, ThermodynamicsTest, InterpolationTest, BetaEquilibriumTest | 20 | Full tabulated EOS: thermodynamics, interpolation, NR temperature inversion, beta equilibrium |
# All tests (recommended)
cd build && ctest --output-on-failure
# Python analysis suite
python3 -m pytest tests/python -v
# Specific C++ suite
./build/bin/granite_tests --gtest_filter="GRMHDGRTest.*"
# Expected output
[ PASSED ] 105 tests.
[ SKIPPED ] 2 tests.Based on the full audit of all repository documents, the following content is confirmed absent or requires significant update:
| Missing Content | Priority | Notes |
|---|---|---|
gauge_wave benchmark results in BENCHMARKS.md
|
HIGH | Configuration exists; run results not yet published |
GW waveform section in BENCHMARKS.md
|
MEDIUM | Infrastructure exists; Ψ₄ not active in production runs yet |
Updated diagnostic_handbook.md with v0.6.7 behavior |
HIGH | Current version partially reflects pre-fix era; α_center AMR bug warning absent |
Jupyter notebook tutorial in python/granite_analysis/
|
MEDIUM | No worked example for loading checkpoints and plotting constraint norms |
CI runner for validation_tests.yaml |
MEDIUM | The YAML spec exists; no automated test executor yet |
validate.py script |
MEDIUM | Referenced in DEVELOPER_GUIDE but not implemented |
| Previously Missing | Resolved In | Notes |
|---|---|---|
[Unreleased] section in CHANGELOG.md
|
v0.6.7.2 | Now tracks v0.7 planned items |
| Wiki page: Parameter-Reference | v0.6.7.1 |
Parameter-Reference.md live on wiki |
| Wiki page: Architecture-Overview | v0.6.7.1 |
Architecture-Overview.md live on wiki |
| Wiki page: Physics-Formulations | v0.6.7.1 |
Physics-Formulations.md live on wiki |
run_granite.py docs subcommand |
v0.6.7.2 | Sphinx build + --open flag implemented |
python/requirements.txt |
v0.6.7.2 | Flat dependency file for CI/Docker environments |
25 broken relative links in docs/
|
v0.6.7.2 | Full audit and correction pass completed |
| C++ engine banner version mismatch | v0.6.7.2 |
src/core/cli_parser.cpp banner corrected to 0.6.8
|
Raw FileNotFoundError traceback in CLI |
v0.6.7.2 | All three callsites hardened with clean sys.exit() messages |
| Document | Issue | Action Required |
|---|---|---|
docs/user_guide/diagnostic_handbook.md |
Reflects pre-v0.6.8 crash behavior; missing α_center AMR level 0 bug warning; missing healthy B2_eq output format |
Full rewrite of §2–5 |
docs/developer_guide/DEVELOPER_GUIDE.md §21 |
Debugging flowchart does not mention α_center AMR level 0 limitation |
Add warning to debugging flowchart |
docs/user_guide/BENCHMARKS.md §5 |
α_center values in tables read from AMR level 0 (coarse) — misleading for puncture physics |
Add footnote to all tables |
-
README.md— Quick Start Guide, benchmark tables, roadmap, known limitations register -
CHANGELOG.md— v0.6.7.2 entry current; Unreleased section tracking v0.7 items -
docs/developer_guide/DEVELOPER_GUIDE.md— architecture, physics, numerical methods, coding standards -
docs/user_guide/BENCHMARKS.md— all production run numbers reproducible -
docs/theory/SCIENCE.md— governing equations and B5_star scenario -
docs/developer_guide/COMPARISON.md— feature matrix with citations -
docs/getting_started/Installation.md— all platform install paths -
.github/CONTRIBUTING.md,CODE_OF_CONDUCT.md,SECURITY.md - All benchmark YAML configurations (
B2_eq,single_puncture,gauge_wave,B5_star) benchmarks/validation_tests.yaml- All Python orchestration scripts (
run_granite.py,run_granite_hpc.py,health_check.py) - All
granite_analysispackage modules (cli/sim_tracker.py,cli/dev_benchmark.py,core/,utils/) - All 107 C++ unit tests (105 passed, 2 skipped, 20 suites)
Use this table to find the right document for any topic:
| Topic | Primary Document | Secondary Document |
|---|---|---|
| Quick start | README.md |
docs/getting_started/Installation.md |
| Installation | docs/getting_started/Installation.md |
README.md §Quick Start |
| Architecture overview | docs/developer_guide/DEVELOPER_GUIDE.md §3 |
README.md §Repository Structure |
| CCZ4 physics equations | docs/theory/SCIENCE.md §4.1 |
docs/developer_guide/DEVELOPER_GUIDE.md §5.1 |
| GRMHD Valencia equations | docs/theory/SCIENCE.md §4.3 |
docs/developer_guide/DEVELOPER_GUIDE.md §5.3 |
| All 22 CCZ4 variables | docs/developer_guide/DEVELOPER_GUIDE.md §4.3 |
docs/theory/SCIENCE.md §4.1 |
| All 9 GRMHD variables | docs/developer_guide/DEVELOPER_GUIDE.md §4.4 |
docs/theory/SCIENCE.md §4.3 |
| GridBlock memory layout | docs/developer_guide/DEVELOPER_GUIDE.md §4.1 |
CHANGELOG.md Phase 5 H2 |
| GRMetric3 interface | docs/developer_guide/DEVELOPER_GUIDE.md §4.2 |
include/granite/grmhd/grmhd.hpp |
| KO dissipation σ=0.1 rule | docs/developer_guide/DEVELOPER_GUIDE.md §6.5 |
docs/theory/SCIENCE.md §5.2, CHANGELOG.md Phase 4D H1
|
| AMR engine design | docs/developer_guide/DEVELOPER_GUIDE.md §8 |
CHANGELOG.md v0.6.0 |
| Bowen-York p_t=0.0840 | docs/developer_guide/DEVELOPER_GUIDE.md §7.1 |
benchmarks/B2_eq/params.yaml |
| Sommerfeld vs. copy BCs | docs/developer_guide/DEVELOPER_GUIDE.md §9 |
CHANGELOG.md Phase 6 |
| GW extraction Ψ₄ | docs/developer_guide/DEVELOPER_GUIDE.md §10.1 |
docs/theory/SCIENCE.md §6.1 |
| Constraint monitoring | docs/developer_guide/DEVELOPER_GUIDE.md §10.2 |
docs/user_guide/diagnostic_handbook.md §4 |
| NaN forensics | docs/developer_guide/DEVELOPER_GUIDE.md §21.2 |
docs/user_guide/diagnostic_handbook.md §5 |
| Known fixed bugs | docs/developer_guide/DEVELOPER_GUIDE.md §20 |
CHANGELOG.md (all phases) |
| Simulation debugging | docs/developer_guide/DEVELOPER_GUIDE.md §21 |
docs/user_guide/diagnostic_handbook.md |
| All YAML parameters | docs/developer_guide/v0.6.5_master_dictionary.md |
docs/developer_guide/DEVELOPER_GUIDE.md §18 |
| Benchmark results | docs/user_guide/BENCHMARKS.md |
README.md §Benchmark Results |
| Code comparison | docs/developer_guide/COMPARISON.md |
README.md §How GRANITE Compares |
| B5_star scenario | docs/theory/SCIENCE.md §3 |
docs/developer_guide/DEVELOPER_GUIDE.md §25 |
| Physical unit system | docs/theory/SCIENCE.md §7 |
docs/developer_guide/DEVELOPER_GUIDE.md Appendix A |
| Key references | docs/theory/SCIENCE.md §8 |
docs/developer_guide/DEVELOPER_GUIDE.md Appendix B |
| HPC deployment | docs/user_guide/DEPLOYMENT_AND_PERFORMANCE.md |
docs/developer_guide/DEVELOPER_GUIDE.md §19 |
| Performance profiling | docs/developer_guide/DEVELOPER_GUIDE.md §19.5 |
docs/user_guide/DEPLOYMENT_AND_PERFORMANCE.md |
| Contributing workflow | .github/CONTRIBUTING.md |
docs/developer_guide/DEVELOPER_GUIDE.md §17 |
| Testing | docs/developer_guide/DEVELOPER_GUIDE.md §16 |
tests/ |
| Pre-PR checklist | docs/developer_guide/DEVELOPER_GUIDE.md §16.4 |
.github/CONTRIBUTING.md |
| Adding features | docs/developer_guide/DEVELOPER_GUIDE.md §17 |
.github/CONTRIBUTING.md |
| Citing GRANITE | README.md §Citing GRANITE |
CITATION.cff |
| Roadmap | README.md §Roadmap |
docs/developer_guide/DEVELOPER_GUIDE.md §24 |
| Known limitations | README.md §Known Limitations |
docs/developer_guide/DEVELOPER_GUIDE.md §20 |
| Versioning policy | README.md §Versioning Policy |
CHANGELOG.md |
| Post-processing | python/granite_analysis/ |
viz/README.md |
| Live telemetry | granite_analysis.cli.sim_tracker |
docs/user_guide/diagnostic_handbook.md |
| Container deployment | containers/Dockerfile |
containers/granite.def |
GRANITE Documentation Index — v0.6.8 — May 9, 2026
Maintained by LiranOG. Update this page whenever a new document is added or an existing document has its scope changed.
v0.6.8 · Repository · Issues
- 🩺 Simulation Health & Debugging
- 📊 Benchmarks & Validation
- 🗂️ AMR Design
- 🖥️ HPC Deployment
- 🌀 VORTEX Engine
"Simulate the unimaginable."