Skip to content

v0.34.0

Choose a tag to compare

View all tags
@GernotMaier GernotMaier released this 15 Jul 12:39
· 383 commits to main since this release
v0.34.0
f2f68e0
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

This release of simtools, the simulation toolkit for the CTA Observatory, includes the following main changes:

API Changes

  • Changed simtools-simulate-prod grid-registration output handling.

    The command-line option was renamed from --pack_for_grid_register to
    --grid_output_path.

    When --grid_output_path is configured, simulation products are now written as
    plain files into that directory instead of creating a registration tarball.
    This includes copied sim_telarray logs, CORSIKA logs, histogram (.hdata)
    files, existing model archives, moved event output files, and a gzipped
    simtools.log.gz. (#2320)

Bugfixes

  • Allow integration tests to run from working directories containing a pytest configuration
    (see simpipe issue #34). (#2312)

  • Fixes simtools-simulate-prod job-grid precedence by rejecting ambiguous mixes of --job_grid_file/--job_grid_row with explicit production parameters.

    • Job-grid row resolution now happens inside simulate_prod before settings are loaded; the generic post-load override path was removed.
    • Added parser/source tracking so CLI/YAML conflicts fail clearly while argparse defaults remain valid.
    • Updated tests, docs, and the integration config to cover direct single-row job-grid execution.

    (#2328)

  • Resolve reading of ecsv-style tables for simtools-validate-cumulative-psf. (#2342)

  • Fixed schema validation so dimensionless integer columns in generated job grids stay integer-typed instead of being written to ECSV as floats. (#2343)

Documentation

  • Improve documentation on Apptainer ORAS image tags (suffix -apptainer). (#2317)
  • Add dedicated documentation pages on unit, integration and science tests plus a description on how to generate test resources. (#2338)

New Features

  • Add application to generate and validate integration test resources: simtools-resources-test-generate.
    Used for simtools-tests test generation. (#2279)

  • Allow to run integration and unit tests with resources from a configurable path (--test_resources_path).
    Example:

     pytest -n auto --no-cov --model_version=7.0.0 \\
     --test-resources-path=../simtools-tests/simtools-tests/v0.33.0/integration_tests \\
     tests/integration_tests
 ``` ([#2283](https://github.com/gammasim/simtools/pull/2283))

  • Add generic runtime environment file option to simtools-run-application and simtools-resources-test-generate.

    Use --runtime_environment_file pointing to e.g.

    runtime_environment:
  container_engine: podman
  image: ghcr.io/gammasim/simtools-prod:20260622-v78010-v2025-11-30-rc-generic
  network: simtools-mongo-network
  environment_file: .env
  options:
    - "--arch amd64"
    - "-v /<whatever path>/simpipe:/workdir/external/simpipe:ro"

    (#2284)

  • Add support for running simtools-simulate-prod with parameters sourced directly from a production job grid ECSV file.
    Allows to use --job_grid_file / --job_grid_row to select a specific row from a job grid file to run a simulation with the parameters defined in that row. (#2291)

  • Added backend-neutral parameter-scan grids for NSB and proton bias-curve simulations, with optional scan fields supported by the HTCondor submission generator. (#2298)

  • Change simulation production grid from RA/Dec based to HA/Dec based.
    Simplify production grid table with less columns and more descriptive column names.
    Update sim_telarray metadata to include array Az/Ze and HA/Dec information. (#2313)

  • Improve simtools-plot-corsika-limits:

    • correctly handle viewcone limits for cases of point-like simulations
    • improve plotting by adding projections of distance and energy axes to 2D distribution panels
    • set consistent limits to the axes when plotting: always use the broad-range simulation limits

    (#2319)

  • Add functionality to simtools-resources-test-generate to run a single test generation config file using the --config_file option.

    Example:

     simtools-resources-test-generate --test_directory . --simtools_version v0.34.0 --overwrite_collection_files \
         --config_file simtools-tests/v0.34.0/integration_tests/config_files/generate_array_config_alpha_north_5.0.0.config.yml
 ``` ([#2321](https://github.com/gammasim/simtools/pull/2321))

  • Enable --runtime_environment_file setting of run time environment (e.g., a container environment) with correct path settings for test resource generation and generic application running. (#2323)

  • Introduce schema-validation for sim_telarray metaparameters.
    To export the full metadata description including all model parameters, use application simtools-export-sim-telarray-metadata-schema. (#2324)

  • Add schema URL to metadata of job grid table. Add functionality to simtools-validate-file-using-schema to read this schema from the header file. (#2327)

  • Generalize the creation and use of trigger-event histograms with several major improvements:

    Add a generic application, simtools-write-trigger-histograms, to build and write trigger-event
    histograms from reduced event-data files. The following applications were updated to use these
    generic histograms:

    • simtools-compare-productions
    • simtools-production-derive-corsika-limits
    • simtools-plot-simulated-event-distributions
    • simtools-production-derive-monte-carlo-statistics

    Improve the estimation of required Monte Carlo statistics in
    simtools-production-derive-monte-carlo-statistics by using the generic histograms and by
    providing a more robust estimate of the relative statistical uncertainty.

    Improve file-type related utilities:

    • add a utility application, simtools-inspect-file, to print the main structure and metadata of a
      simtools-generated file.
    • add the new modules simtools.io.file_type and simtools.io.file_inspector with helper
      functions to identify and inspect files.

    (#2329)

  • Calculate mean, max, and stdev of shower-reuse as function of energy, core, and angular distance:

    • filled and saved with simtools-write-trigger-histograms
    • plotted with simtools-plot-simtel-events

    (#2331)

Maintenance

  • Improve robustness when writing reduced event-data HDF5 files with simtools-write-reduced-event-lists. (#2296)

  • Increase performance and efficiency of simtools-write-reduced-event-lists. (#2310)

  • Add flake8-cognitive-complexity to pre-commit stage. (#2311)

  • Improve efficiency and memory footprint for reading of reduced event data and histogram filling
    for simtools-production-derive-corsika-limits. (#2316)

  • Simplify naming of sim_telarray configuration files written to disk for simulation production:

    • telescope configuration files are now of type CTAO-LSTS-01.cfg, CTAO-SSTS-25.cfg
    • array configuration files are of type CTAO-South-beta.cfg
    • raises an exception if existing configuration files are overwritten by an application.
    • add a run-number level to the model sub directory to avoid overwriting for applications using several models.

    (#2318)

  • Review and improve pylint options with less disabled messages. (#2330)

  • Major refactor of application documentation generation:

    • read command line parameters from their definition in the configuration module (instead of repeating the help messages in the docstring)
    • allow to read the integration test configuration file and use them as use-case examples in the application documentation
      (#2333)

  • Improve guidance for coding agents and generalize (so they can now be used with Copilot, Claude, and Codex):

    • add a generic AGENTS.md (replaces .github/copilot-instructions.md)
    • move skills to .agents/skills and add a documentation skill.

    (#2337)

  • Change trigger for simtools-prod and simtools-dev container generation. Don't push when running on main. Push for tags of type vXX.YY.ZZ-rc (release candidates). (#2339)

  • Update of tests/resources with re-generated files. General update to model version v7.0.0. (#2340)

Assets 2
Loading