Generate overwrite parameter grid (generalized)

[GernotMaier]

This is based on #2248 with the following changes are applied:

• call python modules directly instead of using workflow approach using run-application
• remove hardwired threshold and energy ranges and allow to set it on the command line
• streamline code to minimize dublications
• remove simtools-generate-parameter-scan-grid (not needed anymore, as we call modules directly)

[GernotMaier]

@EshitaJoshi - I am trying to understand the default of 1. here. We need to be better what nsb_rate means - sometimes it is used as multiplier to dark, but we also have a simulation model parameter which gives the NSB rate per pixel. I am not sure what this one is.

job_grid_io.py:

def _deserialize_job_row(serialized_row):
    """Deserialize one stored row to the in-memory job-row schema."""
    job_row = {
        "primary": serialized_row["primary"],
        "cores_per_shower": int(serialized_row["cores_per_shower"]),
        "showers_per_run": int(serialized_row["showers_per_run"]),
        "nsb_rate": float(serialized_row.get("nsb_rate", 1.0)),
        "model_version": serialized_row["model_version"],
        "array_layout_name": serialized_row["array_layout_name"],
        "corsika_le_interaction": serialized_row["corsika_le_interaction"],
        "corsika_he_interaction": serialized_row["corsika_he_interaction"],
        "run_number": int(serialized_row["run_number"]),
    }

[GernotMaier]

@EshitaJoshi - this is a copy of your branch with some simplifications applied. Feel free to change the base branch for this merge request to see the changes.

I've checked the output grid, but didn't text the submission with HT Condor etc. Please do this testing.

[Copilot]

Pull request overview

Copilot reviewed 24 out of 24 changed files in this pull request and generated 7 comments.

[EshitaJoshi]

I think the flag needs to change to --pack_for_grid_register and then it works for htcondor

[GernotMaier]

pack_for_grid_register changed to grid_output_path (is clearer). Is there still somewhere pack_for_grid_register used?

[EshitaJoshi]

Seems to be used still in simulate-prod as I get this error when I try to submit the .condor file: simulate_prod: error: unrecognized arguments: --grid_output_path ./simtools-output/default

[EshitaJoshi]

No wait, i guess i need a new apptainer image?

[EshitaJoshi]

Thanks a lot for the generalisation! This looks good to me - I also tested it on htcondor and it all works as expected.

[ctao-sonarqube]

Quality Gate passed

Issues
0 New issues
0 Fixed issues
0 Accepted issues

Measures
0 Security Hotspots
97.6% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube