Add configurable detector version support for datasets #11
+210
−48
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
wdconinc
reviewed
Nov 13, 2025
.gitlab-ci.yml
Outdated
| DETECTOR_CONFIG: "epic_craterlake" | ||
| DETECTOR_VERSION: "main" | ||
|
|
||
| image: eicweb.phy.anl.gov:4567/containers/eic_container/eic_xl:${IMAGE_TAG} |
Contributor
There was a problem hiding this comment.
Can this run in eic_ci?
Suggested change
| image: eicweb.phy.anl.gov:4567/containers/eic_container/eic_xl:${IMAGE_TAG} | |
| image: eicweb.phy.anl.gov:4567/containers/eic_container/eic_ci:${IMAGE_TAG} |
Contributor
Author
There was a problem hiding this comment.
good point. i can make the container a configurable variable too and try it.
- Add DETECTOR_CONFIG and DETECTOR_VERSION environment variables - Set global defaults to epic_craterlake and main in .gitlab-ci.yml - Configure eAu BeAGLE samples to use epic_craterlake_without_zdc - Split SIDIS jobs into eAu and ep categories - Split DIS eA jobs into eAu and other nuclei categories - Export detector variables in determine_timing.sh before run.sh calls 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
- Add workflow_dispatch inputs for image_tag, detector_config, and detector_version - Make container image tag configurable via IMAGE_TAG variable in GitLab CI - Pass workflow inputs as variables to GitLab pipeline trigger - Set sensible defaults for all configurable variables This allows manual workflow triggers with custom container images and detector configurations while maintaining backward compatibility. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
- Add workflow_dispatch input for container_name (default: eic_xl)
- Add CONTAINER_NAME variable to GitLab CI (default: eic_xl)
- Update image path to use ${CONTAINER_NAME} variable
- Pass container_name from workflow to GitLab pipeline
This allows switching between different container images like
eic_xl and eic_ci via workflow dispatch.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
- Add RESULTS_BASE variable combining all 4 config variables
- Update all artifact paths to use ${RESULTS_BASE}
- New structure: results/${CONTAINER_NAME}/${IMAGE_TAG}/${DETECTOR_CONFIG}/${DETECTOR_VERSION}/datasets/
This allows multiple configurations to be run in parallel with
separate artifact directories for each combination of settings.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Update default CONTAINER_NAME from eic_xl to eic_ci in both GitLab CI variables and GitHub workflow dispatch inputs. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
Add echo statements to .timings template to debug whether DETECTOR_CONFIG and DETECTOR_VERSION are being set correctly from job variables. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
Move RESULTS_BASE calculation from global variables to runtime in each job template's script section. This ensures RESULTS_BASE is calculated using the actual DETECTOR_CONFIG and other variables after job-specific overrides are applied. Previously, RESULTS_BASE was evaluated once globally with the default DETECTOR_CONFIG value and never recalculated when jobs overrode DETECTOR_CONFIG. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
- Add debug echo statements to show DETECTOR_CONFIG, DETECTOR_VERSION, and RESULTS_BASE - Update logfile path to use RESULTS_BASE instead of hardcoded results/ 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
…variables - Add DETECTOR_CONFIG_OVERRIDE and DETECTOR_VERSION_OVERRIDE support to .nevents template - Use YAML anchors to define variables once and reuse in both nevents and timings jobs - Ensures both job types write to and read from the same RESULTS_BASE path 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
rahmans1
force-pushed
the
add-detector-config-support
branch
from
b785a77 to
dcdc451
Compare
Add NEVENTS_PER_TEST as a global variable (default: 100) with override support via NEVENTS_PER_TEST_OVERRIDE. Also re-add EBEAM and PBEAM global variables with override support that were lost during rebase. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
…imings - Add INFO logging to .nevents and .timings templates showing all variables - Replace DEBUG with INFO in determine_timing.sh - Remove redundant exports from determine_timing.sh (variables come from CI) - Print EBEAM, PBEAM, and NEVENTS_PER_TEST in addition to existing variables 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Briefly, what does this PR introduce?
Parameterize CI (GitLab + GitHub Actions) and pipeline jobs to support configurable container/image/detector/version/beam settings and to place results under a structured RESULTS_BASE. Add debug logging and split some job matrices into separate, explicit jobs.
What kind of change does this PR introduce?
Please check if this PR fulfills the following:
Does this PR introduce breaking changes? What changes might users need to make to their code?
Yes
eic/job_submission_condor#58
Does this PR change default behavior?