Skip to content

Add back epilogs to JobConfig/cosmic/NoField.fcl #457

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
brownd1978 merged 2 commits into from
Sep 30, 2025
Merged

Add back epilogs to JobConfig/cosmic/NoField.fcl #457

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
b446b7a
Add back epilogs to JobConfig/cosmic/NoField.fcl
Sep 18, 2025
d1c2477
Add extrapolition to samDatasetsSummary.sh for speed up
oksuzian Sep 19, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions JobConfig/cosmic/NoField.fcl
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,13 @@ physics.producers.g4run.Mu2eG4CommonCut: @local::Cosmic.Mu2eG4CommonCutCosmicNoF
physics.producers.FindMCPrimary.PrimaryGenIds: ["CosmicCRY", "CosmicCORSIKA"]
# since these data aren't resampled we have to apply the cosmic time offset here
physics.producers.cosmicTimeOffset.cosmicModuleLabel : "generate"

#
# final configuration
#
#include "Production/JobConfig/common/epilog.fcl"
#include "Production/JobConfig/primary/epilog.fcl"
#
# need to offset the GenParticles

physics.end_paths : [ EndPath ]
Expand Down
27 changes: 23 additions & 4 deletions Scripts/POMS/samDatasetsSummary.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,12 +1,19 @@
#!/bin/bash
# usage
if [[ -z "$1" ]]; then
echo "Usage: $0 <dataset>" >&2
echo "Usage: $0 <dataset> [--sample-files N]" >&2
echo " --sample-files N: Number of files to sample for generated events calculation (default: 10)" >&2
exit 1
fi

# Get the dataset name
dataset="$1"

# Parse command line arguments
sample_files=10
if [[ "$2" == "--sample-files" && -n "$3" ]]; then
sample_files="$3"
fi
# Obtain summary once
summary_txt=$(samweb list-definition-files --summary "$dataset" 2>/dev/null)
nfiles=$(echo "$summary_txt" | awk '/File count:/ {print $3}')
Expand All @@ -24,9 +31,21 @@ if (( nfiles == 0 )); then
exit 1
fi

# Calculate total generated events from all files
generated=$(samweb list-definition-files "$dataset" 2>/dev/null | \
xargs -n1 samweb get-metadata | awk '/dh.gencount/ { sum += $2 } END { print sum+0 }')
# Calculate total generated events by extrapolating from sampled files
# OPTIMIZATION: Use SAM's metadata query to get dh.gencount directly
sample_sum=$(samweb list-definition-files "$dataset" 2>/dev/null | \
head -"$sample_files" | while read file; do
samweb get-metadata "$file" 2>/dev/null | awk '/dh.gencount/ {print $2}'
done | awk '{sum += $1} END {print sum+0}')

# Extrapolate to total number of files
if (( sample_sum > 0 && sample_files > 0 )); then
avg_per_file=$((sample_sum / sample_files))
generated=$((avg_per_file * nfiles))
echo "Debug: Sampled $sample_files files for dh.gencount, sum=$sample_sum, avg=$avg_per_file, extrapolated=$generated" >&2
else
generated=0
fi

printf "Triggered: %s\nGenerated: %s\nFiles: %s\nSize: %s\n" "$triggered" "$generated" "$nfiles" "$size"

Expand Down