Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DM-29338: Incorporate Gen 3 crosstalk in HiTS runs #80

Merged
merged 5 commits into from Jun 4, 2021

Conversation

mrawls
Copy link
Collaborator

@mrawls mrawls commented Jun 4, 2021

This PR updates a whole bunch of pipelines and adopts a new default AP Pipeline template name, goodSeeing (instead of deep).

@mrawls mrawls requested a review from isullivan June 4, 2021 02:10
Copy link
Contributor

@isullivan isullivan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Very helpful comments and descriptions throughout, thank you.


submitPath: ${PWD}/bps/{outCollection}
computeSite: ncsapool
# Memory allocated for each quantum, in MBs
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you know the maximum that can be requested? If so, that would be good to include.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't know, and therefore haven't included it 🙃 in practice, 2000-4000 is enough for most jobs, and bumping it up to 12000 or so is enough to get the occasional held job through.

# Note: keep timestamp in outCollection so you don't get a zillion sub-runs
output : 'u/${USER}/{payloadName}'
outCollection: '{output}/{timestamp}'
# UPDATE THIS to specify what data to process
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perhaps expand this comment to include the additional keys beyond "band" and "instrument" that people might want to specify.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a good suggestion, but I have empirically found that doing DECam runs with complex data queries can lead to an extremely lengthy (many days) query time. The example is intentionally minimal, but may be able to be a bit more instructive in the future when this is resolved.

@mrawls mrawls merged commit 19541ef into master Jun 4, 2021
@mrawls mrawls deleted the tickets/DM-29338 branch June 4, 2021 21:38
kfindeisen added a commit that referenced this pull request Jul 7, 2023
The default ApPipe coadds have been goodSeeing since at least
June 2021 (#80).
kfindeisen added a commit that referenced this pull request Jul 18, 2023
The default ApPipe coadds have been goodSeeing since at least
June 2021 (#80).
kfindeisen added a commit that referenced this pull request Jul 18, 2023
The default ApPipe coadds have been goodSeeing since at least
June 2021 (#80).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants