Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Get AGB workflow running on HPC #2051

Closed
3 of 5 tasks
Aariq opened this issue May 6, 2024 · 3 comments
Closed
3 of 5 tasks

Get AGB workflow running on HPC #2051

Aariq opened this issue May 6, 2024 · 3 comments
Assignees

Comments

@Aariq
Copy link
Member

Aariq commented May 6, 2024

Maybe just needs more cores??

Things to try:

  • Give main worker more cores. Maybe it needs additional memory to serialize objects??
  • Allow workers to access data store directly rather than forcing main worker to handle all the passing of targets objects
  • Run just esa_agb target as a test this works locally with 16gb RAM, just takes a long time
  • Create run.R script that I can submit as a SLURM job so interactive session isn't needed to be up and running for many hours.
  • Set wall time for heavy-duty workers much higher (like, 24 hours?)
@Aariq Aariq self-assigned this May 6, 2024
@Aariq Aariq added the 🛑 blocked Can't make progress on this issue because of another issue label May 9, 2024
@Aariq
Copy link
Member Author

Aariq commented May 9, 2024

Bug in geotargets preventing me from using custom launchers for heavy-duty targets: njtierney/geotargets#66

@Aariq Aariq removed the 🛑 blocked Can't make progress on this issue because of another issue label May 13, 2024
@Aariq
Copy link
Member Author

Aariq commented May 14, 2024

Might be good idea to split big datasets into tiles then do slopes on each tile with dynamic branching

@Aariq
Copy link
Member Author

Aariq commented May 15, 2024

Haven't quite gotten it to work, but goign to close this and open a follow-up issue

@Aariq Aariq closed this as completed May 15, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant