Skip to content

Automatically download files if they are missing - #225

Merged
drewoldag merged 2 commits into
mainfrom
issue/122/auto-download-files-as-needed
May 10, 2025
Merged

Automatically download files if they are missing#225
drewoldag merged 2 commits into
mainfrom
issue/122/auto-download-files-as-needed

Conversation

@drewoldag

@drewoldag drewoldag commented May 9, 2025

Copy link
Copy Markdown
Contributor

Automatically download files if they are missing when running verbs from the command line. When creating SorchaObservatory object, request that the layup cache files are used.

Fixes #126 #108 .

Describe your changes.
Two big changes here

  1. We check to see if all the expected bootstrap files are present when running convert, predict, or orbitfit from the commandline. If anything is missing, we download it (and decompress if needed). This should fix issue automatically download the files layup bootstrap does if it's not been run before one of the other layup verbs runs #126.
  2. When creating a LayupObservatory object, we instantiate a SorchaObservatory object under the hood. The SorchaObservatory can take a cache directory as input, which we were not passing. Now we will, so the SorchaObservatory object will look in the layup cache directory for the files it needs. This should fix issue sorcha <-> layup caching conflict #108.

Review Checklist for Source Code Changes

  • Does pip install still work?
  • Have you written a unit test for any new functions?
  • Do all the units tests run successfully?
  • Does Layup run successfully on a test set of input files/databases?
  • Have you used black on the files you have updated to confirm python programming style guide enforcement?

@drewoldag drewoldag self-assigned this May 9, 2025
@drewoldag drewoldag linked an issue May 9, 2025 that may be closed by this pull request
3 tasks
@drewoldag
drewoldag marked this pull request as ready for review May 9, 2025 21:47
@drewoldag
drewoldag requested review from Copilot and wilsonbb May 9, 2025 21:47

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR implements a mechanism to automatically download missing bootstrap files when running various command‐line verbs and also ensures that the cache directory is properly passed to the underlying Sorcha observatory object. Key changes include:

  • Updating CLI argument names from “c” to “config” for consistency.
  • Adding calls to download missing bootstrap files in the predict, orbitfit, convert, and comet modules.
  • Instantiating LayupObservatory with a specified cache_dir derived from CLI arguments.

Reviewed Changes

Copilot reviewed 9 out of 9 changed files in this pull request and generated no comments.

Show a summary per file
File Description
src/layup_cmdline/predict.py Updated config argument and added bootstrap file checking
src/layup_cmdline/orbitfit.py Updated config handling and added bootstrap file checking
src/layup_cmdline/convert.py Updated config argument and added bootstrap file checking
src/layup_cmdline/comet.py Updated config argument and added bootstrap file checking
src/layup_cmdline/bootstrap.py Updated config argument name
src/layup/utilities/data_processing_utilities.py Updated LayupObservatory to accept a cache_dir argument
src/layup/utilities/bootstrap_utilties/download_utilities.py Added helper function to download missing files
src/layup/predict.py Passed cache_dir to LayupObservatory in predict function
src/layup/orbitfit.py Passed cache_dir to LayupObservatory in orbitfit function
Comments suppressed due to low confidence (2)

src/layup_cmdline/predict.py:287

  • [nitpick] The block of code for loading and validating the configuration file is repeated in multiple CLI modules. Consider refactoring this logic into a shared utility function to reduce duplication.
configs = LayupConfigs(); if args.config: find_file_or_exit(args.config, "-c, --config")

src/layup/utilities/bootstrap_utilties/download_utilities.py:62

  • The function uses _decompress as a processor in the partial function, but no definition or import for _decompress is provided in this diff. Please ensure that _decompress is properly defined or imported.
fetch_partial = partial(retriever.fetch, processor=_decompress, progressbar=True)

@drewoldag drewoldag changed the title WIP - Automatically download files if they are missing Automatically download files if they are missing May 9, 2025
@drewoldag
drewoldag merged commit e064861 into main May 10, 2025
@drewoldag
drewoldag deleted the issue/122/auto-download-files-as-needed branch May 10, 2025 15:24
@drewoldag drewoldag mentioned this pull request May 19, 2025
3 tasks
@Little-Ryugu Little-Ryugu mentioned this pull request Jul 27, 2026
5 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

automatically download the files layup bootstrap does if it's not been run before one of the other layup verbs runs sorcha <-> layup caching conflict

3 participants