Skip to content

Conversation

@Andrey170170
Copy link
Collaborator

Now distributed downloader should be runnable as a package.

Added supplementary documentation for all script files, since they are not packaged with python files and have to be created by the user.

Replaced hardcoded `REPO_ROOT` references with `DISTRIBUTED_DOWNLOADER_PATH` for dynamic path management across scripts. Added `initializer` as a callable function in `initialization.py` and exported relevant modules in `__all__`. Updated project scripts in `pyproject.toml` to reflect new module paths.
Updated main script to include a new environment variable for the package path. Expanded documentation in README, including CLI usage and detailed script explanations, and introduced docs/scripts_README.md for supplementary materials.
fixed logs path in `schedule_creation.slurm`
Andrey170170 and others added 4 commits May 18, 2025 21:55
Updated `MPI_downloader_verifier.py` to include a `recheck` parameter, allowing the rechecking of completed batches. Adjusted related scripts, documentation, and Slurm job handling to support the new flag for better flexibility in verification workflows.
Co-authored-by: Elizabeth Campolongo <38985481+egrace479@users.noreply.github.com>
Co-authored-by: Elizabeth Campolongo <38985481+egrace479@users.noreply.github.com>
@Andrey170170 Andrey170170 merged commit 42fe439 into main May 22, 2025
@Andrey170170 Andrey170170 deleted the package_fixes branch May 22, 2025 14:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants