Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fix dataset downloading #7864

Merged
merged 43 commits into from
May 28, 2024
Merged

Fix dataset downloading #7864

merged 43 commits into from
May 28, 2024

Commits on May 2, 2024

  1. Make dataset cache lifetime configurable

    @zhiltsov-max
    zhiltsov-max committed May 2, 2024
    Configuration menu
    Copy the full SHA
    d7bb1c1 View commit details
    Browse the repository at this point in the history

Commits on May 3, 2024

  1. Use flock to lock export cache

    @zhiltsov-max
    zhiltsov-max committed May 3, 2024
    Configuration menu
    Copy the full SHA
    7f151b1 View commit details
    Browse the repository at this point in the history

Commits on May 7, 2024

  1. - use redlock for export cache locking 

    - prolong export cache lifetime on cache requests
- change 500 to 404 on missing cache file during downloading
- remove rq job removal after result retrieval
- make export filenames more distinct
- fix possible infinite deferring of dependent rq jobs
    @zhiltsov-max
    zhiltsov-max committed May 7, 2024
    Configuration menu
    Copy the full SHA
    f48e807 View commit details
    Browse the repository at this point in the history

  2. update server deps

    @zhiltsov-max
    zhiltsov-max committed May 7, 2024
    Configuration menu
    Copy the full SHA
    4c58bf5 View commit details
    Browse the repository at this point in the history

Commits on May 8, 2024

  1. Merge branch 'develop' into zm/fix-dataset-downloading

    @zhiltsov-max
    zhiltsov-max committed May 8, 2024
    Configuration menu
    Copy the full SHA
    d7fa31d View commit details
    Browse the repository at this point in the history

  2. Fix deps after merge

    @zhiltsov-max
    zhiltsov-max committed May 8, 2024
    Configuration menu
    Copy the full SHA
    5c22d28 View commit details
    Browse the repository at this point in the history

  3. Refactor some code

    @zhiltsov-max
    zhiltsov-max committed May 8, 2024
    Configuration menu
    Copy the full SHA
    49baadf View commit details
    Browse the repository at this point in the history

  4. fix import

    @zhiltsov-max
    zhiltsov-max committed May 8, 2024
    Configuration menu
    Copy the full SHA
    de73fe6 View commit details
    Browse the repository at this point in the history

  5. Fix linter

    @zhiltsov-max
    zhiltsov-max committed May 8, 2024
    Configuration menu
    Copy the full SHA
    35ce1f6 View commit details
    Browse the repository at this point in the history

  6. Update changelog

    @zhiltsov-max
    zhiltsov-max committed May 8, 2024
    Configuration menu
    Copy the full SHA
    7b671f5 View commit details
    Browse the repository at this point in the history

Commits on May 9, 2024

  1. Fix failing cloud storage tests

    @zhiltsov-max
    zhiltsov-max committed May 9, 2024
    Configuration menu
    Copy the full SHA
    4a4b5d0 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'develop' into zm/fix-dataset-downloading

    @zhiltsov-max
    zhiltsov-max committed May 9, 2024
    Configuration menu
    Copy the full SHA
    05f6dd1 View commit details
    Browse the repository at this point in the history

Commits on May 14, 2024

  1. Add rough test

    @zhiltsov-max
    zhiltsov-max committed May 14, 2024
    Configuration menu
    Copy the full SHA
    2c4be73 View commit details
    Browse the repository at this point in the history

  2. Refactor code

    @zhiltsov-max
    zhiltsov-max committed May 14, 2024
    Configuration menu
    Copy the full SHA
    5dc3e56 View commit details
    Browse the repository at this point in the history

  3. Refactor test

    @zhiltsov-max
    zhiltsov-max committed May 14, 2024
    Configuration menu
    Copy the full SHA
    4c24e10 View commit details
    Browse the repository at this point in the history

  4. Merge remote-tracking branch 'origin/zm/fix-dataset-downloading' into… 

    … zm/fix-dataset-downloading
    @zhiltsov-max
    zhiltsov-max committed May 14, 2024
    Configuration menu
    Copy the full SHA
    98ad20e View commit details
    Browse the repository at this point in the history

  5. Refactor some code

    @zhiltsov-max
    zhiltsov-max committed May 14, 2024
    Configuration menu
    Copy the full SHA
    963f657 View commit details
    Browse the repository at this point in the history

  6. Update deps

    @zhiltsov-max
    zhiltsov-max committed May 14, 2024
    Configuration menu
    Copy the full SHA
    2bf19db View commit details
    Browse the repository at this point in the history

  7. Add redlock docs link

    @zhiltsov-max
    zhiltsov-max committed May 14, 2024
    Configuration menu
    Copy the full SHA
    c0769f0 View commit details
    Browse the repository at this point in the history

  8. Add more tests

    @zhiltsov-max
    zhiltsov-max committed May 14, 2024
    Configuration menu
    Copy the full SHA
    087ac42 View commit details
    Browse the repository at this point in the history

  9. Add more tests

    @zhiltsov-max
    zhiltsov-max committed May 14, 2024
    Configuration menu
    Copy the full SHA
    0d75d4a View commit details
    Browse the repository at this point in the history

  10. Add more tests, update export test

    @zhiltsov-max
    zhiltsov-max committed May 14, 2024
    Configuration menu
    Copy the full SHA
    ea71c7d View commit details
    Browse the repository at this point in the history

Commits on May 16, 2024

  1. Update deps

    @zhiltsov-max
    zhiltsov-max committed May 16, 2024
    Configuration menu
    Copy the full SHA
    7085076 View commit details
    Browse the repository at this point in the history

  2. Add rq cleanup to tests with multiple iterations

    @zhiltsov-max
    zhiltsov-max committed May 16, 2024
    Configuration menu
    Copy the full SHA
    f8d142f View commit details
    Browse the repository at this point in the history

  3. Update comment

    @zhiltsov-max
    zhiltsov-max committed May 16, 2024
    Configuration menu
    Copy the full SHA
    9479888 View commit details
    Browse the repository at this point in the history

  4. Remove unused imports

    @zhiltsov-max
    zhiltsov-max committed May 16, 2024
    Configuration menu
    Copy the full SHA
    da06903 View commit details
    Browse the repository at this point in the history

  5. Refactor lock release

    @zhiltsov-max
    zhiltsov-max committed May 16, 2024
    Configuration menu
    Copy the full SHA
    700bf2c View commit details
    Browse the repository at this point in the history

  6. Rename lock function

    @zhiltsov-max
    zhiltsov-max committed May 16, 2024
    Configuration menu
    Copy the full SHA
    9ccafc5 View commit details
    Browse the repository at this point in the history

  7. Add api compatibility test

    @zhiltsov-max
    zhiltsov-max committed May 16, 2024
    Configuration menu
    Copy the full SHA
    c2b3049 View commit details
    Browse the repository at this point in the history

  8. Merge branch 'develop' into zm/fix-dataset-downloading

    @zhiltsov-max
    zhiltsov-max committed May 16, 2024
    Configuration menu
    Copy the full SHA
    228e82e View commit details
    Browse the repository at this point in the history

  9. Fix tests

    @zhiltsov-max
    zhiltsov-max committed May 16, 2024
    Configuration menu
    Copy the full SHA
    4ddc79d View commit details
    Browse the repository at this point in the history

  10. Merge remote-tracking branch 'origin/zm/fix-dataset-downloading' into… 

    … zm/fix-dataset-downloading
    @zhiltsov-max
    zhiltsov-max committed May 16, 2024
    Configuration menu
    Copy the full SHA
    447b4df View commit details
    Browse the repository at this point in the history

Commits on May 17, 2024

  1. Merge branch 'develop' into zm/fix-dataset-downloading

    @zhiltsov-max
    zhiltsov-max committed May 17, 2024
    Configuration menu
    Copy the full SHA
    2a67f2e View commit details
    Browse the repository at this point in the history

  2. Restore old job removal behavior

    @zhiltsov-max
    zhiltsov-max committed May 17, 2024
    Configuration menu
    Copy the full SHA
    4471aa9 View commit details
    Browse the repository at this point in the history

  3. Merge remote-tracking branch 'origin/zm/fix-dataset-downloading' into… 

    … zm/fix-dataset-downloading
    @zhiltsov-max
    zhiltsov-max committed May 17, 2024
    Configuration menu
    Copy the full SHA
    de2c805 View commit details
    Browse the repository at this point in the history

  4. Update downloading test

    @zhiltsov-max
    zhiltsov-max committed May 17, 2024
    Configuration menu
    Copy the full SHA
    e76382d View commit details
    Browse the repository at this point in the history

Commits on May 27, 2024

  1. Fix comments

    @zhiltsov-max
    zhiltsov-max committed May 27, 2024
    Configuration menu
    Copy the full SHA
    2af76a8 View commit details
    Browse the repository at this point in the history

  2. Remove unused import

    @zhiltsov-max
    zhiltsov-max committed May 27, 2024
    Configuration menu
    Copy the full SHA
    aafc369 View commit details
    Browse the repository at this point in the history

Commits on May 28, 2024

  1. Raise an exception on unlocking expired lock

    @zhiltsov-max
    zhiltsov-max committed May 28, 2024
    Configuration menu
    Copy the full SHA
    275a274 View commit details
    Browse the repository at this point in the history

  2. Remove unexpected requirements changes

    @zhiltsov-max
    zhiltsov-max committed May 28, 2024
    Configuration menu
    Copy the full SHA
    297d686 View commit details
    Browse the repository at this point in the history

  3. Merge branch 'develop' into zm/fix-dataset-downloading

    @zhiltsov-max
    zhiltsov-max committed May 28, 2024
    Configuration menu
    Copy the full SHA
    8d5401c View commit details
    Browse the repository at this point in the history

  4. Remove extra requirements

    @zhiltsov-max
    zhiltsov-max committed May 28, 2024
    Configuration menu
    Copy the full SHA
    664ed28 View commit details
    Browse the repository at this point in the history

  5. Merge remote-tracking branch 'origin/zm/fix-dataset-downloading' into… 

    … zm/fix-dataset-downloading
    @zhiltsov-max
    zhiltsov-max committed May 28, 2024
    Configuration menu
    Copy the full SHA
    6fc3f63 View commit details
    Browse the repository at this point in the history