Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CI: Add the dvc '--no-run-cache' option to avoid RunCacheNotSupported error #3273

Merged
merged 3 commits into from
May 27, 2024

Conversation

seisman
Copy link
Member

@seisman seisman commented May 27, 2024

dvc 3.51 breaks with following error (https://github.com/GenericMappingTools/pygmt/actions/runs/9247930872/job/25437578302?pr=3182):

Run dvc pull --verbose && ls -lhR pygmt/tests/baseline/
  dvc pull --verbose && ls -lhR pygmt/tests/baseline/
  shell: /usr/bin/bash -l {0}
  env:
    OS: ubuntu-latest
    PYTHON: 3.10
    NUMPY: 1.[2](https://github.com/GenericMappingTools/pygmt/actions/runs/9247930872/job/25437578302?pr=3182#step:6:2)3
    MAMBA_ROOT_PREFIX: /home/runner/micromamba
    MAMBA_EXE: /home/runner/micromamba-bin/micromamba
    CONDARC: /home/runner/work/_temp/setup-micromamba/.condarc
2024-05-27 01:48:[3](https://github.com/GenericMappingTools/pygmt/actions/runs/9247930872/job/25437578302?pr=3182#step:6:3)4,944 DEBUG: v3.51.0 (conda), CPython 3.10.1[4](https://github.com/GenericMappingTools/pygmt/actions/runs/9247930872/job/25437578302?pr=3182#step:6:4) on Linux-6.[5](https://github.com/GenericMappingTools/pygmt/actions/runs/9247930872/job/25437578302?pr=3182#step:6:5).0-1021-azure-x86_[6](https://github.com/GenericMappingTools/pygmt/actions/runs/9247930872/job/25437578302?pr=3182#step:6:6)4-with-glibc2.35
2024-05-2[7](https://github.com/GenericMappingTools/pygmt/actions/runs/9247930872/job/25437578302?pr=3182#step:6:7) 01:48:34,944 DEBUG: command: /home/runner/micromamba/envs/pygmt/bin/dvc pull --verbose
2024-05-27 01:4[8](https://github.com/GenericMappingTools/pygmt/actions/runs/9247930872/job/25437578302?pr=3182#step:6:8):35,191 ERROR: failed to pull data from the cloud - run-cache is not supported for http filesystem: https://dagshub.com/GenericMappingTools/pygmt.dvc
Traceback (most recent call last):
  File "/home/runner/micromamba/envs/pygmt/lib/python3.10/site-packages/dvc/commands/data_sync.py", line 35, in run
    stats = self.repo.pull(
  File "/home/runner/micromamba/envs/pygmt/lib/python3.10/site-packages/dvc/repo/__init__.py", line 58, in wrapper
    return f(repo, *args, **kwargs)
  File "/home/runner/micromamba/envs/pygmt/lib/python3.10/site-packages/dvc/repo/pull.py", line 30, in pull
    processed_files_count = self.fetch(
  File "/home/runner/micromamba/envs/pygmt/lib/python3.10/site-packages/dvc/repo/__init__.py", line 58, in wrapper
    return f(repo, *args, **kwargs)
  File "/home/runner/micromamba/envs/pygmt/lib/python3.10/site-packages/dvc/repo/fetch.py", line 138, in fetch
    self.stage_cache.pull(remote)
  File "/home/runner/micromamba/envs/pygmt/lib/python3.10/site-packages/dvc/stage/cache.py", line 2[9](https://github.com/GenericMappingTools/pygmt/actions/runs/9247930872/job/25437578302?pr=3182#step:6:9)0, in pull
    return self.transfer(odb, self.repo.cache.legacy)
  File "/home/runner/micromamba/envs/pygmt/lib/python3.[10](https://github.com/GenericMappingTools/pygmt/actions/runs/9247930872/job/25437578302?pr=3182#step:6:10)/site-packages/dvc/stage/cache.py", line 248, in transfer
    raise RunCacheNotSupported(message)
dvc.stage.cache.RunCacheNotSupported: run-cache is not supported for http filesystem: https://dagshub.com/GenericMappingTools/pygmt.dvc

2024-05-27 01:48:35,195 DEBUG: Analytics is enabled.
2024-05-27 01:48:35,229 DEBUG: Trying to spawn ['daemon', 'analytics', '/tmp/tmpv7kiuswp', '-v']
2024-05-27 01:48:35,233 DEBUG: Spawned ['daemon', 'analytics', '/tmp/tmpv7kiuswp', '-v'] with pid 2[14](https://github.com/GenericMappingTools/pygmt/actions/runs/9247930872/job/25437578302?pr=3182#step:6:15)7

@seisman seisman added the run/test-gmt-dev Trigger the GMT Dev Tests workflow in PR label May 27, 2024
@seisman seisman added this to the 0.13.0 milestone May 27, 2024
@seisman seisman added maintenance Boring but important stuff for the core devs skip-changelog Skip adding Pull Request to changelog labels May 27, 2024
@seisman seisman changed the title Temporarily pin dvc to <3.51 CI: Temporarily pin dvc to <3.51 May 27, 2024
@weiji14
Copy link
Member

weiji14 commented May 27, 2024

Looking at iterative/dvc#10433 and iterative/dvc.org#5242, maybe we should try using dvc pull --no-run-cache ... temporarily (at least until run-cache is enabled on http)?

@seisman seisman changed the title CI: Temporarily pin dvc to <3.51 CI: Add the dvc '--no-run-cache' option to avoid RunCacheNotSupported error May 27, 2024
@seisman seisman removed the run/test-gmt-dev Trigger the GMT Dev Tests workflow in PR label May 27, 2024
@seisman seisman merged commit 466c8b6 into main May 27, 2024
14 of 17 checks passed
@seisman seisman deleted the pin-dvc branch May 27, 2024 04:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
maintenance Boring but important stuff for the core devs skip-changelog Skip adding Pull Request to changelog
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants