New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. Weβll occasionally send you account related emails.
Already on GitHub? Sign in to your account
use cache dir as a default remote when importing from local repo #2915
use cache dir as a default remote when importing from local repo #2915
Conversation
If the URL is local dvc repo, the import/get should fetch from the source project cache. Fixes iterative#2599
2b0eef0
to
779a0a1
Compare
If the URL is local dvc repo, the import/get should fetch from the source project cache. Fixes iterative#2599
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
General remark:
I think that we should not modify config of remote repo
EDIT:
@efiop had the same suggestion and it has not been resolved yet
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Some minor things, LGTM besides that.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks good! Just a few more minor comments.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks good! Thanks!
@maykulkarni Could you also create a ticket for the docs or create a PR for docs to reflect this new logic? |
# check if the URL is local and no default remote is present | ||
# add default remote pointing to the original repo's cache location | ||
if os.path.isdir(url): | ||
rconfig = RemoteConfig(repo.config) | ||
if not _default_remote_set(rconfig): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@maykulkarni @efiop does this logic mean that if there's a default remote setup, still DVC skips the local cache? Because I think it should try the local cache first regardless of remotes.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@jorgeorpinel Yes, precisely. It is hard for us to try both right now, as we don't have that kind of cascading remote logic implemented. I would wait for someone to ask for this.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sorry, I'm not sure I understand:
-
Every DVC repo has a cache directory right? So in theory it can always be the first place to get stuff from when using get/import from another local project. Why does this require complicated remote checking logic?
-
Does this PR really close
import
: get from source project cache ifurl/path
are in the local systemΒ #2599 per that issue's description and discussion?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
-
Yes, but "first place to get stuff from" means that there is a second place, and our code is not ready to cascade from first to second yet, if it can't find all the stuff it needs in the local cache dir.
-
From what we've understood initially - yes, but now it looks like we didn't quite understand each other.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I see. OK!
If the URL is local dvc repo, the import/get should fetch from the source project cache.
Fixes #2599
β Have you followed the guidelines in the Contributing to DVC list?
π Check this box if this PR does not require documentation updates, or if it does and you have created a separate PR in dvc.org with such updates (or at least opened an issue about it in that repo). Please link below to your PR (or issue) in the dvc.org repo.
β Have you checked DeepSource, CodeClimate, and other sanity checks below? We consider their findings recommendatory and don't expect everything to be addressed. Please review them carefully and fix those that actually improve code or fix bugs.
Thank you for the contribution - we'll try to review it as soon as possible. π