-
Notifications
You must be signed in to change notification settings - Fork 2.7k
feat(manager/cargo): support git dependencies #36444
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
feat(manager/cargo): support git dependencies #36444
Conversation
…port-git-dependencies-cargo
…ult to '*' instead of crashing. This is necessary in cases where you have a git dependency without a specified ref (no `rev`, `tag` or `branch`)
`cargo` versioning never makes sense for git dependencies, since the ref always has to be an exact match. If the current value looks like semver, we use `semver` versioning. Otherwise we use `loose` versioning.
Co-authored-by: Michael Kriese <michael.kriese@gmx.de>
Co-authored-by: RahulGautamSingh <rahultesnik@gmail.com>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM, except the comments and the protocol change situation
Co-authored-by: RahulGautamSingh <rahultesnik@gmail.com>
@@ -20,7 +20,8 @@ export function applyGitSource( | |||
platform === 'github' | |||
? GithubTagsDatasource.id | |||
: GitlabTagsDatasource.id; | |||
const { protocol, source, full_name } = parseGitUrl(git); | |||
const httpUrl = getHttpUrl(git); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is this working fine on your test repo?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yeah. I tested with git dependencies both using URLs of the form https://github.com/...
and ssh://git@github.com/...
, and they both work. Did you have any concerns about this? An edge case I may not be aware of?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I was wondering why you case didn't occur earlier as we use the same function at other places for the same purpose.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
That I don't know. The dependency update breaks without this when using ssh://git@github.com/...
style URLs with tags in the Cargo.toml
. Here is the Cargo.toml file used, and this is the error message I get:
INFO: Renovate is exiting with a non-zero code due to the following logged errors
"loggerErrors": [
{
"name": "renovate",
"level": 50,
"logContext": "h4eoLGnOiFF5APflydRsn",
"repository": "okkero/renovate-test",
"url": "/graphql",
"baseUrl": "ssh://github.com/api/",
"resolvedUrl": "ssh://github.com/api/graphql",
"msg": "Request Error: cannot parse url"
}
]
I see that massageUrl does something similar, although it seems more thorough and that it covers more cases.
Should we reuse that here instead? Or should this replacement be done at a different level entirely? I mean, it seems to me that this has to be done at some point or other, but I don't know what would be the best way to go about it...
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do you think that would work, @RahulGautamSingh , or do you recommend another approach? It just seems that the protocol for the registry URL can't be ssh
, even though cargo supports it. So, the GitHub API must be called through https
, which makes sense, so it has to be converted at some point. I am just unsure where exactly it would be best to do that.
"currentValue": "0.1.0", | ||
"datasource": "crate", | ||
"currentValue": undefined, | ||
"datasource": "git-refs", | ||
"depName": "git_dep_with_version", | ||
"depType": "dependencies", | ||
"managerData": { | ||
"nestedVersion": true, | ||
}, | ||
"skipReason": "git-dependency", |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The version was detected before, but now it is unspecified? 🤔
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Good catch. Why is currentValue
now undefined?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Right, hmm. So, for cargo git dependencies, the version
field does not specify which version of the crate to fetch, but may exist alongside branch
, rev
or tag
as a sort of sanity check: https://doc.rust-lang.org/cargo/reference/specifying-dependencies.html#the-role-of-the-version-key
So, to me it seems correct that currentValue
would be unspecified in this case, but I'm not exactly sure what to do with the version key for git dependencies, or whether we should do anything at all.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Could you please point to me which change cause this? I am unable to find the change that triggered this
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think it's this: 8bbf203#diff-09657ce916b4a05cf4f9d8b64a7c5fc2eb4e4d6078cf7c45583656dc230899c1R65
@mkniewallner Can you confirm?
@@ -20,7 +20,8 @@ export function applyGitSource( | |||
platform === 'github' | |||
? GithubTagsDatasource.id | |||
: GitlabTagsDatasource.id; | |||
const { protocol, source, full_name } = parseGitUrl(git); | |||
const httpUrl = getHttpUrl(git); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I was wondering why you case didn't occur earlier as we use the same function at other places for the same purpose.
"currentValue": "0.1.0", | ||
"datasource": "crate", | ||
"currentValue": undefined, | ||
"datasource": "git-refs", | ||
"depName": "git_dep_with_version", | ||
"depType": "dependencies", | ||
"managerData": { | ||
"nestedVersion": true, | ||
}, | ||
"skipReason": "git-dependency", |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Could you please point to me which change cause this? I am unable to find the change that triggered this
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't know. These were added before I took over the PR. Perhaps @mkniewallner can fill us in?
I tried removing these, and then two tests broke. It's not clear to me exactly what's happening here.
FAIL lib/modules/manager/cargo/artifacts.spec.ts > modules/manager/cargo/artifacts > returns updated Cargo.lock with precise version update
AssertionError: expected [ { …(2) } ] to match object [ …(2) ]
- Expected
+ Received
[
{
"cmd": "cargo update --config net.git-fetch-with-cli=true --manifest-path Cargo.toml --workspace",
+ "options": {
+ "cwd": "/tmp/github/some/repo",
+ "encoding": "utf-8",
+ "env": {
+ "HOME": "/home/user",
+ "HTTPS_PROXY": "https://example.com",
+ "HTTP_PROXY": "http://example.com",
+ "LANG": "en_US.UTF-8",
+ "LC_ALL": "en_US",
+ "NO_PROXY": "localhost",
+ "PATH": "/tmp/path",
},
- {
- "cmd": "cargo update --config net.git-fetch-with-cli=true --manifest-path Cargo.toml --package dep1@1.0.0 --precise 1.0.1",
+ "maxBuffer": 10485760,
+ "timeout": 900000,
+ },
},
]
❯ lib/modules/manager/cargo/artifacts.spec.ts:142:27
140| { file: { contents: undefined, path: 'Cargo.lock', type: 'addition' } },
141| ]);
142| expect(execSnapshots).toMatchObject([
| ^
143| {
144| cmd:
FAIL lib/modules/manager/cargo/artifacts.spec.ts > modules/manager/cargo/artifacts > returns updated Cargo.lock when a preceding dependency triggers an update in a later dependency
AssertionError: expected [ Array(1) ] to deeply equal [ { file: { …(3) } } ]
- Expected
+ Received
[
{
"file": {
- "contents": undefined,
+ "contents": "
+ [[package]]
+ name = \"dep1\"
+ version = \"1.0.1\"
+
+ [[package]]
+ name = \"dep2\"
+ version = \"1.0.2\"
+
+ [[package]]
+ name = \"dep3\"
+ version = \"1.0.0\"
+ ",
"path": "Cargo.lock",
"type": "addition",
},
},
]
❯ lib/modules/manager/cargo/artifacts.spec.ts:310:7
308| config,
309| }),
310| ).toEqual([
| ^
311| { file: { contents: undefined, path: 'Cargo.lock', type: 'addition' } },
312| ]);
Changes
This PR reintroduces Cargo git dependencies, reverted in #36412. The original PR adding support for git dependencies also introduced two regressions. This PR addresses both:
cargo
versioning by default for git dependencies, instead making a best effort decision betweensemver
andloose
based on the current value's shape.currentValue
if it is not defined, effectively skipping git dependencies without a specified ref, leaving it up to the lock file maintenance schedule to update the lock file.Context
Documentation (please check one with an [x])
How I've tested my work (please select one)
I have verified these changes via: