Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

dependabot could not clone private repository for swift that located same organization and granted access. #8027

Open
1 task done
r-plus opened this issue Sep 13, 2023 · 8 comments
Labels
F: private-registries 💂‍♂️ Issues about using private registries with Dependabot; may be paired with an R: label. L: swift Swift packages T: bug 🐞 Something isn't working

Comments

@r-plus
Copy link

r-plus commented Sep 13, 2023

Is there an existing issue for this?

  • I have searched the existing issues

Package ecosystem

swift

Package manager version

5.8

Language version

5.8

Manifest location and content before the Dependabot update

/Package.swift

repository is appbrew/try-dependabot-spm, appbrew is the organization name.

// swift-tools-version: 5.8
// The swift-tools-version declares the minimum version of Swift required to build this package.

import PackageDescription

let package = Package(
    name: "try-dependabot-spm",
    products: [
        // Products define the executables and libraries a package produces, making them visible to other packages.
        .library(
            name: "try-dependabot-spm",
            targets: ["try-dependabot-spm"]),
    ],
    dependencies: [
        .package(url: "git@github.com:appbrew/try-dependabot-spm-private.git", exact: "0.0.1"),
    ],
    targets: [
        // Targets are the basic building blocks of a package, defining a module or a test suite.
        // Targets can depend on other targets in this package and products from dependencies.
        .target(
            name: "try-dependabot-spm"),
        .testTarget(
            name: "try-dependabot-spmTests",
            dependencies: ["try-dependabot-spm"]),
    ]
)

and Package.resolved

{
  "pins" : [
    {
      "identity" : "try-dependabot-spm-private",
      "kind" : "remoteSourceControl",
      "location" : "git@github.com:appbrew/try-dependabot-spm-private.git",
      "state" : {
        "revision" : "e645ddfa48964b5b81ab3fba2976e0989d3d176e",
        "version" : "0.0.1"
      }
    }
  ],
  "version" : 2
}

dependabot.yml content

version: 2
updates:
  - package-ecosystem: "swift"
    directory: "/"
    schedule:
      interval: "weekly"

Updated dependency

try-dependabot-spm-private repo is not yet updated from 0.0.1, so I expect dependabot will not create pull request without error.

What you expected to see, versus what you actually saw

I expect dependabot will not create pull request without error.

But dependabot will show repository clone error even if grant dependabot access to try-dependabot-spm-private repo that describing in this docs.

スクリーンショット 2023-09-14 0 35 08
  proxy | 2023/09/13 15:22:36 proxy starting, commit: 93c4a893d794d736d84e940a79420e8d1180c0bd
  proxy | 2023/09/13 15:22:36 Listening (:1080)
updater | 2023-09-13T15:22:37.794093800 [721527124:main:WARN:src/devices/src/legacy/serial.rs:222] Detached the serial input due to peer close/error.
updater | time="2023-09-13T15:22:40Z" level=info msg="guest starting" commit=b073e069d366dc2f68bb8ef0134feb3c29cacadd
updater | time="2023-09-13T15:22:40Z" level=info msg="starting job..." fetcher_timeout=10m0s job_id=721527124 updater_timeout=45m0s updater_version=3ca52579b79278b4007287c479a40acdee6fb88a-swift
updater | 2023/09/13 15:22:42 INFO Raven 3.1.2 ready to catch errors
updater | 2023/09/13 15:22:43 INFO <job_721527124> Starting job processing
  proxy | 2023/09/13 15:22:44 [002] GET https://github.com:443/appbrew/try-dependabot-spm/info/refs?service=git-upload-pack
  proxy | 2023/09/13 15:22:44 [002] * authenticating git server request (host: github.com)
  proxy | 2023/09/13 15:22:44 [002] 200 https://github.com:443/appbrew/try-dependabot-spm/info/refs?service=git-upload-pack
  proxy | 2023/09/13 15:22:44 [004] POST https://github.com:443/appbrew/try-dependabot-spm/git-upload-pack
  proxy | 2023/09/13 15:22:44 [004] * authenticating git server request (host: github.com)
  proxy | 2023/09/13 15:22:44 [004] 200 https://github.com:443/appbrew/try-dependabot-spm/git-upload-pack
  proxy | 2023/09/13 15:22:44 [006] POST https://github.com:443/appbrew/try-dependabot-spm/git-upload-pack
  proxy | 2023/09/13 15:22:44 [006] * authenticating git server request (host: github.com)
  proxy | 2023/09/13 15:22:44 [006] 200 https://github.com:443/appbrew/try-dependabot-spm/git-upload-pack
updater | 2023/09/13 15:22:44 INFO <job_721527124> Finished job processing
updater | time="2023-09-13T15:22:44Z" level=info msg="task complete" container_id=job-721527124-file-fetcher exit_code=0 job_id=721527124 step=fetcher
updater | 2023/09/13 15:22:46 INFO Raven 3.1.2 ready to catch errors
updater | 2023/09/13 15:22:48 INFO <job_721527124> Starting job processing
  proxy | 2023/09/13 15:22:50 [008] GET https://github.com:443/appbrew/try-dependabot-spm-private.git/info/refs?service=git-upload-pack
  proxy | 2023/09/13 15:22:50 [008] * authenticating git server request (host: github.com)
  proxy | 2023/09/13 15:22:50 [008] 404 https://github.com:443/appbrew/try-dependabot-spm-private.git/info/refs?service=git-upload-pack
  proxy | 2023/09/13 15:22:50 [008] * auth'd git request returned 404, retrying without auth
  proxy | 2023/09/13 15:22:50 [008] * de-auth'd request returned 401, replacing response
  proxy | 2023/09/13 15:22:50 [010] GET https://github.com:443/appbrew/try-dependabot-spm-private.git/info/refs?service=git-upload-pack
  proxy | 2023/09/13 15:22:50 [010] * authenticating git server request (host: github.com)
  proxy | 2023/09/13 15:22:50 [010] 404 https://github.com:443/appbrew/try-dependabot-spm-private.git/info/refs?service=git-upload-pack
  proxy | 2023/09/13 15:22:50 [010] * auth'd git request returned 404, retrying without auth
  proxy | 2023/09/13 15:22:50 [010] * de-auth'd request returned 401, replacing response
  proxy | 2023/09/13 15:22:50 [012] GET https://github.com:443/appbrew/try-dependabot-spm-private.git/info/refs?service=git-upload-pack
  proxy | 2023/09/13 15:22:50 [012] * authenticating git server request (host: github.com)
  proxy | 2023/09/13 15:22:51 [012] 404 https://github.com:443/appbrew/try-dependabot-spm-private.git/info/refs?service=git-upload-pack
  proxy | 2023/09/13 15:22:51 [012] * auth'd git request returned 404, retrying without auth
  proxy | 2023/09/13 15:22:51 [012] * de-auth'd request returned 401, replacing response
  proxy | 2023/09/13 15:22:51 [014] GET https://github.com:443/appbrew/try-dependabot-spm-private.git/info/refs?service=git-upload-pack
  proxy | 2023/09/13 15:22:51 [014] * authenticating git server request (host: github.com)
  proxy | 2023/09/13 15:22:51 [014] 404 https://github.com:443/appbrew/try-dependabot-spm-private.git/info/refs?service=git-upload-pack
  proxy | 2023/09/13 15:22:51 [014] * auth'd git request returned 404, retrying without auth
  proxy | 2023/09/13 15:22:51 [014] * de-auth'd request returned 401, replacing response
  proxy | 2023/09/13 15:22:51 [016] GET https://github.com:443/appbrew/try-dependabot-spm-private.git/info/refs?service=git-upload-pack
  proxy | 2023/09/13 15:22:51 [016] * authenticating git server request (host: github.com)
  proxy | 2023/09/13 15:22:51 [016] 404 https://github.com:443/appbrew/try-dependabot-spm-private.git/info/refs?service=git-upload-pack
  proxy | 2023/09/13 15:22:51 [016] * auth'd git request returned 404, retrying without auth
  proxy | 2023/09/13 15:22:51 [016] * de-auth'd request returned 401, replacing response
  proxy | 2023/09/13 15:22:52 [018] GET https://github.com:443/appbrew/try-dependabot-spm-private.git/info/refs?service=git-upload-pack
  proxy | 2023/09/13 15:22:52 [018] * authenticating git server request (host: github.com)
  proxy | 2023/09/13 15:22:52 [018] 404 https://github.com:443/appbrew/try-dependabot-spm-private.git/info/refs?service=git-upload-pack
  proxy | 2023/09/13 15:22:52 [018] * auth'd git request returned 404, retrying without auth
  proxy | 2023/09/13 15:22:52 [018] * de-auth'd request returned 401, replacing response
  proxy | 2023/09/13 15:22:52 [020] GET https://github.com:443/appbrew/try-dependabot-spm-private.git/info/refs?service=git-upload-pack
  proxy | 2023/09/13 15:22:52 [020] * authenticating git server request (host: github.com)
  proxy | 2023/09/13 15:22:52 [020] 404 https://github.com:443/appbrew/try-dependabot-spm-private.git/info/refs?service=git-upload-pack
  proxy | 2023/09/13 15:22:52 [020] * auth'd git request returned 404, retrying without auth
  proxy | 2023/09/13 15:22:52 [020] * de-auth'd request returned 401, replacing response
  proxy | 2023/09/13 15:22:52 [022] GET https://github.com:443/appbrew/try-dependabot-spm-private.git/info/refs?service=git-upload-pack
  proxy | 2023/09/13 15:22:52 [022] * authenticating git server request (host: github.com)
  proxy | 2023/09/13 15:22:52 [022] 404 https://github.com:443/appbrew/try-dependabot-spm-private.git/info/refs?service=git-upload-pack
  proxy | 2023/09/13 15:22:52 [022] * auth'd git request returned 404, retrying without auth
  proxy | 2023/09/13 15:22:52 [022] * de-auth'd request returned 401, replacing response
updater | 2023/09/13 15:22:53 ERROR <job_721527124> Fetching git@github.com:appbrew/try-dependabot-spm-private.git
updater | warning: 'try-dependabot-spm-private': skipping cache due to an error: Failed to clone repository git@github.com:appbrew/try-dependabot-spm-private.git:
updater |     Cloning into bare repository '/home/dependabot/.cache/org.swift.swiftpm/repositories/try-dependabot-spm-private-664e0590'...
updater |     fatal: could not read Password for 'https://git@github.com': terminal prompts disabled
updater | Fetching git@github.com:appbrew/try-dependabot-spm-private.git
updater | warning: 'try-dependabot-spm-private': skipping cache due to an error: Failed to clone repository git@github.com:appbrew/try-dependabot-spm-private.git:
updater |     Cloning into bare repository '/home/dependabot/.cache/org.swift.swiftpm/repositories/try-dependabot-spm-private-664e0590'...
updater |     fatal: could not read Password for 'https://git@github.com': terminal prompts disabled
updater | error: Failed to clone repository git@github.com:appbrew/try-dependabot-spm-private.git:
updater |     Cloning into bare repository '/home/dependabot/dependabot-updater/repo/.build/repositories/try-dependabot-spm-private-664e0590'...
updater |     fatal: could not read Password for 'https://git@github.com': terminal prompts disabled
updater | 2023/09/13 15:22:53 ERROR <job_721527124> /home/dependabot/common/lib/dependabot/shared_helpers.rb:343:in `run_shell_command'
updater | 2023/09/13 15:22:53 ERROR <job_721527124> /home/dependabot/swift/lib/dependabot/swift/file_parser/dependency_parser.rb:38:in `formatted_deps'
updater | 2023/09/13 15:22:53 ERROR <job_721527124> /home/dependabot/swift/lib/dependabot/swift/file_parser/dependency_parser.rb:24:in `block (2 levels) in parse'
updater | 2023/09/13 15:22:53 ERROR <job_721527124> /home/dependabot/common/lib/dependabot/shared_helpers.rb:194:in `with_git_configured'
updater | 2023/09/13 15:22:53 ERROR <job_721527124> /home/dependabot/swift/lib/dependabot/swift/file_parser/dependency_parser.rb:23:in `block in parse'
updater | 2023/09/13 15:22:53 ERROR <job_721527124> /home/dependabot/common/lib/dependabot/shared_helpers.rb:41:in `block in in_a_temporary_repo_directory'
updater | 2023/09/13 15:22:53 ERROR <job_721527124> /home/dependabot/common/lib/dependabot/shared_helpers.rb:41:in `chdir'
updater | 2023/09/13 15:22:53 ERROR <job_721527124> /home/dependabot/common/lib/dependabot/shared_helpers.rb:41:in `in_a_temporary_repo_directory'
updater | 2023/09/13 15:22:53 ERROR <job_721527124> /home/dependabot/swift/lib/dependabot/swift/file_parser/dependency_parser.rb:20:in `parse'
updater | 2023/09/13 15:22:53 ERROR <job_721527124> /home/dependabot/swift/lib/dependabot/swift/file_parser.rb:17:in `parse'
updater | 2023/09/13 15:22:53 ERROR <job_721527124> /home/dependabot/dependabot-updater/lib/dependabot/dependency_snapshot.rb:99:in `parse_files!'
updater | 2023/09/13 15:22:53 ERROR <job_721527124> /home/dependabot/dependabot-updater/lib/dependabot/dependency_snapshot.rb:88:in `initialize'
updater | 2023/09/13 15:22:53 ERROR <job_721527124> /home/dependabot/dependabot-updater/lib/dependabot/dependency_snapshot.rb:21:in `new'
updater | 2023/09/13 15:22:53 ERROR <job_721527124> /home/dependabot/dependabot-updater/lib/dependabot/dependency_snapshot.rb:21:in `create_from_job_definition'
updater | 2023/09/13 15:22:53 ERROR <job_721527124> /home/dependabot/dependabot-updater/lib/dependabot/update_files_command.rb:16:in `perform_job'
updater | 2023/09/13 15:22:53 ERROR <job_721527124> /home/dependabot/dependabot-updater/lib/dependabot/base_command.rb:52:in `run'
updater | 2023/09/13 15:22:53 ERROR <job_721527124> bin/update_files.rb:23:in `<main>'
updater | 2023/09/13 15:22:53 INFO <job_721527124> Sending event dc0968eb0ab240f2bc71de1549ff1a5a to Sentry
  proxy | 2023/09/13 15:22:53 [024] POST https://sentry.io:443/api/1451818/store/
  proxy | 2023/09/13 15:22:53 [024] 200 https://sentry.io:443/api/1451818/store/
updater | 2023/09/13 15:22:54 INFO <job_721527124> Finished job processing
updater | 2023/09/13 15:22:54 INFO Results:
updater | Dependabot encountered '1' error(s) during execution, please check the logs for more details.
updater | +---------------+
updater | |    Errors     |
updater | +---------------+
updater | | unknown_error |
updater | +---------------+
updater | time="2023-09-13T15:22:54Z" level=info msg="task complete" container_id=job-721527124-updater exit_code=0 job_id=721527124 step=updater

Native package manager behavior

no output without error.

$ swift package resolve
$

Images of the diff or a link to the PR, issue, or logs

No response

Smallest manifest that reproduces the issue

No response

@r-plus r-plus added the T: bug 🐞 Something isn't working label Sep 13, 2023
@r-plus r-plus changed the title dependabot could not clone private repository for swift that located same organization dependabot could not clone private repository for swift that located same organization and granted access. Sep 13, 2023
@jakecoffman jakecoffman added F: private-registries 💂‍♂️ Issues about using private registries with Dependabot; may be paired with an R: label. L: swift Swift packages labels Oct 12, 2023
@tiwoc
Copy link

tiwoc commented Oct 24, 2023

I'm running into the same thing. @r-plus, have you found a workaround, by any chance?

@deivid-rodriguez
Copy link
Contributor

I think the fix for this should be a matter of replacing scp-style urls before shelling out to SwiftPM.

As a workaround, can you try replacing the url with an https url?

@tiwoc
Copy link

tiwoc commented Oct 24, 2023

It's the same issue with https URLs, unfortunately. Dependabot just seems to ignore that we've given it access to the private repo which contains the private dependency.

@deivid-rodriguez
Copy link
Contributor

deivid-rodriguez commented Oct 24, 2023

Did you replace it in both manifest and lockfile? Can you copy redacted logs, just to see what the errors with https look like?

@tiwoc
Copy link

tiwoc commented Oct 24, 2023

Yep, there's no trace of any non-https git URLs in our project.

Package.swift:

let package = Package(
[]
    dependencies: [
        .package(url: "https://github.com/redacted-org/redacted-repo", exact: "0.4.8"),
[]
    ]
[]
)

Package.resolved:

[]
    {
      "identity" : "redacted-repo",
      "kind" : "remoteSourceControl",
      "location" : "https://github.com/redacted-org/redacted-repo",
      "state" : {
        "revision" : "redacted-sha",
        "version" : "0.4.8"
      }
    },
[]

Logs:

[…]
  proxy | 2023/10/23 19:46:07 [104] GET https://github.com:443/redacted-org/redacted-repo/info/refs?service=git-upload-pack
  proxy | 2023/10/23 19:46:07 [104] * authenticating git server request (host: github.com)
  proxy | 2023/10/23 19:46:07 [104] 404 https://github.com:443/redacted-org/redacted-repo/info/refs?service=git-upload-pack
  proxy | 2023/10/23 19:46:07 [104] * auth'd git request returned 404, retrying without auth
  proxy | 2023/10/23 19:46:07 [104] * de-auth'd request returned 401, replacing response
[…]
updater | error: Failed to clone repository https://github.com/redacted-org/redacted-repo:
updater |     Cloning into bare repository '/home/dependabot/dependabot-updater/repo/Packages/.build/repositories/redacted-repo-aed36908'...
updater |     fatal: could not read Username for 'https://github.com': terminal prompts disabled
updater | 2023/10/23 19:48:31 ERROR <job_739621514> /home/dependabot/common/lib/dependabot/shared_helpers.rb:344:in `run_shell_command'
updater | 2023/10/23 19:48:31 ERROR <job_739621514> /home/dependabot/swift/lib/dependabot/swift/file_parser/dependency_parser.rb:39:in `formatted_deps'
updater | 2023/10/23 19:48:31 ERROR <job_739621514> /home/dependabot/swift/lib/dependabot/swift/file_parser/dependency_parser.rb:25:in `block (2 levels) in parse'
updater | 2023/10/23 19:48:31 ERROR <job_739621514> /home/dependabot/common/lib/dependabot/shared_helpers.rb:195:in `with_git_configured'
updater | 2023/10/23 19:48:31 ERROR <job_739621514> /home/dependabot/swift/lib/dependabot/swift/file_parser/dependency_parser.rb:24:in `block in parse'
updater | 2023/10/23 19:48:31 ERROR <job_739621514> /home/dependabot/common/lib/dependabot/shared_helpers.rb:42:in `block in in_a_temporary_repo_directory'
updater | 2023/10/23 19:48:31 ERROR <job_739621514> /home/dependabot/common/lib/dependabot/shared_helpers.rb:42:in `chdir'
updater | 2023/10/23 19:48:31 ERROR <job_739621514> /home/dependabot/common/lib/dependabot/shared_helpers.rb:42:in `in_a_temporary_repo_directory'
updater | 2023/10/23 19:48:31 ERROR <job_739621514> /home/dependabot/swift/lib/dependabot/swift/file_parser/dependency_parser.rb:21:in `parse'
updater | 2023/10/23 19:48:31 ERROR <job_739621514> /home/dependabot/swift/lib/dependabot/swift/file_parser.rb:18:in `parse'
updater | 2023/10/23 19:48:31 ERROR <job_739621514> /home/dependabot/dependabot-updater/lib/dependabot/dependency_snapshot.rb:99:in `parse_files!'
updater | 2023/10/23 19:48:31 ERROR <job_739621514> /home/dependabot/dependabot-updater/lib/dependabot/dependency_snapshot.rb:90:in `initialize'
updater | 2023/10/23 19:48:31 ERROR <job_739621514> /home/dependabot/dependabot-updater/lib/dependabot/dependency_snapshot.rb:22:in `new'
updater | 2023/10/23 19:48:31 ERROR <job_739621514> /home/dependabot/dependabot-updater/lib/dependabot/dependency_snapshot.rb:22:in `create_from_job_definition'
updater | 2023/10/23 19:48:31 ERROR <job_739621514> /home/dependabot/dependabot-updater/lib/dependabot/update_files_command.rb:17:in `perform_job'
updater | 2023/10/23 19:48:31 ERROR <job_739621514> /home/dependabot/dependabot-updater/lib/dependabot/base_command.rb:53:in `run'
updater | 2023/10/23 19:48:31 ERROR <job_739621514> bin/update_files.rb:24:in `<main>'
[…]

@tiwoc
Copy link

tiwoc commented Oct 25, 2023

Ok, there's a workaround that was helpfully suggested to me by GitHub support: with a GH access token in Dependabot secrets, we can register GH as a package registry and use that instead of relying on setting permissions in org settings (which doesn't seem to work for SPM, so this issue should still be kept open).

[…]
registries:
  github-private:
    type: git
    url: https://github.com
    username: x-access-token
    password: ${{secrets.GITHUB_TOKEN_NAME_REPLACE_ME}}

updates:
  - package-ecosystem: "swift"
    schedule:
      interval: "daily"
    registries:
      - github-private
[…]

@MattSkala
Copy link

Dependabot can now clone the private repository with the workaround suggested in #8027 (comment), but it still fails to authenticate when downloading a package from Github Package Registry (published via KMMBridge). Is this supported? I couldn't find any related info the docs.

proxy | 2024/01/10 22:13:36 [116] 401 [https://maven.pkg.github.com:443/[...].zip](https://maven.pkg.github.com/[...].zip)

@Wei18
Copy link

Wei18 commented Sep 30, 2024

It's the same issue with https URLs & env GITHUB_TOKEN || env GITHUB_DEPENDABOT_CRED_TOKEN...

url: "https://\(GITHUB_TOKEN)@github.com/\(repo).git"

url: "https://\(GITHUB_DEPENDABOT_CRED_TOKEN)@github.com/\(repo).git"

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
F: private-registries 💂‍♂️ Issues about using private registries with Dependabot; may be paired with an R: label. L: swift Swift packages T: bug 🐞 Something isn't working
Projects
None yet
Development

No branches or pull requests

6 participants