Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Swift Package Manager Support? #1245

Closed
satishbabariya opened this issue Jul 4, 2019 · 41 comments
Closed

Swift Package Manager Support? #1245

satishbabariya opened this issue Jul 4, 2019 · 41 comments
Labels
T: new-ecosystem Requests for new ecosystems/languages

Comments

@satishbabariya
Copy link

satishbabariya commented Jul 4, 2019

Is there any plans for spm support?, some of the following details may help

  1. use Package.swift in file_fetcher
  2. use .package(url: "https://github.com/abc/abc.git", from: "1.0.0) for dependency in file_parser

Listing Dependency using Command Line (exec)

swift package dump-package --type json

For Package Metadata

swift package describe --type json
  1. use git tag for checking new version update

I don't know much ruby, but I can help in creating specific dependency updater for package file.

@greysteil
Copy link
Contributor

We'd love to add spm support. Are there any docs and/or example Package.swift files you can link to?

@satishbabariya
Copy link
Author

here is the documentation on spm https://github.com/apple/swift-package-manager/tree/master/Documentation

and here is the example of package file https://github.com/vapor/vapor/blob/master/Package.swift

@satishbabariya
Copy link
Author

satishbabariya commented Jul 5, 2019

also here is the dockerfile for test or developement in case

FROM ubuntu:18.04

RUN export DEBIAN_FRONTEND=noninteractive DEBCONF_NONINTERACTIVE_SEEN=true && apt-get -q update && \
    apt-get -q install -y \
    libatomic1 \
    libbsd0 \
    libcurl4 \
    libxml2 \
    tzdata \
    && rm -r /var/lib/apt/lists/*

ARG SWIFT_PLATFORM=ubuntu18.04
ARG SWIFT_BRANCH=swift-5.0.1-release
ARG SWIFT_VERSION=swift-5.0.1-RELEASE

ENV SWIFT_PLATFORM=$SWIFT_PLATFORM \
    SWIFT_BRANCH=$SWIFT_BRANCH \
    SWIFT_VERSION=$SWIFT_VERSION

RUN SWIFT_URL=https://swift.org/builds/$SWIFT_BRANCH/$(echo "$SWIFT_PLATFORM" | tr -d .)/$SWIFT_VERSION/$SWIFT_VERSION-$SWIFT_PLATFORM.tar.gz \
    && curl -fSsL $SWIFT_URL -o swift.tar.gz \
    && export GNUPGHOME="$(mktemp -d)" \
    && tar -xzf swift.tar.gz --directory / --strip-components=1 \
    && rm -r "$GNUPGHOME" swift.tar.gz \
    && chmod -R o+r /usr/lib/swift

# Print Installed Swift Version
RUN swift --version

@stale stale bot added the wontfix label Oct 23, 2019
@feelepxyz feelepxyz added T: feature-request Requests for new features and removed wontfix labels Oct 23, 2019
@infin8x infin8x added T: new-ecosystem Requests for new ecosystems/languages and removed T: feature-request Requests for new features labels Jul 2, 2020
@orj
Copy link

orj commented Oct 15, 2020

Gonna add my voice to those calling for Swift Package Manager support in Dependabot.

@lyzkov
Copy link

lyzkov commented Feb 21, 2021

Where is SPM hook for pushing updatedeps commit to git repo? Where is a bot here? Or dependabot is little bit more advanced yet powerful?

@apps4everyone
Copy link

Any progress here? Would be great to have SPM soon....

@greysteil
Copy link
Contributor

Cc @feelepxyz and @asciimike for awareness. I know the Dependabot team are heads down on some other features and not planning to expand to new ecosystems imminently - they'll have the latest, though.

@dependabot dependabot deleted a comment from kurgol May 24, 2021
@asciimike
Copy link
Contributor

Xref #3772

@loganblevins
Copy link

Bumping for SPM vote

@juliensechaud
Copy link

up !

@Panajev
Copy link

Panajev commented Sep 6, 2022

Is there any update or any roadmap on this? Speaking as a GitHub Enteprise customer that will get this months behind the .com version so the delay in announcing anything for the regular versions are even more painful :/.

@dependabot dependabot deleted a comment from stale bot Sep 12, 2022
@jeffwidman
Copy link
Member

Related: #5562

We are starting to get a little more caught up on our backlog / infra tasks as a team, so I expect in the not too distant future we'll start looking at adding ecosystems again. But I'm afraid we don't have anything specific to share just yet.

In the meantime, if Swift support is important to you, please 👍 the original issue... it helps us gauge the popularity of different ecosystems that are requested. No need to add a comment, that just spams the folks following this issue for updates.

@0xTim
Copy link
Contributor

0xTim commented Feb 1, 2023

@jeffwidman any news to share here - we're looking at picking this up to get the support over the line so interested to know if it's worth spending the effort on

@afern247
Copy link

Any update on this?

@baptistedajon
Copy link

Interested to have updates too :)

@daytime-em
Copy link

Putting myself down as interested in SwiftPM support!

@o-nnerb
Copy link

o-nnerb commented Apr 15, 2023

I'm waiting for it! 🔎

@afern247
Copy link

Same, can we add support please

@deivid-rodriguez
Copy link
Contributor

Please, if your comment is just a reaction to show support for this feature, do use reactions! See https://github.com/dependabot/dependabot-core#no-1-comments.

@deivid-rodriguez
Copy link
Contributor

Hei!

Just some heads up that we're actively working on this and plan to ship Swift support during this quarter.

@loganblevins
Copy link

@deivid-rodriguez Thank you! Wonderful news.

@caiofbpa
Copy link

@deivid-rodriguez please note that there's a difference in the way Swift Libraries/Executables and Swift Apps deal with SwiftPM:

Swift Libraries/Executables (Libraries, Command Line Tools, Server-side Swift)

  • Have a Package.swift file defining dependency version rules
  • Have a Package.resolved file with the "pins" for specific package versions actually used
  • Interactions are mostly editing Package.swift and running swift package resolve or swift package update

Swift Apps (iOS, macOS)

  • Have a *.xcodeproj/project.pbxproj file defining dependency version rules (among other project stuff)
  • Have a *.xcodeproj/project.xcworkspace/xcshareddata/swiftpm/Package.resolved file with the "pins" for specific package versions actually used
  • Interactions are mostly through Xcode manually editing version rules and clicking at Update to Latest Package Versions

Adding support to Swift Libraries/Executables should be trivial, but supporting Swift Apps doesn't look easy to automate. If you could confirm what kind of support is being worked on, that could help some of us to set our expectations in terms of use cases.

@deivid-rodriguez
Copy link
Contributor

Yes, we're only working in the first case as of now.

@pyrtsa
Copy link

pyrtsa commented Jul 12, 2023

  • Interactions are mostly through Xcode manually editing version rules and clicking at Update to Latest Package Versions

I think you can automate this part as well by using the xcodebuild -resolvePackageDependencies command with some further options.

@deivid-rodriguez
Copy link
Contributor

Hello!

We have enabled beta testing for Swift Package Manager version updates. To opt-in to it, you need to specify the enable-beta-ecosystems: true top level configuration, and swift as the package ecosystem. Like this

version: 2
enable-beta-ecosystems: true
updates:
  - package-ecosystem: "swift"
    directory: "/"
    schedule:
      interval: "weekly"

Feel free to try it out and report any issues you find.

Note that there's a known issue with a specific Swift dependency where we end up adding some unnecessary changes to the Package.resolved file. Swift should deal with it just fine but I can see how it could be annoying. No need to report that one issue if you find it since we already know about it :)

Thanks!

@redfearnk
Copy link

redfearnk commented Jul 25, 2023

Have you completed the Xcode project file version or only the Package.swift version, as outlined above? For what it's worth, I think MOST users will integrate through the Xcode project file and not through the Package.swift file

@deivid-rodriguez
Copy link
Contributor

Only through Package.swift for now. Will open a separate issue for Xcode projects once we close this one and release the initial version supporting SwiftPM.

@0xTim
Copy link
Contributor

0xTim commented Jul 26, 2023

Supporting Xcode apps is going to be hard, if not impossible. There's a resolved file, but the location of it is liable to move on any Xcode update. The manifest itself is stored in a the project.pbxproj in a custom format. That format (and location) is not defined anywhere authoritative and can change at any point (and has done so in the past)

@redfearnk
Copy link

Hopefully Apple will make the Xcode version better in the future. I don't see any reason why they can't just use a Package.swift

@pyrtsa
Copy link

pyrtsa commented Jul 26, 2023

I can't recommend this, but as a potential workaround, it seems to me swift package commands do the right thing in an Xcode project if you just create a symlink next to the package file pointing to your project's *.xcodeproj/project.xcworkspace/xcshareddata/swiftpm/Package.resolved.

@0xTim
Copy link
Contributor

0xTim commented Jul 27, 2023

Hopefully Apple will make the Xcode version better in the future. I don't see any reason why they can't just use a Package.swift

I can't speak for Apple but given the new package format introduced in playgrounds for iPad to build 'real' apps I suspsect we'll get there but it's going to be a slow process to migrate everyone and all all the additional features required to SwiftPM

@timdawborn
Copy link

timdawborn commented Jul 28, 2023

@deivid-rodriguez Where would you like bug reports for this beta release — here, or in another issue? We've been struggling to get this to run without crashing on our iOS app. The stacktrace from Dependabot is currently:

updater | 2023/07/28 04:22:26 ERROR <job_701761721> error: 'repo': Source files for target OurDependencies should be located under 'Sources/OurDependencies', or a custom sources path can be set with the 'path' property in Package.swift
updater | 2023/07/28 04:22:26 ERROR <job_701761721> /home/dependabot/common/lib/dependabot/shared_helpers.rb:345:in `run_shell_command'
updater | 2023/07/28 04:22:26 ERROR <job_701761721> /home/dependabot/swift/lib/dependabot/swift/file_parser/dependency_parser.rb:37:in `formatted_deps'
updater | 2023/07/28 04:22:26 ERROR <job_701761721> /home/dependabot/swift/lib/dependabot/swift/file_parser/dependency_parser.rb:23:in `block (2 levels) in parse'
updater | 2023/07/28 04:22:26 ERROR <job_701761721> /home/dependabot/common/lib/dependabot/shared_helpers.rb:187:in `with_git_configured'
updater | 2023/07/28 04:22:26 ERROR <job_701761721> /home/dependabot/swift/lib/dependabot/swift/file_parser/dependency_parser.rb:22:in `block in parse'
updater | 2023/07/28 04:22:26 ERROR <job_701761721> /home/dependabot/common/lib/dependabot/shared_helpers.rb:41:in `block in in_a_temporary_repo_directory'
updater | 2023/07/28 04:22:26 ERROR <job_701761721> /home/dependabot/common/lib/dependabot/shared_helpers.rb:41:in `chdir'
updater | 2023/07/28 04:22:26 ERROR <job_701761721> /home/dependabot/common/lib/dependabot/shared_helpers.rb:41:in `in_a_temporary_repo_directory'
updater | 2023/07/28 04:22:26 ERROR <job_701761721> /home/dependabot/swift/lib/dependabot/swift/file_parser/dependency_parser.rb:19:in `parse'
updater | 2023/07/28 04:22:26 ERROR <job_701761721> /home/dependabot/swift/lib/dependabot/swift/update_checker/version_resolver.rb:40:in `fetch_latest_resolvable_version'
updater | 2023/07/28 04:22:26 ERROR <job_701761721> /home/dependabot/swift/lib/dependabot/swift/update_checker/version_resolver.rb:20:in `latest_resolvable_version'
updater | 2023/07/28 04:22:26 ERROR <job_701761721> /home/dependabot/swift/lib/dependabot/swift/update_checker.rb:66:in `fetch_latest_resolvable_version'
updater | 2023/07/28 04:22:26 ERROR <job_701761721> /home/dependabot/swift/lib/dependabot/swift/update_checker.rb:21:in `latest_resolvable_version'
updater | 2023/07/28 04:22:26 ERROR <job_701761721> /home/dependabot/common/lib/dependabot/update_checkers/base.rb:75:in `preferred_resolvable_version'
updater | 2023/07/28 04:22:26 ERROR <job_701761721> /home/dependabot/common/lib/dependabot/update_checkers/base.rb:263:in `preferred_version_resolvable_with_unlock?'
updater | 2023/07/28 04:22:26 ERROR <job_701761721> /home/dependabot/common/lib/dependabot/update_checkers/base.rb:255:in `numeric_version_can_update?'
updater | 2023/07/28 04:22:26 ERROR <job_701761721> /home/dependabot/common/lib/dependabot/update_checkers/base.rb:205:in `version_can_update?'
updater | 2023/07/28 04:22:26 ERROR <job_701761721> /home/dependabot/common/lib/dependabot/update_checkers/base.rb:45:in `can_update?'
updater | 2023/07/28 04:22:26 ERROR <job_701761721> /home/dependabot/dependabot-updater/lib/dependabot/updater/operations/update_all_versions.rb:204:in `requirements_to_unlock'
updater | 2023/07/28 04:22:26 ERROR <job_701761721> /home/dependabot/dependabot-updater/lib/dependabot/updater/operations/update_all_versions.rb:88:in `check_and_create_pull_request'
updater | 2023/07/28 04:22:26 ERROR <job_701761721> /home/dependabot/dependabot-updater/lib/dependabot/updater/operations/update_all_versions.rb:59:in `check_and_create_pr_with_error_handling'
updater | 2023/07/28 04:22:26 ERROR <job_701761721> /home/dependabot/dependabot-updater/lib/dependabot/updater/operations/update_all_versions.rb:34:in `block in perform'
updater | 2023/07/28 04:22:26 ERROR <job_701761721> /home/dependabot/dependabot-updater/lib/dependabot/updater/operations/update_all_versions.rb:34:in `each'
updater | 2023/07/28 04:22:26 ERROR <job_701761721> /home/dependabot/dependabot-updater/lib/dependabot/updater/operations/update_all_versions.rb:34:in `perform'
updater | 2023/07/28 04:22:26 ERROR <job_701761721> /home/dependabot/dependabot-updater/lib/dependabot/updater.rb:63:in `run'
updater | 2023/07/28 04:22:26 ERROR <job_701761721> /home/dependabot/dependabot-updater/lib/dependabot/update_files_command.rb:38:in `perform_job'
updater | 2023/07/28 04:22:26 ERROR <job_701761721> /home/dependabot/dependabot-updater/lib/dependabot/base_command.rb:52:in `run'
updater | 2023/07/28 04:22:26 ERROR <job_701761721> bin/update_files.rb:23:in `<main>'

In our repo, there is indeed a OurDependencies/Sources/OurDependencies.swift file, so this error was initially rather confusing. The stacktrace shows that it's coming from running swift package show-dependencies --format json from inside the OurDependencies folder (which is what is set as the directory in our swift entry in .github/dependabot.yml).

The error itself looks to be coming from SPM. The logic here in Dependabot appears to only be writing the dependency files, which I'm guessing might only be the Package.* files and might not include the Sources directory?

Locally, going into OurDependencies and removing the Sources folder entirely, I can reproduce this error message from SPM:

$ cd ~/repository/
$ cat .github/dependabot.yml
version: 2
enable-beta-ecosystems: true
updates:
  - package-ecosystem: swift
    directory: "/OurDependencies/"
    open-pull-requests-limit: 99
    schedule:
      interval: daily
      time: "05:00"
      timezone: Australia/Sydney
$ cd OurDependencies
$ git ls-tree --name-only -r HEAD
.gitignore
Package.resolved
Package.swift
README.md
Sources/OurDependencies/OurDependencies.swift
$ swift package show-dependencies --format json > /dev/null ; echo $?
0
$ rm -rf Sources
$ swift package show-dependencies --format json > /dev/null ; echo $?
error: 'ourdependencies': Source files for target OurDependencies should be located under 'Sources/OurDependencies', or a custom sources path can be set with the 'path' property in Package.swift
1
$

Is this expected behaviour?

@deivid-rodriguez
Copy link
Contributor

Hi @timdawborn, thanks for digging into this!

I think a new issue is better, can you copy this to a new issue? If you can also create a small public repository that reproduces it, it would be perfect. In principle, it seems unexpected because the full clone of the repo should be present at repo_contents_path, we only rewrite manifest changes on top of it to figure out updates.

@caiofbpa
Copy link

I'm trying @pyrtsa approach, it sort of works but I need to manually edit every Dependabot PR to make sure both Package.swift and .xcodeproj files point to the same version, every time.

I found this ruby gem that can read and write .xcodeproj bundles, this could help Dependabot deal with the Xcode projects.

Gem: https://github.com/CocoaPods/Xcodeproj
Specific class that deals with SwiftPM dependencies and versions: https://www.rubydoc.info/gems/xcodeproj/Xcodeproj/Project/Object/XCRemoteSwiftPackageReference

Hopefully this helps the implementation to cater for a wider user base!

@deivid-rodriguez
Copy link
Contributor

@timdawborn I ended up being able to reproduce your issue and has been fixed at #7674. Try again and hopefully should just work now.

For everyone else, we've just released Swift support for Version & Security Updates, so let me close this ticket since this has been implemented 🎉.

I'll make sure to open separate feature requests for the things that we left out of this release, and please do report any issues you find.

Cheers!

@timdawborn
Copy link

@deivid-rodriguez Thanks! It's now working like a charm for us 🎉

Screenshot 2023-08-02 at 16 53 07

@deivid-rodriguez
Copy link
Contributor

Glad to hear that! 🎉

@caiofbpa
Copy link

caiofbpa commented Aug 2, 2023

@deivid-rodriguez could you link here the additional issues you created for things left out of this release? Especially the support for using .xcodeproj project bundles are source for version definitions instead of Package.swift?

@deivid-rodriguez
Copy link
Contributor

I had not yet opened that issue, there you go! #7694

@macfanatic
Copy link

Is there any special configuration needed to use updates to internal private packages published on Github via SPM?

We have several libraries that depend upon other smaller internal libraries, just as private repos in our organization. These are specified via HTTPS URLs in our Package.swift file.

Eagerly awaiting support for the app project file itself as well in #7694 :)

@deivid-rodriguez
Copy link
Contributor

I guess configure a git registry.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
T: new-ecosystem Requests for new ecosystems/languages
Projects
Archived in project