0.11 cannot build with package manager, 0.10 can

[rezahussain]

Hello,

It looks like s4tf projects cannot build from the command line if they use the package manager in 0.11, but they do work in 0.10.

This issue is only affecting ubuntu 18.04 LTS, I tried this same project on my mac with 0.11 and it worked ok
test_s4tf.zip

/home/admin2/usr/bin/swift run s4tf_test
Fetching https://github.com/httpswift/swifter.git
Fetching https://github.com/IBM-Swift/BlueSocket.git
Fetching https://github.com/codewinsdotcom/PostgresClientKit
Fetching https://github.com/IBM-Swift/BlueSSLService
Cloning https://github.com/codewinsdotcom/PostgresClientKit
Resolving https://github.com/codewinsdotcom/PostgresClientKit at 1.3.0
Cloning https://github.com/httpswift/swifter.git
Resolving https://github.com/httpswift/swifter.git at 4d89ac464d2d3b931cadc4a13696b32b2fd0dd3e
Updating https://github.com/httpswift/swifter.git
Updating https://github.com/codewinsdotcom/PostgresClientKit
Updating https://github.com/IBM-Swift/BlueSSLService
Updating https://github.com/IBM-Swift/BlueSocket.git
Everything is already up-to-date
error: the Package.resolved file is most likely severely out-of-date and is preventing correct resolution; delete the resolved file and try again

I deleted Package.resolved but it still happens on my ubuntu box. Please let me know if I am missing something obvious or if there is any more info I can provide that will help to isolate this problem.

Thanks

[compnerd]

Thanks for the report, we will look into it soon. It isn't immediately clear why swift-package-manager is unable to resolve the dependency set.

[brettkoonce]

@rezahussain i ran into this when mixing/matching 0.11 and master. clear out your build cache (eg delete .build/* in the folder you are running spm in) and things will hopefully start working correctly. I would suggest just using nightlies until 0.12 is released.

[rezahussain]

Hello @brettkoonce

I just downloaded v11 and nightly again to make sure and did the test again

v0.11 test UBUNTU 18.04 CPU ONLY

(base) admin2@msimegx570:~/dev_projects/test_s4tf/Sources$ rm ../Package.resolved
(base) admin2@msimegx570:~/dev_projects/test_s4tf/Sources$ rm -rf .build
(base) admin2@msimegx570:~/dev_projects/test_s4tf/Sources$ /home/admin2/dev_projects/s4tf_v11/usr/bin/swift run test_s4tf
Updating https://github.com/httpswift/swifter.git
Updating https://github.com/IBM-Swift/BlueSSLService
Updating https://github.com/IBM-Swift/BlueSocket.git
Updating https://github.com/codewinsdotcom/PostgresClientKit
Cloning https://github.com/IBM-Swift/BlueSocket.git
Resolving https://github.com/IBM-Swift/BlueSocket.git at 1.0.52
Cloning https://github.com/IBM-Swift/BlueSSLService
Resolving https://github.com/IBM-Swift/BlueSSLService at 1.0.52
Updating https://github.com/httpswift/swifter.git
Updating https://github.com/codewinsdotcom/PostgresClientKit
Updating https://github.com/IBM-Swift/BlueSocket.git
Updating https://github.com/IBM-Swift/BlueSSLService
Everything is already up-to-date
error: the Package.resolved file is most likely severely out-of-date and is preventing correct resolution; delete the resolved file and try again
(base) admin2@msimegx570:~/dev_projects/test_s4tf/Sources$ 

test UBUNTU 18.04 CPU ONLY Nightly development

(base) admin2@msimegx570:~/dev_projects/test_s4tf/Sources$ rm ../Package.resolved
(base) admin2@msimegx570:~/dev_projects/test_s4tf/Sources$ rm -rf .build
(base) admin2@msimegx570:~/dev_projects/test_s4tf/Sources$ /home/admin2/dev_projects/s4tf_nightly/usr/bin/swift run test_s4tf
Updating https://github.com/httpswift/swifter.git
Updating https://github.com/IBM-Swift/BlueSocket.git
Updating https://github.com/codewinsdotcom/PostgresClientKit
Updating https://github.com/IBM-Swift/BlueSSLService
Everything is already up-to-date
Updating https://github.com/httpswift/swifter.git
Updating https://github.com/codewinsdotcom/PostgresClientKit
Updating https://github.com/IBM-Swift/BlueSSLService
Updating https://github.com/IBM-Swift/BlueSocket.git
Removing https://github.com/IBM-Swift/BlueSocket.git
Removing https://github.com/IBM-Swift/BlueSSLService
error: the Package.resolved file is most likely severely out-of-date and is preventing correct resolution; delete the resolved file and try again
(base) admin2@msimegx570:~/dev_projects/test_s4tf/Sources$ 

This happens on both 0.11 and the nightly at the time of this writing. Let me know if I should do more tests, I also attached the helloworld project(s4tf_test.zip) above.

@compnerd good to hear please let me know what you find.

[BradLarson]

This appears to be only impacting swift run and not swift build.

I encountered this with even a simple project with the following package description:

// swift-tools-version:5.3
// The swift-tools-version declares the minimum version of Swift required to build this package.

import PackageDescription

let package = Package(
    name: "testproject",
    products: [
    ],
    dependencies: [
        .package(url: "https://github.com/tensorflow/swift-models.git", .branch("master")),
    ],
    targets: [
        .target(
            name: "testproject",
            dependencies: [.product(name: "Datasets", package: "swift-models")]),
    ]
)

A reproducer project is attached that exhibits this error when you run swift run testproject. swift build will build this project correctly, just not swift run:
testproject.zip

[brettkoonce]

@BradLarson that works here locally with a nightly for me, fwiw
can reproduce the error with reza's original project
i ran into this the first time ~a month ago, it came back again later, it went away by navigating to the swift-models checkout (eg .build/checkouts/swift-models) and doing a git pull origin master which reset something internally iirc
the only difference in my manifest is i have the

  platforms: [
    .macOS(.v10_13),
  ],

[BradLarson]

@brettkoonce - The nightly that this is working with on your end, is that a macOS nightly? I still get the same error with swift run on our latest Ubuntu nightlies, but haven't tested with a macOS nightly. If it works there, that could be another clue.

[brettkoonce]

@BradLarson ubuntu 18.04 over here, using cuda 10.2 nightlies

[compnerd]

This seems like an issue with swift-package-manager. I see the same behaviour with the latest snapshot of the toolchain (https://swift.org/builds/development/ubuntu1804/swift-DEVELOPMENT-SNAPSHOT-2020-09-17-a/swift-DEVELOPMENT-SNAPSHOT-2020-09-17-a-ubuntu18.04.tar.gz). I think that this is probably best reported to bugs.swift.org.

[compnerd]

Some more details: it is not present in the 5.3 release, but it is present in the oldest snapshot that I see on the website (7/21/20).

[brettkoonce]

if there's reproducer, then yeah go upstream. my notes say i last saw this 8/16, but i remember hitting it randomly ~a month prior to that.
rebuilt from scratch locally + works with with nightly (wget https://storage.googleapis.com/swift-tensorflow-artifacts/nightlies/latest/swift-tensorflow-DEVELOPMENT-cuda10.2-cudnn7-ubuntu18.04.tar.gz):

swift run
Fetching https://github.com/tensorflow/swift-models.git
Fetching https://github.com/google/swift-benchmark
Fetching https://github.com/apple/swift-protobuf.git
Fetching https://github.com/apple/swift-argument-parser
Cloning https://github.com/apple/swift-argument-parser
Resolving https://github.com/apple/swift-argument-parser at 0.2.2
Cloning https://github.com/apple/swift-protobuf.git
Resolving https://github.com/apple/swift-protobuf.git at 1.12.0
Cloning https://github.com/tensorflow/swift-models.git
Resolving https://github.com/tensorflow/swift-models.git at master
Cloning https://github.com/google/swift-benchmark
Resolving https://github.com/google/swift-benchmark at f70bf472b00aeaa05e2374373568c2fe459c11c7
[135/135] Linking testproject
2020-09-21 22:47:31.693195: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcudart.so.10.2
Hello

[brettkoonce]

and yes spm would be my suspicion

[compnerd]

Filed on JIRA as https://bugs.swift.org/browse/SR-13590

[marcrasi]

A workaround that works for me is to run swift build first, and then run the swift run command that I want.

[rezahussain]

Thanks to everyone!