Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SwiftPM: Treating Package.resolved as a lock file #2212

Draft
wants to merge 2 commits into
base: main
Choose a base branch
from
Draft
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
238 changes: 238 additions & 0 deletions proposals/NNNN-swiftpm-package-resolved-lock-file.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,238 @@
# Treating `Package.resolved` as a lock file

* Proposal: [SE-NNNN](https://github.com/apple/swift-evolution/pull/2212/files?short_path=14ad3ef#diff-14ad3ef424e6ca9cf3077fd1a3c0e898a34a2b14523c1358edbbf6fa1a5def16)
* Authors: [Franz Busch](https://github.com/FranzBusch)
* Status: **Pitch**
* Implementation: Awaiting implementation

## Introduction

The Swift package manager is an essential part of the ecosystem and provides
functionality to download your dependencies, version them and build your
packages. Furthermore, the Swift package manager provides a locking mechanism
for dependencies to avoid accidental updates. Locking of dependencies has become
more relevant in the past years due to many different supply chain attacks in
multiple language ecosystems.

## Motivation

Since, the early days of the Swift package manager it had a couple of essential
commands to manage the dependencies of a package and building the package. These
commands are `swift package resolve`, `swift package update` and `swift build`. Over
the years, those commands gained more and more options to customize their
behaviour; however, there doesn't exist a clear definition when each of those
commands resolves or updates dependencies and how this interacts with the
presence or absence of a `Package.resolved` file. Furthermore, a lot of the
commands don't treat the `Package.resolved` file as a lock file and often update
its contents.

This proposal aims to define a clear behavior for the Swift package manager when
it resolves and updates dependencies.

## Proposed solution

### Deprecated: swift package resolve

We propose to deprecate this command since it is confusing in behavior. It
currently does both a resolution and download. Additionally, other commands may
trigger a resolution as well. We propose to replace it with swift package fetch
below.

### swift package fetch

Fetches the dependencies of a package.

#### Description

If a `Package.resolved` file is present, this command will fetch all dependencies
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: Restating swift package fetch instead of this command would make this sentence easier to read.

and make them locally available. Making them locally available means cloning the
git repositories or downloading the artefact. Subsequent commands will be able
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: did you mean artifacts?

to run offline unless the `Package.resolved` file changes. If the versions in the
`Package.resolved` cannot be fetched this command will fail with an error. If the
contents of the `Package.resolved` are not up-to-date with the contents of the
`Package.swift` this command will prompt the user to ask to update if run from an
interactive terminal otherwise it will fail with an error.

> Note: Determining if the `Package.resolved` is out of data is done by storing a
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: out of date*

hash of the `Package.swift` in the `Package.resolved` and comparing it when
executing the fetch command. Implemented in swift-package-manager/pull/6698
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: reference to a pull request is not a link in this form


If there is no `Package.resolved` file then this command will generate a new file
by resolving the dependency graph. Afterwards, it will fetch all dependencies.
Resolving the dependency graph might require to fetch the dependencies in the
case of git dependencies. Importantly, this will not resolve to the latest
versions since it might use the local or system wide cache.


#### Options

##### **--verbose/—very-verbose**
Produces verbose/very verbose output.

##### **--quiet**
Produces quiet output that does only produce warning or higher level
output.

##### **--package-path**
The path to the `Package.swift`. By default, Swift PM searches for
FranzBusch marked this conversation as resolved.
Show resolved Hide resolved
the `Package.swift` file in the current directory.

##### **--require-resolved-file**
Requires the `Package.resolved` file to be present. If the
lock resolved file is missing, Swift PM will exit with an error. This is useful
FranzBusch marked this conversation as resolved.
Show resolved Hide resolved
for CI systems to make sure only packages with `Package.resolved` files are being
built.

##### **--offline**
Prevents Swift PM to make any network requests. This is useful when
FranzBusch marked this conversation as resolved.
Show resolved Hide resolved
the dependencies have already been fetched into a system level cache and
validating that fetch is not making any network requests to grab dependencies
that are not available in the cache.

##### **--automatically-resolve-when-outdated**
Tells Swift PM to resolve the dependency
FranzBusch marked this conversation as resolved.
Show resolved Hide resolved
graph again when the `Package.resolved` file is out-of-date with the `Package.swift`
file. This will skip the prompt that normally happens. Important: This should be
avoided when building applications on CI systems since it might result in
different dependencies being included than the ones specified in the
`Package.resolved`.

##### **--ensure-mirrored**
Requires that every dependency has a configured mirror.

### swift package update

Updates the dependencies of a package.

#### Description

If a `Package.resolved` is present, this command will try to update it to the
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same not about this command as above.

latest package versions.

If there is no `Package.resolved` file, then this command will generate a new one
with the latest versions.

Updating the dependencies includes updating all of the git based dependencies
and pulling the latest state from the repositories. Afterwards, Swift PM will
FranzBusch marked this conversation as resolved.
Show resolved Hide resolved
resolve the dependency graph again and tries to update to the highest version
possible.

#### Options

##### **--verbose**
Produces verbose output and may be specified twice for very verbose
output.

##### **--quiet**
Produces quiet output that does only produce warning or higher level
output.

##### **--package-path**
The path to the `Package.swift`. By default, Swift PM searches for
FranzBusch marked this conversation as resolved.
Show resolved Hide resolved
the `Package.swift` file in the current directory.

##### **--offline**
Prevents Swift PM to make any network requests.
FranzBusch marked this conversation as resolved.
Show resolved Hide resolved

##### **--ensure-mirrored**
Requires that every dependency has a configured mirror.

##### **--package**
Only this package will be updated. Transitives will only be updated if
they must be updated. This flag can be specified multiple times.

##### **--dry-run**
Displays to which version the dependencies would be updated but
doesn’t generate a new `Package.resolved`.

### swift build/test/run changes
When a `Package.resolved` file is present then the build, test and run commands
should check whether the `Package.resolved` file is out-of-date with the
`Package.swift` . If the contents of the `Package.resolved` are not up-to-date with
the contents of the `Package.swift` this command will prompt the user to ask to
update if run from an interactive terminal otherwise it will fail with an error.
This follows how the proposed swift package fetch is working; hence, we also
propose to add a --automatically-resolve-when-outdated to build/test/run.

If no `Package.resolved` is present then these commands should first trigger swift
package fetch before running the actual command.

The following options will be removed from the respective commands:

* `--force-resolved-versions`, `--disable-automatic-resolution`,
`--only-use-versions-from-resolved-file` : This is the new default
* `--skip-update` : This currently only skips the remote update similar to the
newly proposed `--offline` mode

## Compatibility

Changing the behaviour of CLI commands is in general a breaking change for any
tool that uses those commands; however, the new behaviour are gated by upgrading
the Swift version. Tooling that was using the existing commands would need to be
updated to the new semantics. Overall, we believe this is a one-time trade worth
doing to increase the security.

## Migration

Scripts and workflows have to migrated when adopting the new Swift version that
includes the updated commands. The biggest change is the removal of swift
package resolve . Any script that was currently using swift package resolve
needs to evaluate why it was using it and either migrate to swift package fetch
or swift package update.
FranzBusch marked this conversation as resolved.
Show resolved Hide resolved

The build/run/test commands are staying mostly the same and we expect very few
FranzBusch marked this conversation as resolved.
Show resolved Hide resolved
people have to adapt their scripts. The biggest difference is the check that the
`Package.resolved` file is up-to-date which might requires users to add a
FranzBusch marked this conversation as resolved.
Show resolved Hide resolved
`--automatically-resolve-when-outdated`.
FranzBusch marked this conversation as resolved.
Show resolved Hide resolved

## Future directions

### Target/Host specific dependency resolution

In the future, we can extend the the fetch command to be able to fetch
FranzBusch marked this conversation as resolved.
Show resolved Hide resolved
dependencies for specific target/host triplets. This however, is something that
FranzBusch marked this conversation as resolved.
Show resolved Hide resolved
requires changes to the Package manifest and is outside the scope of this
FranzBusch marked this conversation as resolved.
Show resolved Hide resolved
proposal.

### Vendor command

Vendoring dependencies locally is a common request and is closely related to
dependency resolution. However, this should be tackled in a separate proposal
since it also has to take into account how vendors dependencies are built.

### Minimum version resolution

Packages are declaring their dependencies using minim version in SemVer format;
FranzBusch marked this conversation as resolved.
Show resolved Hide resolved
however, during resolution the latest possible version that still satisfies all
constraints is chosen. In practice, this leads to most developers using the
latest version of packages. Hence, almost nobody is building packages with
minimum versions of their dependencies. This often leads to the fact that the
minimum versions that library packages declare are not working. In the future,
we could provide a command to force resolution to pick the lower boundary. This
would allow library authors to test if their packages work with their lowest
declared dependency versions.

### `--precise` and `--aggressive` options for swift package update

In the future, we can add two more options to swift package update to fine tune
FranzBusch marked this conversation as resolved.
Show resolved Hide resolved
to which version a package is updated and how transitive dependencies are
updated. However, adding those options can be done additively once we changed
the semantics.

`--aggressive` Used together with `--package` and forces all transitive dependencies
to be updated as well. This applies to all packages specified via `--package`
flags.

`--precise` Used together with `--package` and allows to specify a specific version
to update the package to. This applies to all packages specified via `--package`
flags.

## Alternatives considered

### Conditionally enable new behaviour on swift-tools-version

We could make the new behaviour only apply when a certain swift-tools-version
has been set; however, this would bring enormous complexity to Swift PM since we
FranzBusch marked this conversation as resolved.
Show resolved Hide resolved
allow to pass a package-path to the various commands and we would need to do a
two step argument parsing depending on what tools version is in the
`Package.swift`.