Publish CLI apps with pubspec.lock, and use it when `global activate`ing.

[bsutton]

I wrote a blog on this one if you want a bit more detail but here is the summary.

https://onepub.dev/Blog?id=fvvuhnofly

The dart guidelines for pubspec dependencies recommend using a version range

dependencies:
  dcli: ^1.0.0

This works great if you are compiling a flutter app.

The version range lets pub find a version that meets all the requirements of various overlapping dependencies and then
the process of compiling your flutter app locks those dependencies to a particular version (i.e. you ship code that will be installed with a know version that you have tested against).

The problem with CLI apps is that when they are installed from pub.dev the version of any particular package is decided at the point of installation.

This means that your CLI app can be installed against versions of dependencies that you haven't tested against.

I've experienced this in the real world on numerous occasions so it's not a theoretical concern.

My suggestion is to make the following changes:

If the package is a CLI app (contains one or more keys under the 'executable' section in pubspec.yaml)

Then publish the pubspec.lock file along with the package.

When installing the CLI app use the lock file to determine what versions to link against the CLI app.

If the package also contains a public api (both dcli and fvm) then use the normal version ranges present in the pubspec.yaml (i.e. ignore the pubspec.lock as we do now) when the package is linked into third party app.

I don't believe this change will break the existing environment and will allow us to create stable CLI apps with dart.

[sigurdm]

Using the pubspec.lock for global activate has both pros and cons.

Pro:

• You run what the developer has tested working.

Against:

• You won't get the benefit of updates in dependencies before the package is uploaded with a new pubspec.lock
• publishing a working pubspec.lock is not trivial, and we might have to teach package authors stuff.
  • if you have dependency overrides to local packages, then they will break when used in a different context
  • if you have git-dependency_overrides they will break the usual pub.dev convention of only having hosted dependencies.

This is something we have been thinking of from time to time. We are still not sure about the answer.

One idea would be enabling global activate <URL> pointing to a pubspec.lock published on the side somewhere - but that is also a bit ugly.

[bsutton]

I actually read the cons as pros.

No package updates is exactly what is required to ensure a stable cli app.
Compiled apps have this advantage now and I would look to have cli apps treated in the same way; All dependencies are fixed at the point of publishing.

As to overrides and git references i would see the same rules as apply to a pubspec, only hosted packages are allowed.

[jakemac53]

If you want to ensure specific versions, I would just pin them in the pubspec instead of using version ranges. This has its own disadvantages, and I wouldn't do it, but it has the same semantics as respecting the pubspec.lock.

[bsutton]

@jakemac53

I would just pin them in the pubspec instead of using version ranges.

This is exactly what I do now, but its an ugly solution as it requires you to pin every package including transitive dependencies.
It also requires that you know not pinning your package will cause problems.

Shipping the pubspec.lock file, provides an automatic mechanism that delivers what the publisher expected.

The concept of 'it does what you would expect' is one of the great strengths of Dart, the current process doesn't do that.

Imagine if a released flutter app resolved its package versions when the user installed it. It would be complete chaos all of your tests would be invalidated because there is no way you can test every combination of packages or future packages that might be released.

The Windows 'DLL hell' came from this very problem. When shipping a windows app in the early days you could never be sure which version of a dll your app would run with.
The solution was to either statically link your windows app or as Microsoft now recommends every app ships its own copies of dlls.

These lessons have been learned the hard way. For Dart CLI apps we seem to have forgotten this.

[jakemac53]

That is true that for transitive deps it is a pain to manually pin them, and just relying on pub to get an initial version solve is a lot easier.

I do agree with you that there are valid reasons to generally want to pin dependencies for a globally activated app (along with cons).

[bsutton]

From google's own documentation:

Don’t commit the following files and directories created by pub:

pubspec.lock  # Except for application packages

The pubspec.lock file is a special case, similar to Ruby’s Gemfile.lock.

For application packages, we recommend that you commit the pubspec.lock file. Versioning the pubspec.lock file ensures changes to transitive dependencies are explicit.

Why would google make this recommendation if always having the latest dep version is a good thing.

They do this so that each developer is working with a know set of package versions and the test environment is also working against the same set of dependencies.

If always having the latest version was actually a good then then committing the pubspec.lock file would not be recommended.

Back in 2003 microsoft solved the dll hell problem by:

Microsoft .Net 1.1, which will be integral to the new Windows Server 2003 operating systems, will support what the company calls "strong binding," said Salmre. "Strong binding means an application or component can bind to a specific version of another component, so you can reuse components or use them in isolation."

That sound rather like shipping a pubspec.lock file with a cli app.

[bsutton]

I'm cross posting from dart-lang/test#2400 for visibility:

Let's try to think about it in a different way.

What would the reaction be from the flutter community be if we said:

Hey we've got this new release process.
When your users deploy a flutter app to their phone, we resolve the package dependencies at that point in time.
This is great because if another flutter app is already installed, the two can share a package.
This way your release bundles are smaller
Your users get the benefit of the latest version of any package, even after you release

At this point your users would remind you of a thing call 'Windows DLL Hell'.

Currently, with respect to CLI apps, the Dart eco system is subjecting developers to the equivalent of DLL Hell.

[bsutton]

Here is another thought on how to think about the problem.

Scenario

We have just finished writing a CLI app and all of the required unit/system/integration tests.

Our CLI tool depends on package 'handy' 1.0.0.

The day before we release our CLI app, 'handy' 1.0.1. is released

Do you:

a) upgrade to handy 1.0.1 and re-run all tests before releasing
b) upgrade to handy 1.0.1 and release without testing

If you answer a) then you are voting to ship pubspec.lock with our CLI apps.

If you answered b) then sorry you don't get the job, because no-one would release an app after a package upgrade without testing it.

b) is what is currently happening with dart CLI apps when a third party publishes an upgraded version of handy after we release.

If you already have a job, choosing option b) is likely to get you sacked.

Why? because as developers we know that stability is far more critical than some possible bug fix, that is just as likely to cause our app to crash as it is to improvement our app.

[natebosch]

This means that your CLI app can be installed against versions of dependencies that you haven't tested against.

If you are worried about the package being installed against different versions of dependencies, are you also worried about it being installed against different versions of the SDK?

I've experienced this in the real world on numerous occasions so it's not a theoretical concern.

Were these places where semver was not followed correctly? Is there more we can do to help package authors follow semver? What types of bugs caused your users pain?

When installing the CLI app use the lock file to determine what versions to link against the CLI app.

There is no guarantee that the pub solve in the lock file will work for the user downloading the package. It would only be guaranteed if

• All packages in the lockfile are compatible with the users version of the SDK.
• No packages have been revoked.

The first criteria is the much more interesting one.
What would we do if the pinned packages can't be used? Refuse to install? Fall back on a pub solve? With a warning or without?

Imagine if a released flutter app resolved its package versions when the user installed it.

I wouldn't expect this if the user installed it through the app store, but I would expect it if the user installed it through pub global activate. It's a different packaging/distribution mechanism so I wouldn't have any expectations for the same behavior.

If you want the behavior of distributing a fixed version that doesn't do any solving, I think it's be to use a packaging mechanism that isn't pub. What value is pub delivering when it isn't solving?

What would the reaction be from the flutter community be if we said:

Hey we've got this new release process.

I think if we said that the app author could not predict the version of the Dart or Flutter SDK at the time of install on the phone, they'd also be unhappy.

If you answered b) then sorry you don't get the job, because no-one would release an app after a package upgrade without testing it.

I answer B and I'm happy with my job. Let's try to keep the discussion civil and not imply that a different consideration of tradeoffs necessarily implies different levels of engineering capability.

I strongly suspect that what you really want here is a different deployment mechanism than pub entirely. If testing against a working environment is the goal, it would be far better to have a deployment mechanism that bakes in the SDK (or compiles to an executable) rather than downloading packages (pinned or solved) against whatever SDK version the user has. This is a gap in the Dart ecosystem, but I don't think pinning packages for pub global activate is the right plug for that gap.

[jsroest]

This is exactly what bit me.

Works, because it honors the pubspec.lock file:

dart pub global activate -s path ~/repos/0000000-create_flutter_mono_repo-dalosy/

Does not work, because it ignores the pubspec.lock file:

dart pub global activate -s git git@bitbucket.org:dalosy/0000000-create_flutter_mono_repo-dalosy.git

In my case, there was a breaking change in the xml package between version 6.1.0 and 6.3.0.

So the obvious fix is to pinpoint specific versions in the pubspec.yaml file, and to not use version ranges ^6.1.0, but I am still a bit confused that installing from source behaves differently than installing from a GitHub repository.

[sigurdm]

Just saw cargo can do something like this with a --locked flag. https://doc.rust-lang.org/cargo/commands/cargo-install.html#dealing-with-the-lockfile

[bsutton]

So the cargo lock path matches what I was expecting. This would require the pub lish command to upload the .lock file. So Dart pub lish --lock causes the .lock file to be uploaded. At activation, if the .lock file was uploaded, if any package can't be resolved then the activate fails with an appropriate message. Deprecated package should resolve - I think that happens now. This will result in a stable cli install.

…

On Mon, 21 Oct 2024, 11:13 pm Sigurd Meldgaard, ***@***.***> wrote: Just saw cargo can do something like this with a --locked flag. https://doc.rust-lang.org/cargo/commands/cargo-install.html#dealing-with-the-lockfile — Reply to this email directly, view it on GitHub <#3668 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAG32ODK5RMBSYBOXTZK33DZ4TVWHAVCNFSM6AAAAABQKAYELSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDIMRWGQ4TSNZZGE> . You are receiving this because you authored the thread.Message ID: ***@***.***>