Support publishing Flutter plugin + Dart standalone hybrid packages

[dcharkes]

I was under the impression that we could publish Flutter plugins that also run in Dart standalone (ignoring the Flutter plugin code) by a pubspec with:

name: mylib_source
description: A new Flutter FFI plugin project.
version: 0.0.1
homepage:

environment:
  sdk: ">=2.17.0 <3.0.0"
  # no flutter:

dependencies:

dev_dependencies:

flutter:
  plugin:
    platforms:
      android:
        package: com.example.mylib_source
        ffiPlugin: true
      ios:
        ffiPlugin: true
      macos:
        ffiPlugin: true
      linux:
        ffiPlugin: true
      windows:
        ffiPlugin: true

This works locally with both the Flutter and Dart standalone SDK. (For example dart pub get && dart test and flutter pub get && flutter test, where dart is not from the Flutter SDK.)

However, pub does not support publishing this:

$ dart pub publish --dry-run
// ...
Package validation found the following error:
* pubspec.yaml allows Flutter SDK version 1.9.x, which does not support the flutter.plugin.platforms key.
  Please consider increasing the Flutter SDK requirement to ^1.10.0 (environment.sdk.flutter)
  
  See https://flutter.dev/docs/development/packages-and-plugins/developing-packages#plugin
Sorry, your package is missing a requirement and can't be published yet.
For more information, see: https://dart.dev/tools/pub/cmd/pub-lish.

Adding the flutter key makes dart pub get fail:

Resolving dependencies... 
Because jni requires the Flutter SDK, version solving failed.

Flutter users should run `flutter pub get` instead of `dart pub get`.

As far as I can see, the only workaround is to publish two packages.

cc @mahesh-hegde @jonasfj

edit: duplicate of #2606

[mahesh-hegde]

For me the result depends only on which SDK dart command is from.

mahesh:~/Code/jnigen/jnigen/example/pdfbox_plugin/dart_example [jni_refactor=] 
$ export PATH=$HOME/.local/dart-sdk/bin:$PATH
mahesh:~/Code/jnigen/jnigen/example/pdfbox_plugin/dart_example [jni_refactor=] 
$ dart pub get
Resolving dependencies... 
Because pdf_info depends on jni from path which requires the Flutter SDK, version solving failed.

Flutter users should run `flutter pub get` instead of `dart pub get`.
[1]
mahesh:~/Code/jnigen/jnigen/example/pdfbox_plugin/dart_example [jni_refactor=] 
$ export PATH=$HOME/.local/flutter/bin:$PATH
mahesh:~/Code/jnigen/jnigen/example/pdfbox_plugin/dart_example [jni_refactor=] 
$ dart pub get
Resolving dependencies... (2.6s)
Got dependencies!

It's confirmed also by the CI behaviour. The check in which I used setup-dart action failed. I updated it to use flutter-action and it passed.

[jonasfj]

Context: We did this years ago because older versions of the Flutter SDK didn't support flutter.plugin.platforms.

I suppose we could tweak the behavior such that the Flutter SDK constraint is expressed as a minimum Dart SDK constraint.

@sigurdm, any thoughts?

[sigurdm]

A few thoughts:

We need to have some story to tell here, but I'm not entirely sure how this kind of module should be published.

I feel it is somewhat surprising for a package that works as a dart-only package has a flutter.plugin.platforms key.

Could it be that there should be two packages, one having the dart ffi part, and another one importing the other, and wrapping it as a flutter plugin?

Even if that is technically feasible there would be downsides to that:

• branding a single package is easier
• publishing a single package is easier

[dcharkes]

@sigurdm another downside is that all your downstream packages have to do the same.

I feel it is somewhat surprising for a package that works as a dart-only package has a flutter.plugin.platforms key.

The contract is that the plugin is used to bundle the same native code for the same Dart API. But that contract is not enforced.

[dnys1]

The requirement that Flutter be installed and used simply by virtue of the environment specifier creates a whole host of issues right now. It makes it very difficult to publish hybrid packages, which is creating a lot of sprawl in the ecosystem. But worse, it means that any project which has just one transitive dependency on such a package must have Flutter installed and use that only.

This negatively impacts the ability to run CI, compile dart for the server, use in Docker, etc. where the Flutter SDK is much less portable and unnecessary because the package may not even use Flutter.

IMO, the presence of an environment specifier should be modified to mean if running with Flutter pub, then this requirement holds.

The requirement for Flutter to be installed and used should only come from an explicit dependency on a Flutter SDK package.

[jonasfj]

@dnys1, @dcharkes what would happen if we simply removed environment.flutter from pubspec.yaml?

I suspect that dart pub publish would fail, because of some validations. We could disable those using dart pub publish --skip-validation. And maybe tweak them to go away.

Though in general it is a bit scary to let people make a top-level flutter section without a environment.flutter SDK constraint.

Just curious.

[dnys1]

Hey @jonasfj, that's correct. As you mention, attempting to publish the following pubspec is unsupported.

name: hybrid_pkg
# ...

environment:
  sdk: ^3.0.0

dependencies:
  # Dart-only deps

flutter:
  plugin:
    platforms:
      ios:
        ffiPlugin: true

In this case, pub requires the presence of environment.flutter. I agree that that is a good design choice. However, as soon as you add that, you can no longer run dart pub anything from a standalone install--you must use the Dart SDK from Flutter.

This is true of the root package or if any transitive dependencies follow this format (which is required for publishing). It essentially colors the entire dependency tree to Flutter.

Importantly, this is despite the fact that the Flutter SDK is not actually needed. There can be zero dependencies/dev dependencies on Flutter in the graph and this requirement still holds.

The flutter section is currently the only way to signal to the Flutter tooling that when this package is used in a Flutter environment, it handles native code appropriately. This allows, for example, iOS/Android code to exist alongside Dart-FFI code in a single package and function in both worlds.

Basically, I don't believe that the presence of either flutter or environment.flutter sections should ever mandate the use of the Flutter SDK, only a direct/transitive dependency on an SDK-vended package. The requirement that environment.flutter be included if a flutter section is present seems totally reasonable, it just throws a major wrench in the works currently.

[jonasfj]

Hmm, it's technically possible to publishing without environment.flutter by using dart pub publish --skip-validation.

This is obviously not intended :)

And I don't know if Flutter will read the flutter section without a flutter SDK dependency. It probably will 🙈

Also SDK detection in Pana (which affects pub.dev UI) might be off. Not sure.

---

On topic: I think it's rather risky to change existing semantics for environment.flutter.
Though I can see how it's attractive.
If we want to go that way we should explore how many packages and versions would be affected.

Or maybe we do need a mechanism to indicate that flutter is optional. Or simply allow omitting environment.flutter when publishing.

Or maybe we need to wait for the native assets work @dcharkes is working on.

[dnys1]

Hmm, it's technically possible to publishing without environment.flutter by using dart pub publish --skip-validation.

This is obviously not intended :)

Good to know! Though I agree there should be a happy path for it.

And I don't know if Flutter will read the flutter section without a flutter SDK dependency. It probably will 🙈

It does 🙂

Also SDK detection in Pana (which affects pub.dev UI) might be off. Not sure.

Yep 😅 dart-lang/pana#1351

---

Changing the semantics of environment.flutter is the most obvious path to me if it remains a (loose) requirement for publishing. Removing the requirement could work, too, except for cases where there truly is a requirement for a certain version of Flutter (though I can't think of a time this couldn't be conveyed solely via the sdk constraint).

It's true that there are few instances where you need this level of hybridization, but it would be very handy if there was first-class support for these kinds of packages. In the past, the only way was to have two separate packages, a Dart one with FFI, and a Flutter one which was a very thin wrapper (but which maybe used method channels and other SDK code). With native assets, there is no longer a need to use Flutter in the actual Dart code, only for the platform-specific configuration which lives outside lib and which the Dart tool will happily ignore.

[Levi-Lesches]

Adding my vote here. Until native assets are fully supported by Dart and Flutter, the flutter: section is needed.

Even once it is, we would still need the following for federated plugins:

flutter:
  plugin:
    implements: flutter_local_notifications

While we may see more and more method channels being replaced by FFI, I don't think federation is going anywhere anytime soon. There is little value to a wrapper package, and it only adds bloat to pubspecs, monorepos, build + analyzer CI's, and Pub in terms of eating up more good names for packages.

I think Flutter 1.9.0 is old enough at this point that it shouldn't be an issue. At the very least, Pub can check that environment.sdk uses a version of Dart that is too new for Flutter 1.9.0 (pre null-safety) and therefore it is impossible for this warning to be relevant.

EDIT @jonasfj If it's decided that this warning should be removed, I would gladly open a PR as this is a blocker on a few of my projects

[jonasfj]

@sigurdm, @dcharkes maybe we should talk next week, I'm not entirely sure what we should do here.
But I'm increasingly convinced we should try to come up with some sort of solution.

[mkustermann]

(Thinking out loud here)

A package may want to work in different environments (dart sdk, flutter sdk - maybe 3rd party sdks in the future). The language has support - e.g. via conditional imports - that allows the package's code paths to behave differently in different environments.

We'd want such a package to:

• declare which environments it supports running in
• define which versions of those environments it supports
• may have environment specific configuration, assets, etc (e.g. encoded in pubspec.yaml or via hook/{build,link}.dart)

So it would seem natural that a pubspec.yaml that looks like this

environment:
  sdk: ^a.b.c
  flutter: ^x.y.z

gets attached the following meaning: This package is usable in normal Dart SDK in version ^a.b.c as well as in Flutter SDK in version ^x.y.z
=> i.e. the presence of environment.flutter doesn't imply it's a requirement, but the constraint applies iff used in that environment

A SDK like the Flutter SDK is basically a wrapper around our lower-level tools (e.g. pub, dart2js, dart2wasm, vm, core libraries, ...). It provides it's own command line interface as well as API surface (e.g. dart:ui). The SDK's package get/upgrade functionality would work on that SDKs environment key.

=> That's probably the key issue here: A flutter packages get will choose different transitive package versions than dart pub get would choose.
=> The consequence here would be that flutters SDK would operate on .flutter_tool/package_config.json and dart SDK would operate on .dart_tool/flutter_config.json.

(Thinking out loud further)

Having a separate version resolution would also allow environment-specific dependencies, so the package when used in flutter may depend on package:flutter_icons but when used in standalone sdk may depend on package:standalone_icons.

One could encode such environment specific imports in pubspec.yaml

environment:
  sdk: ^a.b.c
  flutter: ^x.y.z
dependencies:
  flutter_icons:
    - environment: flutter
    - >= ...
  standalone_icons:
    - environment: sdk
    - >= ...

(without this, the package couldn't depend on flutter-only packages, but all transitive deps would need to support both environments - which may not make sense for them)

(Thinking out loud further)

Any tool that is depending on package_config.json may have to choose between different ones now. That may include analyzer.

[jonasfj]

(just rambling off Martins thoughts)

We should probably drop the name environment if we completely change the semantics.
We could express it like:

dart: 3.5.1   # <-- language version, nothing more.
sdks:         # Supported SDKs
 - dart: ^3.7.0       # <-- Dart is supported
 - flutter: ^3.28.0   # <-- Flutter is supported

I'm not sure we should want SDK-specific dependencies.

=> That's probably the key issue here: A flutter packages get will choose different transitive package versions than dart pub get would choose.

I wonder if there are ways to solve that problem. Ways to allow import "package:flutter/..." similar to dart:io on web, or configurable imports based on the SDK (and an ability for pub deal with that). Ending my ramblings here.