Formally specify how dart: and package: URIs work.

[lrhn]

We currently have our tools support package: URIs and dart: URIs, but we haven't formally specified how they actually work (wrt. canonicalization/normalization/relative URI reference resolution).

I have a fixed definition of package: URIs in mind, and have implemented it in both the Uri class and package:package_config, but we don't have a central repository for that definition.

We have no definition of dart: URIs other than people being allowed to write dart:async. We have internal uses of more complicated dart: URIs, but again no formal specification. We are now considering allowing user-facing URIs like dart:something/lints/recommended.yaml links to specify platform recommended lints. Before doing that, we should specify the rules for such URIs and possibly ensure that the Uri class supports that too.

I'll follow up with a proposal for the dart: URI formalization.

[lrhn]

Dart dart: URIs formalized

Dart supports the dart: URI scheme for imports of platform libraries. The scheme is not specified in any detail. This is an attempt to make a unified and future-prepared formalization for the dart: scheme and how it is resolved.

The only currently user-visible use is imports/exports like import "dart:async";.

This has had some consequences internally in the platform libraries. When using part files in platform libraries, the part "int.dart"; is technically underspecified because it's not a relative URI relative to dart:core. We cannot use part of "core.dart" as the library-pointer in the part file because the part file doesn't have a proper URI, and we are stuck with part of dart.core, a format that we are trying to move away from.

Internally, the front end is using URIs of the form dart:async/future.dart to refer to the actual future.dart part file imported by dart:async (which can also be referred to as dart:async/async.dart). This is a user-facing URI because it shows up in compiler error messages and stack traces.

The Uri class is not aware of dart: URIs and are not treating them specially, and since they are not formatted like proper hierarchical URIs (path doesn't start with /), some relative path resolutions can produce garbage results (similar to package: URIs until we fixed that).

We are now discussing the use of dart:something/something.yaml to allow users to refer to SDK-distributed YAML files in analysis_options.yaml. That means that we have to specify what that means.

Consistent Formalization

We define that a valid dart: URI must have one of the forms:

• dart:name
• dart:name/path

That matches a URI grammar of:

dart-uri = "dart" ":" dart-name [path-absolute]

where dart-name must be a valid "Dart library name", which currently means a valid Dart identifier, and the /path must be a valid absolute URI path as per RFC 3986.

(In comparison, a package URI has the form "package" ":" package-name path-absolute).

The dart:name format is the only format allowed in imports/exports outside of platform libraries, and only when name does not starts with an underscore.

We define that form to be equivalent to dart:name/name.dart, as defined below. We use the expanded form when resolving against the URI, but normalize the expanded form to the the shorter form in all other cases.

A URI of the form dart:name/path is considered a hierarchical URI with a path component consisting of /path. The name is treated like an authority part in a traditional hierarchical URI like http: scheme URIs, just without needing a leading // because the name is not optional. (That is, resolving a relative URI reference against a dart:name/path URI is roughly equivalent to resolving it against http://name/path and then changing http:// to dart: again afterwards.)

We will normalize dart:foo/foo.dart to dart:foo, but use dart:foo/foo.dart when resolving a relative URI reference against it.

A valid URI of the form dart:word/path defines a platform supplied file. The platform provides a way to get access to the content of such a file (if it exists), like a virtual file system. This virtual file system is made available to platform tools. It's not necessarily available to user code, except as explicitly defined:

• Non-platform library imports/exports may use dart:name only, with a name that doesn't start with an underscore. It cannot use dart: URIs as part files. The import/export only works if the platform actually supplies a library of that name. (In that case, the system environment will contain a definition for dart.library.name with the value true, which is one of the reasons the library name must be an identifier).
• Platform library imports/exports/parts/part-ofs may use any dart: URI. This allows dart:async to use part "future.dart"; to refer to dart:async/future.dart, and allows that future.dart to use part of "async.dart"; to refer back to dart:async/async.dart aka. dart:async.
• Analysis options may use dart:something/something/something.yaml. We haven't decided what the somethings are yet. It will refer to a file provided by the SDK. The something doesn't have to correspond to a platform library, the dart: scheme is a segmented namespace based on the following name, with some segments corresponding to platform libraries. We should be aware that any name we use here will possibly be prevented from being a platform library name in the future.

Uri support

We may want to change the Uri class to understand dart: URIs and handle relative URI reference resolution against them, like we have for package: URIs.

Summary

In short, dart:name, where name is a valid Dart identifier, is equivalent to dart:name/name.dart, and resolving against dart:name/path is equivalent to resolving against a traditional hierarchical URI like dart://name/path. The dart: URI just doesn't use, or need, a leading // for the "authority" part because the part is not optional.

[eernstg]

Looks good! But wouldn't it fit better in the language repo?

[lrhn]

I decided not. The language may want to have a say about it too (mainly to say that dart:name and package:packageName/path are valid in imports), but to me it feels like more of a tool specification issue than a language specification issue.

[devoncarew]

cc @pq @mit-mit