proposal: Go 2: disallow imports of external packages in library packages

[metakeule]

Proposal for Go2: Disallow imports of external packages in library packages

Definition of the term main package

A package with the name main containing a function main (aka a program).

Definition of the term library package

A package that is not a main package.

Definition of the term external package

A external package is a library package, that is neither part of the standard library,
nor a package that has the importing package as a subpath.

Examples

• package foo/bar/baz would be an external package when imported to bar/foo but not be an external package when imported to foo/bar

• package fmt would not be an external package since it is part of the standard library

Proposal

This proposal would not change the rules for imports of standard packages, it would
not change the rules for imports of subpackages and it would not change the rules for
imports from any main package.

It would only forbid a library package to import an external library.

Examples

We have the following packages:

• A foo/bar/baz (a library package)
• B foo/biz (a main package/ a program)
• C foo/bar (a library package)
• D bar/bop (a library package)
• E fmt (a standard library package)

According to this proposal the following imports are allowed:

• B importing any of A, C, D, E
• C importing A and E
• A,B,C,D importing E

The following would be rejected:

• D importing C or A
• A importing C or D
• C importing D

Benefit:

• No dependencies between external libraries.
• No references to symbols and functions of an external library in a library package
• No package management necessary (apart from reproducible builds for main packages).
• Decoupling, better maintainance
• Dependencies are owned by the user. May easily be replaced by changing glue code.
• Leads to larger repos in order to avoid the hassle of lots of glue codes for the users.
• Larger repos means less dependencies in the main package.

The standard libraries are not affected and since they are released as a whole, there are no package
management issues with them anyway.

But how can a library package foo then depend on a library package bar?

It won't. However a function of foo can consume an interface that is implemented by some type of bar.

The main package then would import both library packages, passing the required value to foo.
In order for that to work, the developer of foo would offer example glue code.

The developer using package foo, copies the example glue code for the integration to his main package.

So what happens, if any of foo and bar changes in an incompatible way?

We assume that the principal functionality offered of bar would not change. If so, it would make sense
to rename it.

However what could change is the exported symbols, the initialization routine etc.

If so, the main package would not compile. Since the glue code is now owned by the developer of
the main package, it can be easily changed without foo having to be updated. In the worst case
one could create a wrapper implementing the needed interface.

In combination with reproducable builds (e.g. vgo) main would not simply stop working without intervention of the user.

UPDATE

After I bit of reasoning, it seems like it would be better to apply this restrictions only if the importing library is "published", where "published" would be defined as having a domain name as part of the package path. These would give some freedom to mono-repos and the standard library (which was excluded anyway).

[pciet]

Do you have a real world example that is improved by this feature?

[metakeule]

@pciet

Well we have the package management problems with Go since day one.
Countless variants of "solutions" and the complexity involved.

If these dependencies could be avoided in the first place (and that's what would be inforced by this proposal) the better for us all.

The only unavoidable dependency is from the main package. If you write it, you "own" your imported external libraries, and you make sure they work. Only after a while, when updating, the problems arise.

Now if you have library package foo depending on type Bar of package bar and you also need package baz that is depending on type Bar of package bar that changed in an incompatible way, you have a problem. With this proposal foo and baz could not reference type Bar so there would be no dependency and no problem. Inside main you could easily wrap around bar.Bar to fullfill the incompatible interface of foo without any action from the developer of foo.

[mbenkmann]

If neither bar nor foo may depend on the other, nor on a 3rd library, where is the definition of the common interface known by both foo and bar. Your proposal would only work with interfaces from the standard library.

[metakeule]

@mbenkmann

If foo is meant to be used with bar, foo would define an exported interface type that is implemented by bar at the point when they are compatible.

BTW: This encourages a culture of similar interfaces for similar tasks within the community, so libraries can be swapped.

[metakeule]

A good extension of that proposal would be to forbid init functions within libraries.
Then every initialization inside a library would be done via a function call from main.
That would make it more transparent, that there is initialization code and if this code changes there is a chance that the function name changes and such a change would get noticed at compilation time (compared to a change within init which would get unnoticed).

[4ad]

I appreciate the sentiment behind decoupling external libraries and the general idea of making the dependency graph wider rather than deeper, but this is unviable without covariant types, without generics, without disallowing packages that have side effects, and without much stronger type inference (and without surely many other things). I don't see how it would ever be possible in a language like Go.

This would make more sense in a purely functional language.

Since this is much too radical, I'd prefer some mechanism, tooling, or policy (or a combination of all three) that would encourage, or somehow help development of these "independent" libraries through some other means rather than by adding restrictions to the language.

[metakeule]

@mbenkmann
The interface of foo does not have to be known to bar. This is the beauty of interfaces in Go.

[mbenkmann]

You are assuming interfaces that do not include ANY custom types. How do you define useful interfaces for a graphics library if you don't even allow abstractions for Rectangle. You want those interfaces to use [2]int for a Point?

[metakeule]

@4ad
Concerning generics it seems the Go team wants to have them in Go2.
Your statement is very broad. Can you give a concrete example where two packages could not be used in tandem with this proposal?

[metakeule]

@mbenkmann
Why not? Or x,y int. For sure the libraries would look a bit different, but I question, that they would look worse. One would definitely make more use of builtins which would be another good side effect IMHO.

[4ad]

@metakeule I think @mbenkmann's last example is a good one. In general, to be really useful packages need to share data (through types), not just behavior (through interfaces). You can do everything solely with interfaces, but that doesn't make for a good programming model in a language like Go (Lisp would be fine here).

[metakeule]

@4ad
I fail to see why that would result in worse programming. Decoupling is a good thing and it is why we have interfaces in the first place. And decoupling is far more important between packages. Where would we be, if not every package used io.Reader and io.Writer?

[mbenkmann]

And what about a Dialog? With x,y,width,height, a title, a message text, buttons...?

[justinfx]

I don't understand how this would work if library foo needs to build its implementation details on top of library bar. There is no main involved here. Someone is trying to provide others with a library that may be consumed by another library or a main. Your proposal would make it impossible to have private implementations that use existing libraries without asking some other person to pass you through an interface?

So let me make sure I understand this. If there is library "A" which wants to use some sort of embedded kV database or cache, it would instead code against an interface and ask the consumer of this library to pass it in? Now let's say you have someone else with library "B" that uses "A" and third person with main that uses "B"

main -> B -> A

According to your proposal, main would have to import the transitive key value db dependency, and pass it through to B as an interface, which would then have to pass it though to A as an interface?

I'm very lost as to why we would want this situation.

[mbenkmann]

Maybe we should just exchange []unsafe.Pointer in our interfaces. Yay. Who needs type safety.