Pkg3: compatibility constraint granularity

[StefanKarpinski]

Continuing the other half of the discussion on #3.

[StefanKarpinski]

I can see that having upper bounds at the minor version level might be too conservative. This is especially true given the way semantic versioning is supposed to work – i.e. that new minor versions don't break compatibility, they just introduce new features. If a package version, A v1.2.3, works with B v4.5 then it should usually also work with B v4.6. Sometimes B v4.6 may break things, even following SemVer – e.g. by introducing a name collision with another package that requires an explicit import. But in that case, it may be better to release A v1.2.4, which either introduces an upper bound of B v4.5 (in case compatibility can't be easily fixed), or by fixing the package to work with B v4.6. This would solve the Compat issue, as long as new Compat versions are minor. A new major release of Compat would require other packages to change their compatibility metadata, but that's presumably what should happen for a new major version. (Imagine if the way Compat is used changed completely.)

[tkelman]

The scenario in #14 (comment) describes a situation in which upper bounds will occasionally need to be made at patch level. This would not be a common occurrence but it does happen, so I don't think we should make it impossible by design. Compatibility ranges have been capable of expressing all the situations we've seen in metadata - Pkg2 is even capable of using unions of disjoint ranges, but no one has needed that level of complexity yet from what I've seen.

Upper bounds (or the equivalents thereof, expressed in some other way) are about avoiding a known problem. If you set them by default assuming there will be future problems, setting them at minor level is probably too conservative, major is more likely to accurately predict future breakage but also not guaranteed.

[StefanKarpinski]

Sure, there do seem to be scenarios where you want to make a patch-level upper bound. I'd like to figure out a way to discourage it without making it impossible to deal with if it happens. Maybe social discouragement is the best approach though.

[simonbyrne]

If we do go this route, we need better tooling to identify (a) when packages are not using the latest version, and (b) the reason why. Here's a recent example:
JuliaStats/Distributions.jl#560 (comment)

In the end, I set up my REQUIRE file to be the same as that of the user, and sequentially removed packages until I found the problematic one.

[tkelman]

Agreed, but that's an issue with any and all upper bounds, not so specific to granularity of major vs minor vs patch level.

[simonbyrne]

true, but the issue is likely to be more prevalent should the bounds be more strict