add an API option to ignore compat #1607

KristofferC · 2020-01-09T15:31:12Z

Since this change is so small, I might as well probe what people think about this.

This adds an option that just turns off all compat (except the compat in your project) and gives you the latest versions of everything.

julia> Pkg.update()
   Updating `~/.julia/environments/v1.3/Project.toml`
 [no changes]
   Updating `~/.julia/environments/v1.3/Manifest.toml`
 [no changes]

julia> Pkg.update(; ignore_compat=true)
   Updating `~/.julia/environments/v1.3/Project.toml`
 [no changes]
   Updating `~/.julia/environments/v1.3/Manifest.toml`
  [68821587] ↑ Arpack_jll v3.5.0+2 ⇒ v3.7.0+0
  [a81c6b42] ↑ Compose v0.7.3 ⇒ v0.8.0
  [31c24e10] ↑ Distributions v0.21.12 ⇒ v0.22.0
  [ae029012] ↑ Requires v0.5.2 ⇒ v1.0.0

No docs yet because that will be the majority of the work for this PR and I want to see some opinions first.

StefanKarpinski · 2020-01-09T15:38:19Z

Generally best to avoid double negatives. How about compat=[true|false] or respect_compat=[true|false]?

KristofferC · 2020-01-09T15:40:35Z

You will only call this as ignore_compat=true though which I think is more "direct" than respect_compat=false because you are directly saying what to do with the "ignore compat" (the bool is almost unnecessary).

codecov · 2020-01-09T16:09:10Z

Codecov Report

Merging #1607 into master will increase coverage by 0.13%.
The diff coverage is 100%.

@@            Coverage Diff             @@
##           master    #1607      +/-   ##
==========================================
+ Coverage   86.44%   86.58%   +0.13%     
==========================================
  Files          24       24              
  Lines        5387     5390       +3     
==========================================
+ Hits         4657     4667      +10     
+ Misses        730      723       -7

Impacted Files	Coverage Δ
src/Types.jl	`83.14% <ø> (ø)`	⬆️
src/Operations.jl	`92.22% <100%> (ø)`	⬆️
src/PlatformEngines.jl	`70.58% <0%> (+0.13%)`	⬆️
src/BinaryPlatforms.jl	`91.9% <0%> (+0.8%)`	⬆️
src/generate.jl	`100% <0%> (+14.7%)`	⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 6bdf5fc...b0db344. Read the comment docs.

DilumAluthge · 2020-01-09T17:50:19Z

If we want to avoid double negatives, how about something like force_latest = true or override_compat = true?

StefanKarpinski · 2020-01-09T19:15:49Z

Nah, I guess this is ok.

DilumAluthge · 2020-02-14T09:37:58Z

Bump. Is there any objection to merging this? Seems like it could be very useful.

ExpandingMan · 2020-06-08T17:38:19Z

How hard would it be to (perhaps as an additional option) add a comprehensive set of warnings about what compats are being overridden? I think the ideal situation would not be to permanently just always go around ignoring compats, but to make it extremely simple to resolve specific issues. Seems this PR would get us halfway there, but the other half is transparency about what's happening.

KristofferC · 2020-06-08T20:45:00Z

Not too hard, just go through all compat in the end and see what is incompatible. However, since I don't think that this should really be used I am personally not very interested in putting much more effort than this into it.

cstjean · 2020-06-10T21:19:28Z

What should this be used for?

tpapp · 2020-08-07T12:20:22Z

To second @cstjean, can you please elaborate on the use case for this?

KristofferC · 2020-08-07T12:22:46Z

For people that complain about compat bounds.

tpapp · 2020-08-07T12:44:21Z

My concern is that after an upgrade that ignores compat bounds, they would complain about broken software instead.

NightMachinery · 2020-08-07T14:12:37Z

@tpapp commented on Aug 7, 2020, 5:14 PM GMT+4:30:

My concern is that after an upgrade that ignores compat bounds, they would complain about broken software instead.

I have been using Python a lot, and I have yet to see problems out of ignored compat bounds. Yet Julia's strict insistence on compat bounds, coupled with badly defined compats by package authors, is giving me trouble almost 1 in 3 times I install a package. I think a major problem is that when the package authors don't specify a compat bound, e.g., see https://github.com/peterkovesi/PerceptualColourMaps.jl/blob/v0.3.0/REQUIRE, Pkg defaults to respecting their current used versions, and thus installing one package with old deps will cause a cascade of downgrades. My own philosophy is to always use the latest versions, and if sth breaks, stop using it or create a specific env for just that package. It's madness to do snowball-downgrading by default. See also my issue on LanguageServer.jl, where they state that they use ugly hacks just to circumvent this behavior.

A much better approach to preserving compatibility between packages is to use Rich Hickey's philosophy; Don't change existing API in breaking ways. If you really need a breaking change, add a new API, and preserve the old one. Something as simple as dosth and dosthv2. You can even stop maintaining the old API, but don't just drop it to hell. This reduces maintenance costs by a lot, at the price of not introducing new goodies to old dogs. Which is fine, unless security is a major concern, which it is not in most usecases.

tpapp · 2020-08-07T16:01:57Z

Julia's strict insistence on compat bounds, coupled with badly defined compats by package authors, is giving me trouble almost 1 in 3 times

It is hard to address this without specifics.

when the package authors don't specify a compat bound, e.g., see TerminalExtensions

Note that this particular package does have compat bounds (since the modules it uses are standard libraries, compat bounds on julia is sufficient). Generally, compat bounds are now required for automatic merging into the General registry.

much better approach to preserving compatibility between packages is to use Rich Hickey's philosophy; Don't change existing API in breaking ways

This has been discussed extensively on Github and Discourse, so I would rather not open that question here. Briefly, it is not the model Julia subscribes to.

About this PR: my expectation is that instead of complaints about compat bounds, package maintainers would get bug reports about a broken state that would require tracking down problems that happen mostly in other packages (since that's where compat would be turned off).

StefanKarpinski · 2020-08-07T16:25:52Z

We DO suggest that packages follow semver like Julia itself does, which includes not making any breaking changes except in major versions. However, the reality of life is that people don't always do what you suggest they do.

DilumAluthge · 2020-08-07T16:28:07Z

What if, when someone uses this feature, they get a warning message saying that it may break things, and it is their responsibility (not a package maintainer's) to solve any problems that arise when using this feature?

NightMachinery · 2020-08-07T16:30:53Z

@tpapp commented on Aug 7, 2020, 8:31 PM GMT+4:30:

Note that this particular package does have compat bounds (since the modules it uses are standard libraries, compat bounds on julia is sufficient).

Yes, I had mistaken the source of the issue. I edited my comment to point to the real culprit, but unfortunately, it doesn't get reflected in emails. (Is it a good practice to reply to comments instead of editing them to make sure people get the update in their email clients?)

Generally, compat bounds are now required for automatic merging into the General registry.

Perhaps this issue will get dissolved after a year or two then. That (latest) problematic package that
broke my setup is actually fine if I use add ...#master, but it's stable package version uses bad bounds.

About this PR: my expectation is that instead of complaints about compat bounds, package maintainers would get bug reports about a broken state that would require tracking down problems that happen mostly in other packages (since that's where compat would be turned off).

I don't quite understand you here. Can you elaborate further? My impression is that ignoring compat bounds will mostly harm unmaintained packages that haven't updated their deps?

A workaround for this problem might be to add a issue reporting helper tool to Pkg, that outputs all relevant info (OS version, Julia version, Python used by PyCall, ...), and include the ignored bounds in this output. When people open issues, this will be a clear hint that they have ignored bounds, and the author can ask them to try to reproduce under strict bounds. These kinds of stuff are standard practice in, e.g., reporting Emacs issues. The author will ask you to reproduce under a clean env (emacs -Q), if they suspect that other packages are the source of the problem.

KristofferC · 2020-08-07T16:37:03Z

I mostly did this PR to have something to point to when people complain that they are not getting the latest version. They can use this option and when everything breaks, they will reach enlightenment.

tpapp · 2020-08-07T17:43:18Z

I don't quite understand you here. Can you elaborate further? My impression is that ignoring compat bounds will mostly harm unmaintained packages that haven't updated their deps?

Not necessarily. A maintained package may not update their [compat] section to the very latest versions for a while for valid reasons (eg it takes time to update & test code and no one got around to it yet). Please note that "maintained" != "everything happens instantly".

when everything breaks, they will reach enlightenment.

I wish it was so, but in practice I think some people will complain vocally. Being able to say "told you so" doesn't really help anyone in this situation.

That said, if this is one of those pedagogical Zen PRs, it is a nice one.

DilumAluthge · 2020-12-12T06:31:43Z

src/Operations.jl

-                        for (name,vs) in cd
-                            # check conflicts??
-                            all_compat_u_vr[name] = vs
+                    if !ctx.ignore_compat


Suggested change

if !ctx.ignore_compat

if ctx.ignore_compat

@warn "You are using the ignore_compat feature. Use of this feature may cause packages to break. It is the responsibility of the user (not the maintainers of any packages) to solve any problems that arise from the use of this feature." maxlog = 1

else

DilumAluthge · 2020-12-12T06:32:00Z

Needs a rebase?

add an API option to ignore compat

b0db344

KristofferC added the triage label Jan 9, 2020

KristofferC mentioned this pull request Jun 7, 2020

New package: IsoPkg v1.0.0 JuliaRegistries/General#15989

Merged

oxinabox mentioned this pull request Jun 15, 2020

Pkg REPL compatibility bounds #341

Closed

DilumAluthge mentioned this pull request Dec 9, 2020

[POC] Selectively override compat entries #2285

Closed

DilumAluthge reviewed Dec 12, 2020

View reviewed changes

aplavin mentioned this pull request Nov 13, 2024

set compat version for specific package for whole environment #4060

Open

IanButterworth added this to Pkg.jl Jul 2, 2025

github-project-automation bot moved this to New in Pkg.jl Jul 2, 2025

IanButterworth moved this from New to Backlog in Pkg.jl Jul 2, 2025

-                    if !ctx.ignore_compat
+                    if ctx.ignore_compat
+                        @warn "You are using the ignore_compat feature. Use of this feature may cause packages to break. It is the responsibility of the user (not the maintainers of any packages) to solve any problems that arise from the use of this feature." maxlog = 1
+                    else

Uh oh!

add an API option to ignore compat #1607

Are you sure you want to change the base?

add an API option to ignore compat #1607

Uh oh!

Conversation

KristofferC commented Jan 9, 2020

Uh oh!

StefanKarpinski commented Jan 9, 2020

Uh oh!

KristofferC commented Jan 9, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

codecov bot commented Jan 9, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

DilumAluthge commented Jan 9, 2020

Uh oh!

StefanKarpinski commented Jan 9, 2020

Uh oh!

DilumAluthge commented Feb 14, 2020

Uh oh!

ExpandingMan commented Jun 8, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

KristofferC commented Jun 8, 2020

Uh oh!

cstjean commented Jun 10, 2020

Uh oh!

tpapp commented Aug 7, 2020

Uh oh!

KristofferC commented Aug 7, 2020

Uh oh!

tpapp commented Aug 7, 2020

Uh oh!

NightMachinery commented Aug 7, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

tpapp commented Aug 7, 2020

Uh oh!

StefanKarpinski commented Aug 7, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

DilumAluthge commented Aug 7, 2020

Uh oh!

NightMachinery commented Aug 7, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

KristofferC commented Aug 7, 2020

Uh oh!

tpapp commented Aug 7, 2020

Uh oh!

DilumAluthge Dec 12, 2020

Choose a reason for hiding this comment

Uh oh!

DilumAluthge commented Dec 12, 2020

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

KristofferC commented Jan 9, 2020 •

edited

Loading

codecov bot commented Jan 9, 2020 •

edited

Loading

ExpandingMan commented Jun 8, 2020 •

edited

Loading

NightMachinery commented Aug 7, 2020 •

edited

Loading

StefanKarpinski commented Aug 7, 2020 •

edited

Loading

NightMachinery commented Aug 7, 2020 •

edited

Loading