Add caret operator meaning "allegedly compatible with X"

[agnoster]

This idea was brought up in issue #38, so I thought I'd take a stab at an implementation, just because I think it's often easier to discuss a concrete implementation than a general idea. Hope that's not too presumptuous — if you closed the issue because you think it's a bad idea in general, feel free to ignore.

The caret operator has rough semantics "compatible with X". Specifically, it allows any version that is at least the specified version, but less than the next major version.

For example, ^1.2.3 is equivalent to >=1.2.3-0 <2.0.0.

This seems to me to be the semantics you want most of the time: you have a "known good" version of a package, but you're willing to take any newer versions that promise backwards compatibility.

(As far as the code goes, I basically just took the implementation of tilde and made some alterations. It would be pretty easy to do some refactoring to remove duplication, but this seemed to at least be very clear about what's going on. I'd be happy to make improvements if desired.)

[isaacs]

I like using the caret for this. I'll review the implementation tomorrow when I get back to California.

If you want to refactor a little in a second commit, be my guest.

[isaacs]

One more thought: in the semver spec, 0.x versions are a bit special, in that breaking changes require only the minor version to be bumped.

So, perhaps ^0.2.3 should mean >=0.2.3-0 <0.3.0-0, and ^0.0.2 would be just =0.0.2. What do you think?

[isaacs]

Its easier to explain the way you have it here, as "up to the next major version", but the special handling of 0.x would be required to call it "allegedly compatible with".

[agnoster]

I was actually thinking about whether to handle 0.x versions differently, I love your idea. It would also mean that if someday in the future npm's --save* flags used caret it would just work as intended.

[agnoster]

http://semver.org says:

Major version zero (0.y.z) is for initial development. Anything may change at any time. The public API should not be considered stable.

So maybe it should always be an exact match for 0.* versions?

[agnoster]

Given that description, I'd probably go with the following equivalences (taken from test/index.js):

    ['^0.1.2-test', '=0.1.2-test'],
    ['^0.1.2', '=0.1.2'],
    ['^0.1', '>=0.1.0-0 <0.2.0-0'],
    ['^0', '>=0.0.0-0 <1.0.0-0'],

[guiprav]

Really glad to see this going forward. I wasn't aware of the 0.x exception in semver, good to know. Does it mean that the public API could change even in Minor version increments? If that's the case, then does the ['^0.1', '>=0.1.0-0 <0.2.0-0'] equivalence make sense? I would think it should be treated as =0.1.0 or be invalid altogether, otherwise updates might break working code.

Also, what exactly would be the problem if the tilde behavior was replaced by this one? Considering it will always pick backwards-compatible stuff, nothing should break at all. What am I missing?

[isaacs]

Major version zero (0.y.z) is for initial development. Anything may change at any time. The public API should not be considered stable.

So maybe it should always be an exact match for 0.* versions?

I think that part of the "spec" is overly vague. (In fact, I don't think the guides for library authors actually comprise a "spec" anyway, but that's a whole other long dirty story of compromise.)

In practice, the left-most non-zero portion of the main version is the "breaking changes" indicator. So, from 0.1.9 to 0.2.0, there may be breaking changes. Likewise from 0.0.1 to 0.0.2.

So, this is how it should work, imo. Feedback welcome:

^  => *
^0 => >=0.0.0 <1.0.0
^1 => >=1.0.0 <2.0.0
^0.1 => >=0.1.0 <0.2.0
^1.0 => >=1.0.0 <2.0.0
^1.2 => >=1.2.0 <2.0.0
^0.0.1 => =0.0.1
^0.1.2 => >=0.1.2 <0.2.0
^1.2.3 => >=1.2.3 <2.0.0

Then the default npm install --save can just throw a ^ in front of whatever version it finds.

Also, what exactly would be the problem if the tilde behavior was replaced by this one?

The ~ operator is not going away, if that's what you mean.

Considering it will always pick backwards-compatible stuff, nothing should break at all. What am I missing?

It will always pick ALLEGEDLY backwards-compatible stuff, but whether or not anything breaks usually depends on much more than that.

The tilde behavior was copied from rubygems. Changing it now is a terrible idea.

[agnoster]

Yeah, I think it's important that it's understood this is the "allegedly compatible" operator, not "guaranteed to be compatible". People aren't necessarily going to always stick to semver, though ideally if this becomes the default --save behavior it'll help nudge people in that direction.

Initially I felt like we should follow the spec as closely as possible, but it probably makes more sense to follow the conventions people are already observing. Kind of would like to push the spec to reflect this, because it's a good convention; you have the ability to still release backwards-compatible patches before you're ready to declare the package a 1.x.

I'll probably get to those changes this evening. Given the 0.* special casing the implementation for caret has also diverged sufficiently from tilde that I don't think the refactor I had in mind makes as much sense anymore.

[guiprav]

I understand now why tilde behavior can't be replaced by this, even though the quality of backwards-compatibility only being alleged already plays a role in the current behavior (as long as you use anything other than strict version equality). The fact that the behavior was borrowed from Rubygems is reason enough not to change it now.

And, yeah, the looser the rules, the greater the risks of breaking stuff, I get that. Thanks for clarifying. It's awesome enough that you're considering introducing the caret, Isaacs (yes, that's the name of you both pluralized, not Joyent's Isaac's nickname, lol). And setting it as default --save behavior would be great, although it would probably still have to wait for npm's next major release, right?

Initially I felt like we should follow the spec as closely as possible, but it probably makes more sense to follow the conventions people are already observing. Kind of would like to push the spec to reflect this, because it's a good convention; you have the ability to still release backwards-compatible patches before you're ready to declare the package a 1.x.

About this, I personally wouldn't tag anything as a release version before the API becomes at least relatively stable... And once it did, I would probably not call it 1.0 yet. 1.0 would mean something like a holistic project maturity, not just API stability. But maybe that's just me, of course.

[agnoster]

Okay, I basically just copied your suggestions into the tests since it seems reasonable to me :-)

The code feels a little ugly to me, but at least it's straightforward in a pedestrian way. While I'm tempted to write a more general rules engine that could understand concepts like "the left-most non-zero portion incrementing indicates breaking changes" I'm worried that me going that route would result in an incomplete, bug-ridden implementation of half of prolog.

[agnoster]

Anything else you'd like to see done with this? Happy to incorporate any other feedback you have.

[guiprav]

Was that targetted at me or Isaacs? What you have here looks good to me, I'd start using it as soon as possible. I really just want to stick to semver.

[agnoster]

I suppose that was ambiguous: I meant @isaacs since he's the one who ends up deciding if this goes in or not ;-)

[guiprav]

Thinking about this again, I think I actually still have something to say which you both might consider. Isaacs proposed this, but please consider my comments:

^  => * # Should be =0.0.0 for the reasons outlined below.
^0 => >=0.0.0 <1.0.0 # Why would anyone want this? It means that on every update, things will potentially break. If that's what the user really means, he can use 0.x. I think ^0 should be =0.0.0, even if it strikes as surprising...
^1 => >=1.0.0 <2.0.0 # This is okay because all of these versions should be backwards-compatible
^0.1 => >=0.1.0 <0.2.0 # See first note; this should be =0.1.0. On versions <1.0, we should always pick the lowest possible version that matches the expression instead of the highest. That's the only safe bet, imo.
^1.0 => >=1.0.0 <2.0.0 # Perfect
^1.2 => >=1.2.0 <2.0.0 # Perfect
^0.0.1 => =0.0.1 # Perfect
^0.1.2 => >=0.1.2 <0.2.0 # Should be =0.1.2 for the reasons explained above.
^1.2.3 => >=1.2.3 <2.0.0 # Perfect

[agnoster]

I think @isaacs already explained his reasoning here. We already discussed the first-non-zero-component-as-breaking-change-indicator. If you disagree with it I think you should present new evidence or arguments, but I'm inclined to go with @isaacs's assessment of what's common practice in the node community. The spec should seek to mirror and support community practices.

[guiprav]

Ah... true. Never mind, then.

[isaacs]

This should be >=0.1.3 <0.2.0-0.

[isaacs]

However, ^0.0.2 would be exactly =0.0.2

[agnoster]

Oops, forgot to update the readme after the last changes, good catch. It is a little more verbose to explain the exact behavior of the caret operator with all the 0.x special-casing, but I hope my change is clear enough. My feelings definitely would not be hurt if you find a better, clearer way to re-word it ;-)

[isaacs]

Oh, I see, the code already dtrt, but the docs weren't updated. I hit that point, and stopped. Will resume reviewing now :)

[isaacs]

Landed in master, published as v2.1.0. I'll be working on migrating the various things to use this in npm.

Be advised that you won't actually be able to rely on this support for about 6 months or so, once a few stable releases include an npm version with it.

[agnoster]

That's awesome. For deployed applications, where a whole team may be on the same node/npm version, it should be a lot sooner that we can start using the caret operator in the package.json (combined with checking in node_modules), so that it's fairly straightforward to stay on the proper upgrade train for the packages we use. This feels like the best of both worlds to me.

👍!