Join GitHub today
GitHub is home to over 31 million developers working together to host and review code, manage projects, and build software together.Sign up
Consider ^ over ~ for default --save option #4587
If npm install underscore --save automatically used
Additionally approaches like Browserify would benefit from better pruning from semver compatibility being both used and maintained.
I understand it's not a trivial change, so happy to discuss further.
+1 I brought this up in IRC the other day after the pervasive use of "~" bit me during a routine upgrade of one of my modules.
I also opened an issue over on npm/init-package-json#10 (and maybe I was a little harsh).
If the automatic upgrading via full semantic versioning is too offensive, I think it is reasonable to completely lock to a specific version. But I stand by the assertion that the current situation with "~" is crazy.
This was referenced
Feb 2, 2014
This has landed with v1.4.3 with pull request #4589
Honestly, for me and the DocPad community this is a terrible change. I expressed this in the tweets here a while ago: https://twitter.com/balupton/status/430248212117454849
We use to allow minor version updates, but in node land, minors mean things could break for some people, so we started using
The reality is that minor changes introduce new functionality, even if they are not meant to include b/c breaks, this new functionality could break something unintentionally, and the as more minor and revisions come out, b/c breaks compared to the original release it was for.
The only definition of the versions that I have found to be safe, is that included in the tweet I mentioned.
So why would anyone want
Allowing it to be configurable + passed on the command line would also probably get more developers making a conscious choice about their semver ranges on a per-package basis.
Currently, it seems if you don't want to use the default semver, you're effectively punished by npm as you can no longer use any of the
Most will pick the default semver range and not give it another thought simply because it's convenient. Maybe that's a good thing though, as
referenced this issue
Feb 18, 2014
@balupton thanks for voicing your concerns. I think you're looking a little too closely at this in comparison to how things work today, instead of seeing this as an important cultural change.
Consider the breaking scenario:
Now lets consider a world in which most users now have the dependency range set to a semver (^x.y.z).
The update will have broken for many users. Lots of angry responses get sent the author. They release a patch, and the breaking change is fixed. In future the author tests their releases more thoroughly with beta users, now bearing in mind backwards compatibility and understanding the importance of semver.
This is a very healthy correction process.
Consider also that modules that do break with minor revision changes are typically versions
The key use case to bear in mind for this as well is for using Browserify. The only way npm modules can properly scale to work in the browser is by having this cultural change to using semver properly. Knowing backwards compatibility is supported allows modules to share dependencies, which is the only scalable way we can make modules work in the browser without heavy duplication.
Thanks for the detailed response. For me, it seems optimistic, let me try to explain:
Semver is great in concept, but if fails realistically, as one can only know about the b/c breaks that they know about, and too often than not we introduce ones we don't know about, and life goes on, and by the time it's discovered, it's too late to fix it, and anything that was using the old way is now broken, and if that is in a nested dependency that is no longer maintained, that's a big PIA.
The reality of this brings the ideal of semver to work in the way myself and Rod Vagg describe in the twitter conversation, and I can't see a way this is unavoidable. Ideal says one way, reality says another. It's noble to chase an ideal, but when the recommendation of pursuing this ideal is to have things break, and users cry out, in the hopes that the maintainers will and CAN STILL fix the issue, is another thing.
I don't understand these quotes... the solution to maintaing packages isn't adding more risky version ranges, but by getting more maintainers. For instance, for DocPad we now have a team of about 20 people helping maintain our plugins.
We also keep our plugins minor versions in sync with their dependency minor versions. E.g. if we have
This dual-approach has worked really well for us and our users so far.
Rather than trying to change culture's obedience to a standard they've already read, this time by beating them with a stick, wouldn't it be better to just update the standard with what reality actually defines it as? This is the same thing with language, no one uses language as it was set out to be thousands of years ago, it evolves despite the language police best efforts.
For me, as stated in the tweets, the best solution I've found is simply to use
An example of where this would work would be less. Less introduces an improvement that won't break anything, that's a revision release. Less introduces optimisations to their output rendering or changes some obscure API that only a few people use, that's a minor release. Less changes a part of their API that everyone uses or introduces dangerous optimisations for the output by default, that's a major release.
The original specification of semver just doesn't seem to catch these use cases well, or maybe it does but then we'd be on less version 1000, even when only output changes, or obscure APIs that no one should have been using in the first place but maybe someone was.
There is so much benefit to be gained by maintaining a single backwards-compatible version. I completely agree we need to ensure this is realistic, but I am still not convinced why it can't be, and until then would rather try to work towards the ideal.
If there was a bad maintainer that was causing issues, I can always switch back to using
Also, in most cases bug fixes typically only get applied to the latest minor version only. So a user can be stuck on a previous minor, with a bug, that can only be resolved by making a breaking upgrade (consider that the module author has refactored, and can't afford the effort to solve the bug on the old code base).
In terms of browser use, what I am describing is that with the current system we can often end up in situations where the same dependency is being used in different minor versions within the same overall application. This is a duplication of code sent to the browser, making large browser apps impractical.
Semver thus properly enables npm for use in the browser, by ensuring backwards compatibility allowing for duplication to be avoided. It also ensures bug fixes are shared more widely. In the browser scenario I really do think it is more of a necessity than an ideal.
I definitely agree though that we do need to consider these edge cases as you describe and ensure that we are catering for minimal risk in the process.
Considering your example - I'm not sure I fully follow this from the perspective of whether it is an internal API change or not. This is my best interpretation:
Where in this process is the breaking change? It would be good to understand this further if you don't mind sharing.
Thanks for the discussion - it's really good to hear the arguments on this.
In terms of unknown backwards compatibility changes, the point is that these get discovered in the process and resolved. Early upgraders would notice the issues first, and patches get properly applied for subsequent installs. By forcing this process outward, that is how to create an ecosystem of backwards compatibility.
Note also that most npm packages are less than the 1.0 release, so most packages will continue to behave the same though.
Packages over a 1.0 release should at least be able to commit to this effort, as that is the nature of maintaining a project for users.
In terms of risk, I wonder if we really are increasing risk or decreasing risk in fact overall. The first to upgrade does take on a slightly higher risk, but the early adopters then solve the issues for the rest. The issues get put together into one place, and solved quickly, instead of distributing the risk over lots of minor changes.
Thanks for the reply. Here's some replies to the parts that stood out for me.
For us, we release each change independently (or at least try to). So we always try to ensure each minor version range is completely stable. If a bug is found, we release a revision level fix for it — unless the fix requires b/c breaks, in which if the b/c break is for some (it is released as a minor), if the b/c break is for everyone (it is released as a major).
The problem is that semver only talks about the public API, not result output, or in the case for the browser, stylesheets. For instance, if I depended on a browserify component that had the default background color black, but then changed it to white. By semver's definition, stylsheets are excluded as they are not a public API. This leads me onto my next point.
For websites, it does increase risk, as say we are developing locally on our machine, we do a deploy, or in 6 months our website goes down and is then rebooted with new deps. Our website is broken, or doesn't display correctly, something's background color has changed, or our stylesheets are messed up due to different optimisations. Etc.
The expectation that the default would be for us to complain to the author here is really weird. Yes, we can use
Changing the default to
This is why we've done docpad/docpad@1656771 to revert this change when using DocPad, but it is dangerous, as most people using DocPad are new to node, new to npm, new to semver, and they come to us when things break, and things should never break. But unfortunately, they do break with minor versions, and are way harder to fix because the b/c breaks are coupled with new functionality that sometimes just can't be reverted without breaking things for other users. Which is why keeping the
The breaking change is the output changed, which causes our tests to break. Which makes it a breaking change, as we now have to fix something, be it tests or code, or whatever, to get things working again. More unnecessary man effort.
But this is one example applied to tests, it also applies to other things. Here's another example, same structure, different content:
Some other big projects in our experience that follows majors break for everyone, minors could break for some, revisions won't break for anyone, are coffeescript and backbone. These are also projects that could never fix one of their b/c breaks, as because of their size, as soon as they are out, people are already using and implementing them with the different functionality, so any b/c fix is a b/c break for some too.
It's great that you update bugs for each minor, but most maintainers wouldn't.
Exactly as you say, using this stuff in websites does require version lock for production, exactly with
This is an interesting point you raise, in terms of how to draw a line between a style change being part of the public API or internal API.
A change in the look of a visual component is a public API change.
Ideally though, a new style would be a new theme. The default theme would remain the same, retaining the public API. If the user wants to use one of the new themes, they can load from one of those instead, as a new API opt-in.
Small alignment changes or CSS browser compatibility fixes would be seen as patch or revision changes.
Small visual adjustments do sit in a grey area, that very much depends on the context of the component. I think authors could work this out sensibly.
LESS and Parsers
I still think we need to clarify this LESS example - I wouldn't say that your tests breaking is seen as a backwards-incompatibility here. It continues to work fine for users, so your tests being updated is just a patch. Effectively your tests are running against the internal API (the CSS), not the external one (the way the CSS is displayed).
For parsers, yes there are scenarios where backwards compatibility is not possible. But mostly these are edge cases and most users would upgrade without issues.
If I want to provide a project to the public and am worried about a parser introducing a breaking change that will get blamed on me, I can always revert to
Since many of your plugins are parsers, I can certainly see your incentive to stick with
But I don't think your situation is indicative of the default install scenario so I don't see how it is worth overlooking all the benefits for this reason.
Perhaps an alternative fix in your situation could be accommodated - since you are wrapping
As the twitter conversation has continued, for clarity, here are my issues with traditional semver:
The issue with this I've found, is that there are more things that can break rather than just APIs. For instance, you can keep the same API, but the output could change, which could consequently cause breaks. In this case, developers I've found use the following unnamed semantic versioning standard instead. Let's call it realsemver:
Realsemver is what I see in use by pre-processors like CoffeeScript and LESS, and libraries like jQuery and Backbone.
I find this definition to be more realistic, as it seems semver's is shortsighted to only care about only API changes, rather than a vast array of other things that could possibly affect b/c compat.
For instance, let's take Backbone's v1.0.0 release, that would break for everyone. Backbone's v1.1.0 release, that would break for some people.
This is what semver says about this:
Which translates to:
To be honest, I feel I have said all I can say on this. Semver reality is minors often break things, people get upset, and you can't simply fix the b/c break now as the b/c break fix would be a b/c break. Using
To combat this oversight of semver with the flaw that minors will break, people rely on
Ideally I'd like it if npm keeps using
However, I've made my patch to DocPad to workaround this docpad/docpad@1656771
And would much rather spend my time making DocPad better, and spending a little amount of time working around npm, than spending an infinite amount of time in a debate that seems is never going to be accepted.
If you want to pin deps, then there are many ways to do that. You can even use
Ultimately, a convenience option like
"Always pin deps to a specific version" isn't really reasonable in many cases. It means that you won't get updates, even security patches or minor bugfixes. It is too biased in favor of stability rather than convenience for many use-cases.
"Always allow any updates" (eg,
Most npm module authors I've talked to are at least intending to use versions according to the SemVer spec, and most expect that
Meaning comes from humans, not from specs. If you are using modules where a change from
If we're going to have a
I'd argue that the output of a transpiler is absolutely part of the API. That's the return value of the function.
Is it just Node v0.10 that supports
If that's the case, then it's probably too early to setup
@medikoo oh wow, this is a good point. Afaik email@example.com is in node >= v0.10.16.
With this change, any modules published with
Hm, the default
Node 0.10.16 is 9 months old at this point, and there have been numerous
If npm provides yet another reason to upgrade, then that's a GOOD thing for
On Wed, Feb 19, 2014 at 3:50 PM, Alex Kocharin firstname.lastname@example.org:
added a commit
Feb 20, 2014
added a commit
Mar 6, 2014
referenced this issue
Mar 27, 2014
Sorry to beat a dead horse, but we are seeing lots of compatibility issues with this all over the internet, in a few modules I contribute to, in #node.js and elsewhere. Part of it is due to the opaque error message you get when installing a module with a package.json using a caret, e.g.
Changing the default behavior of
This should not have been in a minor revision and I would like to posit that we would be much better served by a less clever default format for