New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Investigate alternatives to NPM #291

Closed
prettydiff opened this Issue Mar 24, 2016 · 24 comments

Comments

Projects
None yet
3 participants
@prettydiff
Owner

prettydiff commented Mar 24, 2016

tldr; Edit 8 Apr 2017

Pulled out of NPM. An alternative is written at prettydiff/biddle and it will become a dependency of Pretty Diff very soon.


Goal

The intention is to find a distribution solution so that Pretty Diff can leave NPM.

Primary Problem

These two quotes from NPM co-founder Laurie Voss (CTO) are the straw that broken the camel's back:

Un-un-publishing is an unprecedented action that we're taking given the severity and widespread nature of breakage, and isn't done lightly,

This action puts the wider interests of the community of NPM users at odds with the wishes of one author; we picked the needs of the many.

Conflict to Open Source

I fundamentally disagree. The needs of a package owner completely and absolutely outweigh everybody else. This is a fundamental issue of open source software. The commercial resolution to this condition exists in various forms of warranties, contracts, and documented obligations. Where such documents are neither expressed nor implied the open source software exists for use entirely at risk to the consuming dependent. This the reality of open source software and is stated as such in the most popular open source software licenses.

NPM forms their position because they are a revenue generating business who sell infrastructure. The most important quality to their business is availability (like any infrastructure model), which, in this case, puts their business into conflict with basic principles of open source software. This is an exceedingly rare edge case even if absolutely fundamental. When availability is in question the value of their service is called into question and their brand is severely harmed.

The reason why this conflict likely exists (speculation) is because NPM traffic growth is largely driven by a dependency model. Use of dependencies requires a choice between freely available work (and assuming all risks of that work and its availability) versus doing that work yourself locally.

Personal Thoughts and Convictions of Liberty

The quoted statement from NPM is the opposite of liberty. I hold the concepts of liberty and leadership above all other sociopolitical establishments and is the primary reason I continue to serve in the US Army after 19 years.

If the subject at hand were of gay rights or racial distinctions, instead of software, people would not be so generally silent on the subject. There would be loud angry boycotts and protests. While the community of NPM users and leaders may distinguish these subjects differently liberty does not. The logical conclusions then are that:

  • The community of NPM is not aware of liberty as a concept (ignorant)
  • The community of NPM is aware of liberty but finds it antagonistic to other competing values (authoritative)

Regardless of whether the thoughts and actions opposing liberty are willful, as an adherent to the practice of supporting and upholding liberty as a paramount virtue I must remove myself from that community and no longer contribute works toward it. I believe a willful opposition to liberty on principle alone is perhaps the greatest of evils and all that it takes for evil to thrive is for good persons to do nothing.

For the definition of liberty I subscribe to the preeminent treatise On Liberty by John Stuart Mill. I have included a few select quotes from the second chapter, the first of which is the thesis of the essay.

If all mankind minus one, were of one opinion, and only one person were of the contrary opinion, mankind would be no more justified in silencing that one person, than he, if he had the power, would be justified in silencing mankind.


Let us suppose, therefore, that the government is entirely at one with the people, and never thinks of exerting any power of coercion unless in agreement with what it conceives to be their voice. But I deny the right of the people to exercise such coercion, either by themselves or by their government. The power itself is illegitimate. The best government has no more title to it than the worst. It is as noxious, or more noxious, when exerted in accordance with public opinion, than when in opposition to it.


Strange it is, that men should admit the validity of the arguments for free discussion, but object to their being "pushed to an extreme;" not seeing that unless the reasons are good for an extreme case, they are not good for any case. Strange that they should imagine that they are not assuming infallibility, when they acknowledge that there should be free discussion on all subjects which can possibly be doubtful, but think that some particular principle or doctrine should be forbidden to be questioned because it is so certain, that is, because they are certain that it is certain. To call any proposition certain, while there is any one who would deny its certainty if permitted, but who is not permitted, is to assume that we ourselves, and those who agree with us, are the judges of certainty, and judges without hearing the other side.

Security Implications

Package Reassignment

NPM can arbitrarily reassign a package name without owner consent or awareness. I understand the occasional need for this and that such power is used exceedingly rarely. NPM can violate a package maintainers' intentions to unpublish by arbitrarily publishing the same code under the same name for continuity. When these two concerns coincide namespace assignment and management of all associated code are fundamentally lost to the package maintainers. This presents immediate risks to product management (for distribution) and thus secondary consequences of associated branding and product health decisions are lost to maintainers. The third level consequences apply to disruptions of contractual obligations that arise from loss of control of maintained packages.

Weak Software and a Community of Dependency Hell

The problem that precipitated the NPM action and quotes above is due to a catastrophic break from the removal a single tiny artifact. This could be characterized as the removal of a grain a straw that instantly destroyed all the buildings in a grand city upon its plucking. The problem there is not the result of an NPM directed action or the malicious convictions of a single evil agent. The problem is a community entirely reliant upon a system of dependencies, without regard for integrity or health, with NPM directly serving as the primary enabler.

This community problem exists because weak software developers value convenience above other more necessary considerations. Audits are not performed (not regularly, not ever) over included dependencies and the dependencies of the primary dependencies are certainly not thought of. These dependencies are frequently included without constraint or safety of any kind. Emerging software developers who are new to software development take this weakness as standard practice and so the weakness is the new baseline. The consequence is that discussions and thoughts of architecture are avoided out-right thereby avoiding ownership of responsibility, so therefore it becomes the user's fault when software breaks due to the dependency madness.

This problem is observed in numerous posted issues to the Atom-Beautify project. Atom-Beautify is a large project containing many dependencies. Even after the project itself is well-written and does everything it can to reinforce concepts of health and integrity it still suffers from dependency hell due to the madness that comes in from certain grandchild dependencies. When a micro nth level dependency fails to come across the line there is only one solution: We tell the user to kindly reinstall. Clearly, this is a user error (due to no fault of the user) that we are powerless to fix.

Secondary Reasons

Additional and unrelated frustrations that break Pretty Diff:

Tertiary Reason

It is hard to qualify NPM download numbers. This is absolutely not related to any project health or security concern, so it is absolutely minor. NPM download counts appear to be total downloads. This includes counts for specifically requested packages, packages downloaded as any dependency type, packages further down the dependency chain, and various versions of the same package on the same dependent and/or dependency tree. This generates some level of confusion in that a package could get 30 (hypothetical) download counts when a single unrelated package is downloaded.

The challenges:

  • There is no way to determine external downloads apart from NPM internal (NPM dependency model) downloads
  • There is no way to determine unique (manual) requests apart from total downloads, which speaks to desirability versus unintended inclusion
  • There is no way to determine download traffic source

See also: #250

It would be nice to have more qualified numbers. In the reality of open source the numbers matter very little. In the rest of the world it is a validation. The rest of the world includes:

  • potential investment
  • family (from whom I rob time to write open source)
  • software developers who don't contribute to open source
@prettydiff

This comment has been minimized.

Show comment
Hide comment
@prettydiff
Owner

prettydiff commented Mar 24, 2016

Examining http://jspm.io

@prettydiff

This comment has been minimized.

Show comment
Hide comment
@prettydiff

prettydiff Mar 24, 2016

Owner

In order to prevent potential malicious entities from taking the NPM names reserved by user austincheney I will not unpublish from NPM. Instead Pretty Diff will upgrade to a new major version number when publication at an alternate venue is announced. Pretty Diff will continue to retain version 1.x.x at NPM, but it will no longer be maintained. This plan also minimizes any disruptions associated with this move.

Owner

prettydiff commented Mar 24, 2016

In order to prevent potential malicious entities from taking the NPM names reserved by user austincheney I will not unpublish from NPM. Instead Pretty Diff will upgrade to a new major version number when publication at an alternate venue is announced. Pretty Diff will continue to retain version 1.x.x at NPM, but it will no longer be maintained. This plan also minimizes any disruptions associated with this move.

@prettydiff

This comment has been minimized.

Show comment
Hide comment
@prettydiff

prettydiff Mar 25, 2016

Owner

Custom (self-hosted) package management

Another solution I have been thinking about is to simple write a new node application that does the job of NPM, but without a registry. You don't need a centrally managed registry, because the web has already solved this problem and applied the solution with great success in the past.

URI to the rescue

URI is always unique, is universal, and can be applied with varying degrees of specificity. Its universal nature solves the NPM namespace collision problem while simultaneously ignoring the friction of public/private or commercial/free.

Listing of basic requirements for package management

  • A data file that defines the package. Something like NPM's package.json.
  • A means to tarball a directory cross-OS.
  • A location where versions are saved
  • A default version, latest, that symlinks to the latest version tarball
  • A notification service (to beacon out when a new package version is available)

Example of an applied solution

The Pretty Diff project has its own domain at http://prettydiff.com/ so I will use this as a potential demonstration. Let's presume there is a http://prettydiff.com/package which contains an index file that points to the default version via symlink so you when go to that address a tarball is returned. Let's also presume there is a http://prettydiff.com/package/versions directory, so for example http://prettydiff.com/packages/versions/1.16.37

When applied in this way the version format is completely custom so long as there is an address that points to tarball of matching version, such as http://prettydiff.com/packages/versions/1.16.37--alpha3. If you want the package to be private then move it to a web-server behind a firewall. Even better would be to move it to an internally available address/domain scheme. Since this idea is reliant upon URI the privacy of a given package is vested in the ability to resolve an address.

No service

The great thing about this is that its an application and not a service. The only required service is a HTTP server (a simple one comes prepackaged with Node). The application must run in response to a command that creates a new package tarball, puts that tarball in the proper location (a version directory on the web server), updates the default package symlink, and optionally beacons that a new version is published.

Beacons

This, like everything else in this idea, is completely open to custom definition. It would just have to be defined in the package's data file so that the format and point of access are known to the distant end where the package is used.

A client application

Unfortunately a client application would also be required to, at least, unpack the tarball. Some optional (ideal) qualities for the client-side app:

  • Install the package to a location defined in the command line PATH variable so that the package can be executed as a global command
  • Check the version against the last accessed beacon so that update notifications can be optionally presented

Potential problems

  • discoverability - It could be hard to find relevant or superior packages without a centralized registry.
Owner

prettydiff commented Mar 25, 2016

Custom (self-hosted) package management

Another solution I have been thinking about is to simple write a new node application that does the job of NPM, but without a registry. You don't need a centrally managed registry, because the web has already solved this problem and applied the solution with great success in the past.

URI to the rescue

URI is always unique, is universal, and can be applied with varying degrees of specificity. Its universal nature solves the NPM namespace collision problem while simultaneously ignoring the friction of public/private or commercial/free.

Listing of basic requirements for package management

  • A data file that defines the package. Something like NPM's package.json.
  • A means to tarball a directory cross-OS.
  • A location where versions are saved
  • A default version, latest, that symlinks to the latest version tarball
  • A notification service (to beacon out when a new package version is available)

Example of an applied solution

The Pretty Diff project has its own domain at http://prettydiff.com/ so I will use this as a potential demonstration. Let's presume there is a http://prettydiff.com/package which contains an index file that points to the default version via symlink so you when go to that address a tarball is returned. Let's also presume there is a http://prettydiff.com/package/versions directory, so for example http://prettydiff.com/packages/versions/1.16.37

When applied in this way the version format is completely custom so long as there is an address that points to tarball of matching version, such as http://prettydiff.com/packages/versions/1.16.37--alpha3. If you want the package to be private then move it to a web-server behind a firewall. Even better would be to move it to an internally available address/domain scheme. Since this idea is reliant upon URI the privacy of a given package is vested in the ability to resolve an address.

No service

The great thing about this is that its an application and not a service. The only required service is a HTTP server (a simple one comes prepackaged with Node). The application must run in response to a command that creates a new package tarball, puts that tarball in the proper location (a version directory on the web server), updates the default package symlink, and optionally beacons that a new version is published.

Beacons

This, like everything else in this idea, is completely open to custom definition. It would just have to be defined in the package's data file so that the format and point of access are known to the distant end where the package is used.

A client application

Unfortunately a client application would also be required to, at least, unpack the tarball. Some optional (ideal) qualities for the client-side app:

  • Install the package to a location defined in the command line PATH variable so that the package can be executed as a global command
  • Check the version against the last accessed beacon so that update notifications can be optionally presented

Potential problems

  • discoverability - It could be hard to find relevant or superior packages without a centralized registry.
@prettydiff

This comment has been minimized.

Show comment
Hide comment
@prettydiff

prettydiff Jul 14, 2016

Owner

Writing alternate solution for NPM. The application is called biddle.

Owner

prettydiff commented Jul 14, 2016

Writing alternate solution for NPM. The application is called biddle.

@prettydiff

This comment has been minimized.

Show comment
Hide comment
@prettydiff

prettydiff Nov 11, 2016

Owner

Pretty Diff is now published with biddle. The publication point is: http://prettydiff.com/downloads/prettydiff/

Automated distribution and installation is now as easy as:

biddle install http://prettydiff.com/downloads/prettydiff/prettydiff_latest.zip
Owner

prettydiff commented Nov 11, 2016

Pretty Diff is now published with biddle. The publication point is: http://prettydiff.com/downloads/prettydiff/

Automated distribution and installation is now as easy as:

biddle install http://prettydiff.com/downloads/prettydiff/prettydiff_latest.zip
@ackalker

This comment has been minimized.

Show comment
Hide comment
@ackalker

ackalker Nov 21, 2016

Sad to see npm/npm#10686 mentioned (and still not fixed). I actually bisected the thing down to a single commit (a botched refactoring/variable renaming attempt). To my knowledge, no-one ever looked at that bad commit as a source of the problem. I think my comment simply got drowned out in the incredible noise from people pasting entire npm logs while trying to prove that they had "the same problem"...

ackalker commented Nov 21, 2016

Sad to see npm/npm#10686 mentioned (and still not fixed). I actually bisected the thing down to a single commit (a botched refactoring/variable renaming attempt). To my knowledge, no-one ever looked at that bad commit as a source of the problem. I think my comment simply got drowned out in the incredible noise from people pasting entire npm logs while trying to prove that they had "the same problem"...

@prettydiff

This comment has been minimized.

Show comment
Hide comment
@prettydiff

prettydiff Nov 21, 2016

Owner

The one that really REALLY did it for me was #5082.

The problem for me isn't that there are defects in software or even the severity of the defects. All software has defects of which some are hard to solve even by the people best positioned to provide the solutions. My problem is that the defects are so incredibly hard to solve that they remain unsolved for so long. Normally in open source software defects will remain open for a long time due to the limited time of volunteers. NPM has a paid staff, hundreds of contributors, thousands of participants, and millions of users. Limited man power isn't an argument for NPM when problems are so dangerously critical.

This is a problem for me, because it is symptomatic of bigger problems. Why would such foundational problems be so hard to solve for or so challenging to detect? Is it because of how the software is organized, managed, or executed? This line of thinking led me to believe the primary problem of NPM is its over-reliance on modularity.

When I was looking into issue 5082. I had trouble following the code. The application was divided into many different modules written by NPM staff. The NPM code style is really sloppy which opens the code to some ambiguity and the tar component was binary heavy with great use of typed arrays. Each of these qualities compounded the problem and likely prevented entry from many people willing to solve this problem. Had the various components been centralized into a single application the problem would have been easier to detect. Even though the code would still be sloppy the problem would be confined to a single known location, so at the very least you would know where to start and could test your way through until test criteria is firmly defined.

Although I identified a technical hurdle limiting understandability of the code it wasn't until I heard the responses to the left-pad problem that I realized the problem isn't technical at all. It is purely a management problem of the code. This is a design problem more so than an execution problem. This is the thinking that led me to design an alternative. I knew I could do this not because of confidence in my programming skills, but because they identified a unique problem to solve that they are less willing or capable of solving themselves.

Owner

prettydiff commented Nov 21, 2016

The one that really REALLY did it for me was #5082.

The problem for me isn't that there are defects in software or even the severity of the defects. All software has defects of which some are hard to solve even by the people best positioned to provide the solutions. My problem is that the defects are so incredibly hard to solve that they remain unsolved for so long. Normally in open source software defects will remain open for a long time due to the limited time of volunteers. NPM has a paid staff, hundreds of contributors, thousands of participants, and millions of users. Limited man power isn't an argument for NPM when problems are so dangerously critical.

This is a problem for me, because it is symptomatic of bigger problems. Why would such foundational problems be so hard to solve for or so challenging to detect? Is it because of how the software is organized, managed, or executed? This line of thinking led me to believe the primary problem of NPM is its over-reliance on modularity.

When I was looking into issue 5082. I had trouble following the code. The application was divided into many different modules written by NPM staff. The NPM code style is really sloppy which opens the code to some ambiguity and the tar component was binary heavy with great use of typed arrays. Each of these qualities compounded the problem and likely prevented entry from many people willing to solve this problem. Had the various components been centralized into a single application the problem would have been easier to detect. Even though the code would still be sloppy the problem would be confined to a single known location, so at the very least you would know where to start and could test your way through until test criteria is firmly defined.

Although I identified a technical hurdle limiting understandability of the code it wasn't until I heard the responses to the left-pad problem that I realized the problem isn't technical at all. It is purely a management problem of the code. This is a design problem more so than an execution problem. This is the thinking that led me to design an alternative. I knew I could do this not because of confidence in my programming skills, but because they identified a unique problem to solve that they are less willing or capable of solving themselves.

@prettydiff

This comment has been minimized.

Show comment
Hide comment
@prettydiff

prettydiff Dec 2, 2016

Owner

Use via biddle is now available in production. The release of branch 2.1.15 with issue #381 will mark a resolution to this issue. Once a convenient name and catalogue system are worked (key value pair) OS and language agnostic self-hosted/managed packages via CDN will be the future.

Owner

prettydiff commented Dec 2, 2016

Use via biddle is now available in production. The release of branch 2.1.15 with issue #381 will mark a resolution to this issue. Once a convenient name and catalogue system are worked (key value pair) OS and language agnostic self-hosted/managed packages via CDN will be the future.

@prettydiff prettydiff referenced this issue Dec 2, 2016

Merged

2.1.15 #395

@prettydiff prettydiff closed this in #395 Dec 4, 2016

@prettydiff

This comment has been minimized.

Show comment
Hide comment
@prettydiff

prettydiff Dec 4, 2016

Owner

Publishing updates with biddle broken.

Owner

prettydiff commented Dec 4, 2016

Publishing updates with biddle broken.

@prettydiff

This comment has been minimized.

Show comment
Hide comment
@prettydiff

prettydiff Dec 4, 2016

Owner

Closing and working biddle defect from #396 and https://github.com/prettydiff/biddle/issues

Owner

prettydiff commented Dec 4, 2016

Closing and working biddle defect from #396 and https://github.com/prettydiff/biddle/issues

@prettydiff prettydiff closed this Dec 4, 2016

@jamiebuilds

This comment has been minimized.

Show comment
Hide comment
@jamiebuilds

jamiebuilds Feb 3, 2017

If anyone runs into this in the future, I've forked the project and published it to npm: https://www.npmjs.com/package/liberty-prettydiff. Shouldn't have to worry about it going out of sync either.

Peace

jamiebuilds commented Feb 3, 2017

If anyone runs into this in the future, I've forked the project and published it to npm: https://www.npmjs.com/package/liberty-prettydiff. Shouldn't have to worry about it going out of sync either.

Peace

@prettydiff

This comment has been minimized.

Show comment
Hide comment
@prettydiff

prettydiff Feb 4, 2017

Owner

@thejameskyle

As a maintainer of Yarn there are a couple things developers need from distribution to be successful.

Distribution

Required Items

Integrity

Yarn has checksums, so this one is done.

Variations

I should have the ability to publish my application in a variety of different flavors simultaneously. If somebody wanted to tinker with my application they should be able to request the entire thing unit tests and all (4mb zip). If they want only command line operation (170kb zip) then they shouldn't have download the other 4mb of nonsense. biddle does this and Yarn could too.

Because NPM and Yarn don't have this many developers publish one version to their git server for storage, and different version to NPM that is only good for use in a limited context. Sometimes, if the application is popular enough and well staffed, they will publish multiple variations to NPM to manually work around this limitation. All of this can be automated.

Packaged Dependencies

Dependency management sucks. It's hard. Fine, I got it. Either you are taking that pain as a product owner or you are forcing that stupidity upon your users. It makes for horribly fragile software that breaks all the time. This problem is magnified for Chinese users behind the Great Firewall. They really hate this madness.

This NPM style dependency madness should be optional and disabled by default. Its great for software developers who are more consumers than publishers and really need that level of convenience, but I wouldn't trust or endorse any application in production like this.

By default dependencies should be rolled up into the published artifact. Consider, for instance, when you download something from Steam. Are you download one large file or a million snowflakes? The installation step should be automatically managed by the distribution app according to instructions provided by the application's author. There is no dependency madness and limited configuration hell. The user just requests their application and achieves immediate satisfaction.

Nice to haves

Alternate Repositories of the Author's Choosing

I am a big believer in liberty. It is ultimately why I gave up on NPM. Regardless of all the best wishes, intentions, and desires a central repository is ultimately a beacon of censorship by two means. On one hand if there is a single universal standard repo then anything they don't like they could forbid, which is censorship by omission. Secondly, they could silence or eliminate something that is published, which is a purge. Let's not forget the primary cause of the Left-Pad failure was a trademark conflict.

The cure for this form of censorship to allow a package publisher the means to publish to any number of locations of their choosing.

Privacy

When a package author has the freedom to choose where they publish to they get privacy for free. It is simply a matter of moving the publication point from the public network to the internal network. Privacy for free.

Confidentiality

I work for the military where confidentiality is important. If the means to distribute software requires beacons to a centralized third party the military will never adopt it. If, on the other hand, there is no third party network traffic and confidentiality labels can be automatically applied to software at the publication point and are read before unpacking the zip the military would be all over it. This isn't just useful to the military, but is also useful to the corporate enterprise where legal documents and financial documents exist at a higher level of classification.

Owner

prettydiff commented Feb 4, 2017

@thejameskyle

As a maintainer of Yarn there are a couple things developers need from distribution to be successful.

Distribution

Required Items

Integrity

Yarn has checksums, so this one is done.

Variations

I should have the ability to publish my application in a variety of different flavors simultaneously. If somebody wanted to tinker with my application they should be able to request the entire thing unit tests and all (4mb zip). If they want only command line operation (170kb zip) then they shouldn't have download the other 4mb of nonsense. biddle does this and Yarn could too.

Because NPM and Yarn don't have this many developers publish one version to their git server for storage, and different version to NPM that is only good for use in a limited context. Sometimes, if the application is popular enough and well staffed, they will publish multiple variations to NPM to manually work around this limitation. All of this can be automated.

Packaged Dependencies

Dependency management sucks. It's hard. Fine, I got it. Either you are taking that pain as a product owner or you are forcing that stupidity upon your users. It makes for horribly fragile software that breaks all the time. This problem is magnified for Chinese users behind the Great Firewall. They really hate this madness.

This NPM style dependency madness should be optional and disabled by default. Its great for software developers who are more consumers than publishers and really need that level of convenience, but I wouldn't trust or endorse any application in production like this.

By default dependencies should be rolled up into the published artifact. Consider, for instance, when you download something from Steam. Are you download one large file or a million snowflakes? The installation step should be automatically managed by the distribution app according to instructions provided by the application's author. There is no dependency madness and limited configuration hell. The user just requests their application and achieves immediate satisfaction.

Nice to haves

Alternate Repositories of the Author's Choosing

I am a big believer in liberty. It is ultimately why I gave up on NPM. Regardless of all the best wishes, intentions, and desires a central repository is ultimately a beacon of censorship by two means. On one hand if there is a single universal standard repo then anything they don't like they could forbid, which is censorship by omission. Secondly, they could silence or eliminate something that is published, which is a purge. Let's not forget the primary cause of the Left-Pad failure was a trademark conflict.

The cure for this form of censorship to allow a package publisher the means to publish to any number of locations of their choosing.

Privacy

When a package author has the freedom to choose where they publish to they get privacy for free. It is simply a matter of moving the publication point from the public network to the internal network. Privacy for free.

Confidentiality

I work for the military where confidentiality is important. If the means to distribute software requires beacons to a centralized third party the military will never adopt it. If, on the other hand, there is no third party network traffic and confidentiality labels can be automatically applied to software at the publication point and are read before unpacking the zip the military would be all over it. This isn't just useful to the military, but is also useful to the corporate enterprise where legal documents and financial documents exist at a higher level of classification.

@jamiebuilds

This comment has been minimized.

Show comment
Hide comment
@jamiebuilds

jamiebuilds Feb 4, 2017

Variations: If a single package is this flexible, you should be breaking it apart into separate packages. Many tiny packages are controversial, but ultimately the right way to share code.

In regards to file size, Yarn has yarn clean and the .yarnclean file for removing unnecessary content.

Packaged Dependencies: I disagree, npm's best innovation was the design of the node_modules folder and it's recursive system of resolving dependencies. While it can be optimized, it is the best solution to date. Which is why other package managers have copied it.

Reliability and performance are separate matters that can be addressed. I have friends in Asia-Pacific who quite like the performance and reliability improvements of Yarn.

Alternative Repositories: You've always been able to publish and install packages to other locations with both npm and Yarn. npm has done nothing to lock you into their platform.

Also the primary cause of left-pad was a whiny brat.

Privacy: (see "Alternative Repositories")

Confidentiality: (see "Alternative Repositories")

jamiebuilds commented Feb 4, 2017

Variations: If a single package is this flexible, you should be breaking it apart into separate packages. Many tiny packages are controversial, but ultimately the right way to share code.

In regards to file size, Yarn has yarn clean and the .yarnclean file for removing unnecessary content.

Packaged Dependencies: I disagree, npm's best innovation was the design of the node_modules folder and it's recursive system of resolving dependencies. While it can be optimized, it is the best solution to date. Which is why other package managers have copied it.

Reliability and performance are separate matters that can be addressed. I have friends in Asia-Pacific who quite like the performance and reliability improvements of Yarn.

Alternative Repositories: You've always been able to publish and install packages to other locations with both npm and Yarn. npm has done nothing to lock you into their platform.

Also the primary cause of left-pad was a whiny brat.

Privacy: (see "Alternative Repositories")

Confidentiality: (see "Alternative Repositories")

@prettydiff

This comment has been minimized.

Show comment
Hide comment
@prettydiff

prettydiff Feb 6, 2017

Owner

For all interested parties Pretty Diff will be changing its license in the next published version, please see: #269

Owner

prettydiff commented Feb 6, 2017

For all interested parties Pretty Diff will be changing its license in the next published version, please see: #269

@jamiebuilds

This comment has been minimized.

Show comment
Hide comment
@jamiebuilds

jamiebuilds Feb 6, 2017

So to be explicit, your goal of this license is to be spiteful against npm yes?

jamiebuilds commented Feb 6, 2017

So to be explicit, your goal of this license is to be spiteful against npm yes?

@prettydiff

This comment has been minimized.

Show comment
Hide comment
@prettydiff

prettydiff Feb 6, 2017

Owner

I am not centering this position solely upon NPM. It is an anti-authority/decentralization move.

In my opinion the greatest technological democracies are BGP and DNS. They are decentralized. When one item in those systems falls off the interest isn't harmed, due to redundancy, though it might get a little slower. Those technologies provide no means to enact of enforce censorship. All they do is share routes or name resolution. Even if they wanted it or were so ordered they can only be made to harm their internal users. The network at large remains free and unrestrained, because a change to one item in those systems is by no means a change to multiple items in those systems.

The problem with central authorities is two fold. If everybody is reliant upon the central authority for content or distribution the central authority can enact censorship by blocking publication from users it wishes. There is likely a very good reason to block certain content whether it be obscene material, disgusting and hateful material, or simply an intellectual property violation. Regardless of the reason and no matter how noble the intention this is still censorship. Where the availability of certain content becomes a legal violation it is best to let the legal authorities impose the necessary corrections according to the law, which can be challenged upon appeal.

The other side of that coin is that now the system is a bit less diverse. Most of us don't like hate speech and wish it would just go away, but this is a slippery slope. Eventually matters of political discourse quickly become hateful and minority opinions are better suppressed. It is often necessary for the health and well being of the system to prevent trolling. The challenge is that some opinions are naturally upsetting and challenge the common order. Whether these opinions challenge commonly held assumptions, or the majority opinion, does not necessarily mean they aren't worth having. If you don't enjoy a particular sort of content you don't have to consume it, which isn't the same thing as preventing other people from access by blocking it from the central system.

When there is redundancy and no central system the content remains alive and available long after it is removed from a single location. In a centralized authority there is no redundancy. Sure NPM has mirrors and CDNs so if a file cannot be retrieved from one known address it can be fetched from another address. But, in order to be fetched it is resolved against a name that is centrally managed. For instance I am essentially squatting on the NPM name prettydiff. I cannot stop squatting on this name or allow it to be replaced with something else because then the 75,000 users who download this every month will be harmed. A publisher wishing to use that name for something that is actively maintained in the system cannot, because I am squatting there. For fear of harming the system, and its users, I cannot even give this name away if I wanted to. This is a huge limitation.

Can I publish to Yarn's repository independent of NPM name resolution. I have been told no. Therefore Yarn is even less free than NPM, because it is subject to the authoritarianism on names at NPM as well as imposing a similar authority of its own. The argument in defense is to imagine the horror if somebody registered jquery on Yarn and publishing something different than what is on NPM? My argument is that they would get different software. That isn't bad. It is just different.

Half the problem has long since been solved. The bottle neck is the simplicity and convenience of a single name without a context. This is a problem that has long been solved with conventions such as URI and IP. It has mostly been solved with hashes, aside from the rare hash collision. If the application resolution were by address, opposed to name, then any user could publish an application to Yarn completely independent of NPM, because they are identified differently, such as different domains for resolution.

The other half the problem is to prevent a single authority to own, limit, or regulate what can be published or accessed via a ubiquitous system. This is solved by encouraging duplication and republication while providing rights for those who republish above and beyond those who merely consume the application. Trust and integrity are verifiable with hash comparisons regardless of where an application is served from.

NPM is popular because it is convenient for developers. It isn't convenient for application consumers. If you had to choose between (fill in the blank) and a more democratic approach what would you choose? Historically speaking there is always valid cause to disagree, but the means to allow that disagreement is itself an independent decentralized opinion.

Owner

prettydiff commented Feb 6, 2017

I am not centering this position solely upon NPM. It is an anti-authority/decentralization move.

In my opinion the greatest technological democracies are BGP and DNS. They are decentralized. When one item in those systems falls off the interest isn't harmed, due to redundancy, though it might get a little slower. Those technologies provide no means to enact of enforce censorship. All they do is share routes or name resolution. Even if they wanted it or were so ordered they can only be made to harm their internal users. The network at large remains free and unrestrained, because a change to one item in those systems is by no means a change to multiple items in those systems.

The problem with central authorities is two fold. If everybody is reliant upon the central authority for content or distribution the central authority can enact censorship by blocking publication from users it wishes. There is likely a very good reason to block certain content whether it be obscene material, disgusting and hateful material, or simply an intellectual property violation. Regardless of the reason and no matter how noble the intention this is still censorship. Where the availability of certain content becomes a legal violation it is best to let the legal authorities impose the necessary corrections according to the law, which can be challenged upon appeal.

The other side of that coin is that now the system is a bit less diverse. Most of us don't like hate speech and wish it would just go away, but this is a slippery slope. Eventually matters of political discourse quickly become hateful and minority opinions are better suppressed. It is often necessary for the health and well being of the system to prevent trolling. The challenge is that some opinions are naturally upsetting and challenge the common order. Whether these opinions challenge commonly held assumptions, or the majority opinion, does not necessarily mean they aren't worth having. If you don't enjoy a particular sort of content you don't have to consume it, which isn't the same thing as preventing other people from access by blocking it from the central system.

When there is redundancy and no central system the content remains alive and available long after it is removed from a single location. In a centralized authority there is no redundancy. Sure NPM has mirrors and CDNs so if a file cannot be retrieved from one known address it can be fetched from another address. But, in order to be fetched it is resolved against a name that is centrally managed. For instance I am essentially squatting on the NPM name prettydiff. I cannot stop squatting on this name or allow it to be replaced with something else because then the 75,000 users who download this every month will be harmed. A publisher wishing to use that name for something that is actively maintained in the system cannot, because I am squatting there. For fear of harming the system, and its users, I cannot even give this name away if I wanted to. This is a huge limitation.

Can I publish to Yarn's repository independent of NPM name resolution. I have been told no. Therefore Yarn is even less free than NPM, because it is subject to the authoritarianism on names at NPM as well as imposing a similar authority of its own. The argument in defense is to imagine the horror if somebody registered jquery on Yarn and publishing something different than what is on NPM? My argument is that they would get different software. That isn't bad. It is just different.

Half the problem has long since been solved. The bottle neck is the simplicity and convenience of a single name without a context. This is a problem that has long been solved with conventions such as URI and IP. It has mostly been solved with hashes, aside from the rare hash collision. If the application resolution were by address, opposed to name, then any user could publish an application to Yarn completely independent of NPM, because they are identified differently, such as different domains for resolution.

The other half the problem is to prevent a single authority to own, limit, or regulate what can be published or accessed via a ubiquitous system. This is solved by encouraging duplication and republication while providing rights for those who republish above and beyond those who merely consume the application. Trust and integrity are verifiable with hash comparisons regardless of where an application is served from.

NPM is popular because it is convenient for developers. It isn't convenient for application consumers. If you had to choose between (fill in the blank) and a more democratic approach what would you choose? Historically speaking there is always valid cause to disagree, but the means to allow that disagreement is itself an independent decentralized opinion.

@jamiebuilds

This comment has been minimized.

Show comment
Hide comment
@jamiebuilds

jamiebuilds Feb 6, 2017

So yes.

Your idea of "freedom" and "liberty" are limiting what people are allowed to do (Edit) based on what you think they should be doing.

jamiebuilds commented Feb 6, 2017

So yes.

Your idea of "freedom" and "liberty" are limiting what people are allowed to do (Edit) based on what you think they should be doing.

@jamiebuilds

This comment has been minimized.

Show comment
Hide comment
@jamiebuilds

jamiebuilds Feb 6, 2017

In my opinion the greatest technological democracies are BGP and DNS. They are decentralized.

Ah yes, the completely decentralized system: DNS

dnstreefamily

jamiebuilds commented Feb 6, 2017

In my opinion the greatest technological democracies are BGP and DNS. They are decentralized.

Ah yes, the completely decentralized system: DNS

dnstreefamily

@jamiebuilds

This comment has been minimized.

Show comment
Hide comment
@jamiebuilds

jamiebuilds Feb 6, 2017

Good thing this code is safe on GitHub, wouldn't want one company in control of this code which could take it down at any moment for breaking the law. https://github.com/github/dmca

jamiebuilds commented Feb 6, 2017

Good thing this code is safe on GitHub, wouldn't want one company in control of this code which could take it down at any moment for breaking the law. https://github.com/github/dmca

@jamiebuilds

This comment has been minimized.

Show comment
Hide comment
@jamiebuilds

jamiebuilds Feb 6, 2017

And oh god, I can't believe Yarn has the nerve to assert it's own authority

screen shot 2017-02-06 at 12 42 37 pm

jamiebuilds commented Feb 6, 2017

And oh god, I can't believe Yarn has the nerve to assert it's own authority

screen shot 2017-02-06 at 12 42 37 pm

@jamiebuilds

This comment has been minimized.

Show comment
Hide comment
@jamiebuilds

jamiebuilds Feb 6, 2017

If you want to throw a tantrum about npm following the law and taking down a package after a dispute: Fine, at least your heart seems to be in the right place.

But you aren't punishing npm at all, you're punishing developers like me who just want to write code and not deal with 8 separate package managers.

I'm not adding Biddle as a dependency, I don't trust you after this incident– nor should anyone.

jamiebuilds commented Feb 6, 2017

If you want to throw a tantrum about npm following the law and taking down a package after a dispute: Fine, at least your heart seems to be in the right place.

But you aren't punishing npm at all, you're punishing developers like me who just want to write code and not deal with 8 separate package managers.

I'm not adding Biddle as a dependency, I don't trust you after this incident– nor should anyone.

@prettydiff

This comment has been minimized.

Show comment
Hide comment
@prettydiff

prettydiff Feb 6, 2017

Owner

Your idea of "freedom" and "liberty" are limiting what people are allowed to do (Edit) based on what you think they should be doing.

Perhaps. This software retains a copyright and license, so it isn't in the public domain, which means it is regulated to some extent. As the project owner, for better or worse, I control that direction.

But you aren't punishing npm at all, you're punishing developers like me who just want to write code and not deal with 8 separate package managers.

My intention is not to punish anybody. If anybody is being punished for this decision it is primarily this project and software reliant upon it. I have lost considerable traffic, and as a result external credibility. In January of 2016 I had just shy of a million NPM downloads almost entirely from outside the NPM system.

In conclusion this is free software. I have invested my time and money in this work without ever asking for payment or labor. I provide it to you and everybody else as a courtesy, because community feedback makes identifying defects and suggesting enhancements far easier. We are all better for it. I am not forcing you to use or consume this software. If you choose to not use this software I don't really care, because countless others will. There are many alternatives available of which most are far more popular and on NPM/Yarn. Go use one of those.

I will leave you with this: http://www.constitution.org/jsm/liberty.htm

I'm not adding Biddle as a dependency

I have gotten by with git submodules.

Owner

prettydiff commented Feb 6, 2017

Your idea of "freedom" and "liberty" are limiting what people are allowed to do (Edit) based on what you think they should be doing.

Perhaps. This software retains a copyright and license, so it isn't in the public domain, which means it is regulated to some extent. As the project owner, for better or worse, I control that direction.

But you aren't punishing npm at all, you're punishing developers like me who just want to write code and not deal with 8 separate package managers.

My intention is not to punish anybody. If anybody is being punished for this decision it is primarily this project and software reliant upon it. I have lost considerable traffic, and as a result external credibility. In January of 2016 I had just shy of a million NPM downloads almost entirely from outside the NPM system.

In conclusion this is free software. I have invested my time and money in this work without ever asking for payment or labor. I provide it to you and everybody else as a courtesy, because community feedback makes identifying defects and suggesting enhancements far easier. We are all better for it. I am not forcing you to use or consume this software. If you choose to not use this software I don't really care, because countless others will. There are many alternatives available of which most are far more popular and on NPM/Yarn. Go use one of those.

I will leave you with this: http://www.constitution.org/jsm/liberty.htm

I'm not adding Biddle as a dependency

I have gotten by with git submodules.

@jamiebuilds

This comment has been minimized.

Show comment
Hide comment
@jamiebuilds

jamiebuilds Feb 7, 2017

In conclusion this is free software

I also write a lot of free software and am working to get more people paid for the free software that they create. I don't go punishing my users like you do though.

My intention is not to punish anybody.

Well there's intention and then there's effect. I don't really care what your intentions are.

you're punishing developers like me who just want to write code and not deal with 8 separate package managers.

If anybody is being punished [... it is the] software reliant upon it.

So you admit you're punishing users? And you think that's okay because...

There are many alternatives available of which most are far more popular and on NPM/Yarn. Go use one of those.

I don't actually use this library for anything. I'm just here because you're being a jerk.

PS: I don't think you have a good understanding of "liberty", I think you should read some more.

jamiebuilds commented Feb 7, 2017

In conclusion this is free software

I also write a lot of free software and am working to get more people paid for the free software that they create. I don't go punishing my users like you do though.

My intention is not to punish anybody.

Well there's intention and then there's effect. I don't really care what your intentions are.

you're punishing developers like me who just want to write code and not deal with 8 separate package managers.

If anybody is being punished [... it is the] software reliant upon it.

So you admit you're punishing users? And you think that's okay because...

There are many alternatives available of which most are far more popular and on NPM/Yarn. Go use one of those.

I don't actually use this library for anything. I'm just here because you're being a jerk.

PS: I don't think you have a good understanding of "liberty", I think you should read some more.

@jamiebuilds

This comment has been minimized.

Show comment
Hide comment
@jamiebuilds

jamiebuilds Feb 7, 2017

Also I'm not really worried, your new user-hostile license contradicts its primary function so you can't really make an argument against what I'm doing.

I'm not telling you why though, because if you're gonna go writing your own license you should talk to a lawyer instead of pouring your frustrations over a keyboard.

jamiebuilds commented Feb 7, 2017

Also I'm not really worried, your new user-hostile license contradicts its primary function so you can't really make an argument against what I'm doing.

I'm not telling you why though, because if you're gonna go writing your own license you should talk to a lawyer instead of pouring your frustrations over a keyboard.

Repository owner locked and limited conversation to collaborators Mar 16, 2017

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.