Join GitHub today
GitHub is home to over 36 million developers working together to host and review code, manage projects, and build software together.Sign up
ecstatic's Past, Present and Future #259
I once had a friend tell me that for many people, the second or maybe first thing that someone gets really into, usually in their early 20s, becomes a memory like what people have about punk rock in New York in the '80s. "We were There and we were Doing It," these people would say.
We were of course talking about Node.js. In the neighborhood of 2009 I was in my early 20s, and in fact Node.js was about the second thing I really got into (after webcomics in the mid-aughts as a teen). We were certainly There - with perhaps 40 people in the #node.js irc room around then - and we were definitely Doing It - nobody had written any of the node libraries yet and the possibilities seemed infinite. Someone ported Sinatra to Node.js, someone hacked up a baby package manager on top of CouchDB, a friend of mine wrote a huge mess of utility modules and me? I learned Node.js as a hobby while working on my engineering degree.
It was just a few years later that I got my first job out of college with Nodejitsu. Nodejitsu is probably most famous for being an enterprise npm host before npm Inc was founded in 2014, having incredible journeyed shortly after, but when I worked for them in 2011/2012 they were actually supposed to be the next Heroku. I was an early fan of their platform and to this day I will maintain that the ux of their cli tool jitsu was solid. Mostly due to my association with substack I'm sure, they agreed to hire me as a support engineer.
It was at Nodejitsu that I wrote ecstatic. Over the summer of 2011 the team decided that the way to reach market dominance was going to be building and open sourcing a web framework, called Flatiron, and by the time fall was in full swing they cut an initial release.
Being a support engineer meant I was largely responsible for fielding questions and issues that people had with Flatiron. There were many. A lot of this was understandable given the relative age of the framework: A module had an edge case nobody thought of, or someone had fat fingered a poorly tested procedure and shipped a bug. Often, I would be the one to issue fixes.
One of the things that Flatiron was missing that a lot of people complained about was a static fileserving component. As far as I know Flatiron never shipped one! I think they expected people to use cloudhead's module but, while apparently rfc 2616 compliant, it wasn't written with middlewares in mind. So, I decided to try to solve the problem by doing it myself.
My initial version was really bad - a 30 line call to readFile - and I needed a lot of help to get it working. Commits through 2012 were mostly me and substack. I found an ally in him on this project: I needed something to hand to people trying to use Flatiron, and substack wanted to drop express. We worked together and shipped something.
Nodejitsu never adopted ecstatic as their own. After my time at Nodejitsu, for better or worse, ecstatic remained my responsibility alone. In the end, Flatiron didn't gain an appreciable amount of market share and in fact ecstatic eventually dropped Flation support.
For the past 7 years or so, ecstatic has largely been a solo operation. That's not to say that I haven't received any patches in this time, and in fact you can see a whole list of contributors due to someone at Google telling me that I needed to give them partial copyright. However, I've been the primary reviewer and gatekeeper on all external contributions and otherwise the primary developer of new features.
On April 23rd, I received a stern letter from npm support. As far as I can gather, this is the series of events that transpired:
You may recall the case of left-pad. In case you missed the story in 2016, it went like this: A developer registered a project on npm named "kik", a labor of love open source framework. One day, a corporation came along and demanded he change his project name. When he objected, npm gave him no choice in the matter. Presented with this decision, he ragequit and had npm delete every single one of his modules from npm. This broke a whole bunch of code because other projects were depending on his work. npm, in a controversial decision, restored the specific package@version being used by babel and other breaking tools.
As a developer, I at the time sided with npm. After all, this guy threw a temper tantrum that broke a bunch of stuff for everyone! His packages were open source, so npm was legally free to host them. He didn't have a leg to stand on!
But it also shows how little control you have when you use the npm platform. They're free to modify your packages at any time without your consent, while refusing to remove data when asked. When this company came down on this guy, npm left him out to dry and there was nothing he could do about it.
I think my recent problems have further highlighted some of these issues with npm. It's not that I don't take security seriously - I think it's important that people can't get hacked - but on a certain level I definitely felt like npm was shaking me down for free work, and it didn't feel good.
This is what happened next:
This is where http-server and mime come in.
http-server to this day remains the primary connection to Nodejitsu. This package was and is primarily intended to be used as a development server, and these days is ultimately a thin wrapper around ecstatic. Funny enough, ecstatic ships its own cli component, but inertia and historically better docs win here.
mime is a simple package that has somehow caused me, and now the http-server maintainers, a whole mess of grief. ecstatic uses mime to intelligently set the content-type header in responses. The mime module went through a major rewrite a few years ago, and dropped support for .types files, which I believe are an Apache thing. Either way, the upgrade path was fairly painful but eventually happened about a month ago.
Unfortunately, http-server is a little behind on the upgrade and it turns out it's a bit of a beast - an honest surprise. The ramifications of this are that the http-server team's package broke when npm unpublished the ecstatic versions in a way that couldn't easily be fixed by an upgrade on http-server's end. Unsurprisingly, people were pretty angry.
Over the years, I've become more and more disillusioned with open source software, and these recent events with npm have been a pretty consistent demonstration of a number of the issues I've had.
When I was a teen and first learning about Linux, one of the major things I read about was the difference between Free Software and Open Source Software. Free Software - free as in Freedom, not beer - was software that was supposed to be protected from corporate interests by ensuring that modifications to source code would by license need to be shared with the world, and came from many of the frustrations of working with proprietary, properly closed source software over the years. Open Source, on the other hand, is concerned with a different flavor of freedom. It includes the freedom to not think very hard when hacking on cool side projects sure - and I think this is why I liked using MIT licenses in 2011 - but crucially, it also includes the freedom to use and modify someone's code for internal use without contributing those changes upstream. This is by design.
This manifests as a freedom to exploit. Choosing a license like the MIT license becomes sort of a Devil's bargain: Your project is significantly more likely to get used by other people and become low key famous, but because nobody has an obligation to help you, they won't. From a business perspective this makes total sense. From my perspective: nobody ever paid me to work on ecstatic outside my time at Nodejitsu, and in fact nobody was ever paid to work on ecstatic long term.
In general, open source is pretty thankless. People complain a lot and nobody helps you. Maintaining a non-famous non-sponsored module is lonely and exhausting. Often, somebody will come by with not a suggestion but a complaint, and then get huffy at me when I don't have time to work on it. People don't seem to understand that not only do I typically work a full time job but I'm also extremely non-neurotypical in a way that significantly decreases my spoons when things are bad. This doesn't even mention life events that get in the way: just this week as I've been dealing with these issues, I lost my job and had to attend a funeral.
Meanwhile, I hear about people making bank off of my free labor:
and see them get angry when more free labor is denied :
It's especially funny to me when people think I'll be offended by them using someone else's module. Oh, not my problem anymore? Sick!
But this is what I mean. When working on open source, people expect things from you for little reward, and eventually the open source maintainers that weren't able to monetize their work burn out and become proper leftists. It's often not a matter of if, but when.
Mikeal observed that what counts as good code changes as well, and this is further complicated by the fact that I was an abysmally bad junior developer in 2011. Not only are there strange issues with the API - multiple ways of spelling each option and half-baked error handling being some of my favorites - but the code quality is also quite simply bad. Structurally I find the code pretty confusing and I blame it for issues such as this edge case around 404.html files and directory listings. In many ways, ecstatic isn't a very good base on which to build a successful static fileserving component. In another universe, I would have probably rewritten it from scratch.
As it stands, I see myself drifting away from Node.js and the code I've written in it. I started writing Python as my daily driver in late 2016, something I've been very happy about. I stopped working on wzrd.in years ago. I left the IRC room in a huff after my concerns weren't taken seriously. I've fallen behind on what's now considered idiomatic ecmascript. I'm beginning to feel a little like an aged rock musician with a small and aging fanbase and a boring, regular life removed from the heyday - a little like J Mascis buying a VW Golf.
But I kept maintaining ecstatic, the last project connecting me to who I was in 2011. Despite all of the issues I've had with the Node.js community, with open source, with angry people demanding things from me, with all the open issues I couldn't close because of missing spoons, I kept shipping small fixes and merging people's pull requests. Now, with these recent developments with npm and the security advisory, I've finally decided I've had enough.
Here's the plan.
Here are some suggested migration paths to mitigate this:
Finally: A thank you to everyone who has used ecstatic in kindness, to all that have sent me encouraging words, and to everyone who has sent me patches over the years. Up to this point, I would have said that you've made all this work worthwhile.
So long, and thanks for all the fish.
 To be fair to this poster, he eventually thought better of things, edited his comment to be less sassy and left some nicer ones. However, screenshots are forever, and even nice people can say mean things sometimes.
referenced this issue
May 8, 2019
Just want to thank you for being open and honest and working hard on ecstatic. I think it's really valuable to hear this perspective, especially from someone who does maintain a widely-used module. I'm sorry things turned out this way and you deserve better. For what it's worth, I've been a happy user of ecstatic for my personal site's local development server for years and I really appreciate being able to use it for so long.
I'm also curious if you think you'll be using the GPL more in the future? Or just releasing less (Node) software? Either way best of luck in whatever your future holds, and I hope the life issues clear up soon. Take care of yourself <3
Are these going to be pushed to NPM before you check out? Looking at the package on NPM, only 4.1.2 of the ones you listed is published.
@jfhbrook Thank you too! You have always made it worthwhile to hack on ecstatic.
I'm surprised but fully understand your sentiment. I'm gonna miss having a web server to hack on (as opposed to configure e.i. nginx and apache) to validate front end ideas.
Sorry about not finding time to fix the mime type stuff. It has been in the back of my mind months (or years perhaps).