What's the deal with breaking changes?

[benbalter]

Moving the discussion from #1109 (comment), via @dhcole:

I understand this position for application development, but with Jekyll, the problem is more complicated. Jekyll is an application for people who download and run it on their own. But it's a service for the countless more who use it as part of gh-pages. And services are not semver'd. There's no way for me to set a version on what Jekyll engine is used when I push to gh-pages.

This makes unexpected errors problematic, because doing the same thing (change content, git push) results in unpredictable behavior (something that was working is now broken and requires extra troubleshooting).

This sounds like an internal discussion between Jekyll maintainers and GitHub staff, since they are not necessary the same now. Probably this kind of an issue has more to do with whether GitHub should have updated pages to use Jekyll v1 before addressing it's issues with backwards compatibility. Not sure there's an issue queue for that though :)

The main question is this: Should gh-pages be a consistent hosting service, or is it expectable to have gh-pages break from time to time with major updates to Jekyll?

How you answer that question would largely answer how you should proceed here. If the answer is consistency, then I'd think you'd need to preserve some less palatable conventions like this, because breaking them is actually less desirable. The exceptions of using an application, where you have control over the version, and a service, where you do not, are different and important to consider.

My preference is clearly toward a reliable service, and I think application development should consider that as a priority. I defer to @parkr @mattr- @mojombo and @benbalter on how to handle from here.

[parkr]

As I said in the other issue, Ben and I went through a couple changes to improve backwards-compatibility, but ignored the exclude and include options because they had changed so long ago. It was my oversight and "near-sightedness" about jekyll's CLI usage that caused it in the end - I apologize!

The philosophy Ben shared with me and one I think we should take to hear is don't modify behaviour on GitHub Pages without plenty of warning (e.g. relative vs. absolute permalinks). When things go wrong and are changed, have an easily-findable docs page which explains the problem and solution in clear terms. Hopefully the Jekyll blog posts will help with this in the future. Whenever GH Pages updates to a newer version of Jekyll, the blog post on jekyllrb.com will be there to link to (if so desired).

To some degree, we'll need allow for breaking changes. At the same time, I agree that we should minimize the frequency of breaking changes and do whatever we can to offer some means of backwards compatibility for GH Pages' use.

I'd love to get @mattr-'s thoughts, and @mojombo's thoughts if he has the time.

[parkr]

I was just bumped back to an old issue (#453) in which @mojombo wrote the following:

Jekyll is not Rails. Convention over configuration is not our motto. Simplicity and obviousness are what Jekyll cares about. Out of the box, Jekyll should be brutally obvious in how it functions.

... and ...

Backwards compatibility is also extremely important for GitHub Pages, and I don't want this 1.0 release to turn into a shitshow of "why did my site totally break and do surprising things when all I did was push like normal?"

So I would say this project, if GH Pages wishes to continue using it, should be especially aware of breaking changes and, at all costs, prevent pushing changes that break current functionality unless it is necessary to do so.

I think it would be appropriate to break things in the default behaviour, but there should - as much as possible - be some means of preserving the previous behaviour for GH Pages.

[mattr-]

@parkr you summed up my feelings exactly.

We're also pushing the project faster than it's ever gone before. Jekyll had almost zero activity six months ago. Between then and now, we've released two major versions (one of them being 1.0), closed a ton of issues, merged a fair amount of pull requests, etc. (@parkr did it mostly). I'm actually kinda surprised we didn't break more stuff.

[benbalter]

Full disclosure, my background is from the WordPress community where breaking changes is verboten to say the least, so the idea of causing users pain just to achieve more pristine or computer sciency code is foreign to me. I think there's a lot of parallels Jekyll can draw from WordPress (hacker culture which often creates its own, better design patterns; commitment to simplicity; non-technical userbase; and transition from blog to CMS-ish). Interesting, a talk my buddy @nacin gave today on essentially this very issue:

WordPress has tens of millions of users worldwide and powers over a fifth of new sites. But such a large and diverse user base presents unique challenges, and as it approaches its tenth birthday, the WordPress codebase is showing its age. So why is it so ubiquitous?The answer lies not in its UI, UX, ecosystem, or community, though those are certainly among its strengths. The answer lies instead in a core philosophy that holds the user above all else. This user-centric design starts not with the interface, but rather with the code itself. Developers should approach software development with an unwavering promise they will deal with the nonsense instead of passing it off to the user.

Some WordPress positions might seem draconian and inflexible. Backwards compatibility is almost never broken. The technical requirements appeal to lowest common denominators. But because the project maintainers deal with all the pain — technical debt, difficulties with PHP, working on as many server configurations (and misconfigurations) as possible — its users don’t have to. Thanks to the WordPress project’s portability efforts, you can pick a web host or a PHP configuration at random and WordPress will run on it. Because of this, adoption has soared. The way WordPress operates is not for everyone. But whether your projects are used by 10 users, or 10 million, it may help you to see your code in an entirely new light. (source)

The service/software distinction is valid, and I think things fall pretty squarely there. But even if this was just a software project, we're building software for hoomans, not computer scientists. Using the example that sparked this all, yes, it's technically more proper to represent data as an array, rather than as a comma separated list, but our users don't care. They don't care what version they're running, or if reaching some arbitrary number gave you the confidence to provide them with a sub-optimal user experience.

Obviously there are case-by-case decisions, but in the end, let's build software for users, not for computers or to achieve logical ideals. What attracted me to Jekyll was exactly what @mojombo outlined. Zen-like simplicity. Let's keep things as simple as possible for the user and absorb all the complexity and chaos in the world on their behalf.

[benbalter]

in which @mojombo wrote the following:

C'mon @parkr you're into politics. You know there's no way to know the founders' true intentions for sure. We can't take such ancient texts literally. We have to treat it as a living document...

[parkr]

@benbalter A living document indeed. 😄 But don't date him - that comment is from a month ago :)

[nacin]

Worth noting that we also have a similar service-software split for WordPress, given that WordPress.com runs a hosted version. It is important for us not only to "absorb all the complexity and chaos," as @benbalter wrote, but it is also important for WordPress.com to absorb some of its own chaos. Basically, if GitHub has unique needs when it comes to Jekyll, then that's something that may need to be solved within GitHub, not necessarily by Jekyll. Just some food for thought. Thanks for the shout-out, Ben.

[tombell]

Given there was an announcement that GitHub pages was upgraded to v1.x.x, it would make sense people upgrade their local Jekyll for building/testing their site to the same version and check before you push to GitHub. The change doesn't really apply unless you 'update' your repo and push, in which I would expect someone to have tested their changes locally first.

[benbalter]

Cross linking @parkr's comments from #1236 as they do a great job of articulating the problem:

Imagine that you have a site that you've been editing with prose.io on GitHub Pages. You don't test whether your site compiles on your local machine: it's just easier to make your edits on prose and push to GitHub and let GitHub do the heavy lifting. You're happily using the default, as you have no ul tags and you haven't hit any of the other downsides of MaRuKu. Now imagine that we update Jekyll and GitHub Pages to use RedCarpet. Next time you go to edit your site and push to GitHub, your site doesn't change like it normally did. You go make another edit to double-check that it's indeed not a problem with Prose. You see your updates on GitHub just like normal but your site still isn't building. You finally clone down your repo, install Jekyll and run it yourself. You're now getting some esoteric error about some markup you have that used a MaRuKu feature that RedCarpet doesn't support and doesn't have a replacement for. The error, of course, is super cryptic and you don't recognize that you're using a new markdown parser because you just used the default. Why did it stop working all of a sudden? You go to Google and StackOverflow for answers but you can't find anything. You finally write a ticket on mojombo/jekyll in hopes that someone else knows what's up and a day later you get a response that the default changed from MaRuKu to RedCarpet and that you need to modify your _config.yml file in order to get the old behaviour back. So now you're out a day's worth of work just because the dev team likes RedCarpet more than MaRuKu, and you're frustrated as hell about it.

Now multiply this times a couple thousand and you have yourself the scale of frustration that the change in default will cause. It seems like such a simple change, but the ramifications are enormous.

[tombell]

Jekyll follows semantic versioning at the end of the day, and pre 1.0 doesn't set the public API in stone, updates to 1.0 can and may have breaking changes that is the life of a project following semantic versioning. As far as I am aware GitHub Pages doesn't just use one markdown render it installs multiple gems for all the supported renderers in Jekyll because people can choose a different renderer.

Also did the 'default' renderer change?

[benbalter]

@tombell the question @parkr was addressing in #1236 was not whether semantic version allowed for breaking changes at major milestones, or whether the project was semantically versioned, but whether given our diverse and often non-technical current and future user base, if when facing decisions such as our preference for one markdown render over another, should we do so in a way that knowingly causes significant pain for our users or should we absorb the complexity of software development on their behalf, and seek more transparent solutions.

[nickmccurdy]

From looking at recent Jekyll changelogs (especially since 2.0), it seems to me like Jekyll does not follow semantic versioning. Configuration file defaults often change to values that seem like they could cause back compatibility issues (for example, handling subdirectories in _posts, or the watch flag for jekyll serve). Feel free to correct me if I'm wrong, but I feel like there have been significant breaking changes since 2.0.

Note: I think GitHub handles their end of the problem (with GitHub Pages) just fine. I would prefer GitHub Pages to allow specifying a Jekyll version for each site, though that would likely be very difficult to set up. I think that the current system of sending warnings is reasonable, but what I'm concerned about is Jekyll's own versioning, and not breaking changes on GitHub Pages in particular.

[parkr]

I think we're doing OK. The _posts thing was a fluke and was reverted IIRC.