Internal documentation for working on Jekyll #5011

Merged
merged 12 commits into from Jul 13, 2016

Conversation

Projects
None yet
8 participants
@parkr
Member

parkr commented Jun 14, 2016

One thing I have not done well is write about how to maintain Jekyll. Historically, I have sort of been a "benevolent dictator" in dealing with issues and pull requests, basically making myself a blocker for moving the project forward at all. It's time to change that.

The purpose of this pull request is to empower all maintainers of Jekyll-owned projects to fulfill all duties, such that no one maintainer makes all the decisions or does the lion share of the work. It's a team effort.

Here is the beginning of some documentation for maintainers on @jekyll/core, @jekyll/plugin-core, or any team which grants write access to any repository on https://github.com/jekyll. It talks about handling issues, reviewing pull requests, and merging pull requests. Anyone who is looking for guidance when working on Jekyll should be able to refer to documentation on the subject at any time.

At the moment, I have:

  1. Maintainers: Triaging and issue
  2. Maintainers: Reviewing a pull request
  3. Maintainers: Merging a pull request
  4. Maintainers: Avoiding burnout
  5. Contributors: Becoming a maintainer

There is still more work to be done, but please feel free to add thoughts here.

This idea borrows heavily from the Homebrew documentation, starting with Maintainers: Avoiding Burnout which @MikeMcQuaid shared with me today.

@kytrinyx & @beardofedu, this is my attempt at workflow-focused documentation rather than reference documentation. This is my first attempt, really, at writing this way so any feedback is welcome!

parkr added some commits Jun 15, 2016

Add 'Reviewing a Pull Request' documentation
@MikeMcQuaid Would love your feedback on this one. Writing from scratch
here and I have the tendency to sound pretty formal. Thoughts?
@parkr

This comment has been minimized.

Show comment
Hide comment
@parkr

parkr Jun 16, 2016

Member

Just completed three more docs: "Avoiding Burnout," which is basically a copy of Homebrew's; "Reviewing a Pull Request"; and "Becoming a Maintainer". I would love some feedback!

Member

parkr commented Jun 16, 2016

Just completed three more docs: "Avoiding Burnout," which is basically a copy of Homebrew's; "Reviewing a Pull Request"; and "Becoming a Maintainer". I would love some feedback!

docs/avoiding-burnout.md
+
+**This guide is for maintainers.** These special people have **write access** to one or more of Jekyll's repositories and help merge the contributions of others. You may find what is written here interesting, but it’s definitely not for everyone.
+
+# 1. Use Homebrew

This comment has been minimized.

@kytrinyx

kytrinyx Jun 16, 2016

Probably not relevant to the Jekyll maintainers.

@kytrinyx

kytrinyx Jun 16, 2016

Probably not relevant to the Jekyll maintainers.

This comment has been minimized.

@kytrinyx

kytrinyx Jun 16, 2016

Or rather... s/Homebrew/Jekyll, probably.

@kytrinyx

kytrinyx Jun 16, 2016

Or rather... s/Homebrew/Jekyll, probably.

This comment has been minimized.

@parkr

parkr Jun 16, 2016

Member

Hehe, you see where I got this document from 😉

@parkr

parkr Jun 16, 2016

Member

Hehe, you see where I got this document from 😉

@kytrinyx

This comment has been minimized.

Show comment
Hide comment
@kytrinyx

kytrinyx Jun 16, 2016

This is so good! Are you still working on the triage one?

This is so good! Are you still working on the triage one?

@beardofedu

This comment has been minimized.

Show comment
Hide comment
@beardofedu

beardofedu Jun 16, 2016

I agree with @kytrinyx, this is a really solid addition to the documentation for the project. I'm still poking around in the existing documentation, so if I see anything of concern I will let you know.

❤️

I agree with @kytrinyx, this is a really solid addition to the documentation for the project. I'm still poking around in the existing documentation, so if I see anything of concern I will let you know.

❤️

@MikeMcQuaid

This comment has been minimized.

Show comment
Hide comment
@MikeMcQuaid

MikeMcQuaid Jun 16, 2016

prompty -> promptly

@MikeMcQuaid

This comment has been minimized.

Show comment
Hide comment
@MikeMcQuaid

MikeMcQuaid Jun 16, 2016

Looks good to me. My only thought is that it might be worth noting that replying quickly is ideal but I think sometimes thorough review may take a long time, weeks even, particularly if there's changed required which then need rereviewed. Might be worth noting that.

Looks good to me. My only thought is that it might be worth noting that replying quickly is ideal but I think sometimes thorough review may take a long time, weeks even, particularly if there's changed required which then need rereviewed. Might be worth noting that.

docs/becoming-a-maintainer.md
+
+You want to maintain Jekyll? Use it often. Do weird things with it. Do normal things with it. Does it work? Does it have any weaknesses? Is there a gap in the product that you think should be filled?
+
+## 1. Help Triage Issues

This comment has been minimized.

@MikeMcQuaid

MikeMcQuaid Jun 16, 2016

There's two 1.s

@MikeMcQuaid

MikeMcQuaid Jun 16, 2016

There's two 1.s

This comment has been minimized.

@parkr

parkr Jun 16, 2016

Member

D'oh!

@parkr

parkr Jun 16, 2016

Member

D'oh!

docs/becoming-a-maintainer.md
+
+## 3. Write Code
+
+As a maintainer, you will be reviewing pull requests which update code. You should feel comfortable with the Jekyll codebase enough to confidently review a put forward.

This comment has been minimized.

@MikeMcQuaid

MikeMcQuaid Jun 16, 2016

Not sure what a put forward means?

@MikeMcQuaid

MikeMcQuaid Jun 16, 2016

Not sure what a put forward means?

This comment has been minimized.

@parkr

parkr Jun 16, 2016

Member

Missing the noun. Thanks!

@parkr

parkr Jun 16, 2016

Member

Missing the noun. Thanks!

docs/merging-a-pull-request.md
+
+All pull requests should be subject to code review. Code review is a [foundational value](https://blog.fullstory.com/what-we-learned-from-google-code-reviews-arent-just-for-catching-bugs-b125a13aa292) of good engineering teams. Besides providing validation of correctness, it promotes a sense of community and gives other maintainers understanding of all parts of the code base. In short, code review is crucial to a healthy open source project.
+
+Before merging a pull request **that changes code**, ensure that code is thoroughly reviewed and has received a thorough review from at least two maintainers.

This comment has been minimized.

@MikeMcQuaid

MikeMcQuaid Jun 16, 2016

Have no idea on your process but it might be worth asking all new code has full unit test coverage.

@MikeMcQuaid

MikeMcQuaid Jun 16, 2016

Have no idea on your process but it might be worth asking all new code has full unit test coverage.

@parkr

This comment has been minimized.

Show comment
Hide comment
@parkr

parkr Jun 16, 2016

Member

Are you still working on the triage one?

@kytrinyx Yes! I am coming up a little short with some common things to look for but I'm sure it will evolve. I'll go triage some issues now for more ideas. 😉

Member

parkr commented Jun 16, 2016

Are you still working on the triage one?

@kytrinyx Yes! I am coming up a little short with some common things to look for but I'm sure it will evolve. I'll go triage some issues now for more ideas. 😉

@parkr

This comment has been minimized.

Show comment
Hide comment
@parkr

parkr Jun 17, 2016

Member

replying quickly is ideal but I think sometimes thorough review may take a long time, weeks even, particularly if there's changed required which then need rereviewed

@MikeMcQuaid I should clarify that further, yes. The idea of "reply quickly" is simply referring to an initial "we got your pr, here are our thoughts" reply. The idea of "resolve quickly" is definitely the idea that it should be merged or closed quickly. It's ideal for sure, though if review takes weeks, would the PR be better broken up into multiple PR's instead?

Member

parkr commented Jun 17, 2016

replying quickly is ideal but I think sometimes thorough review may take a long time, weeks even, particularly if there's changed required which then need rereviewed

@MikeMcQuaid I should clarify that further, yes. The idea of "reply quickly" is simply referring to an initial "we got your pr, here are our thoughts" reply. The idea of "resolve quickly" is definitely the idea that it should be merged or closed quickly. It's ideal for sure, though if review takes weeks, would the PR be better broken up into multiple PR's instead?

parkr added some commits Jun 17, 2016

@parkr

This comment has been minimized.

Show comment
Hide comment
@parkr

parkr Jun 17, 2016

Member

@kytrinyx I wrote more in Triaging an Issue. What do you think?

Member

parkr commented Jun 17, 2016

@kytrinyx I wrote more in Triaging an Issue. What do you think?

@MikeMcQuaid

This comment has been minimized.

Show comment
Hide comment
@MikeMcQuaid

MikeMcQuaid Jun 17, 2016

It's ideal for sure, though if review takes weeks, would the PR be better broken up into multiple PR's instead?

I think so but I also think some work is just complicated in terms of wider system effects (even if a just a few lines of code). May be worth clarifying here but I think it's also fine to leave as-is.

It's ideal for sure, though if review takes weeks, would the PR be better broken up into multiple PR's instead?

I think so but I also think some work is just complicated in terms of wider system effects (even if a just a few lines of code). May be worth clarifying here but I think it's also fine to leave as-is.

+If the issue describes a feature request, ask:
+
+1. Is this a setting? [Settings are a crutch](http://ben.balter.com/2016/03/08/optimizing-for-power-users-and-edge-cases/#settings-are-a-crutch) for doing "the right thing". Settings usually point to a bad default or an edge case that could be solved easily with a plugin. Keep the :christmas_tree: of settings as small as possible so as not to reduce the usability of the product. We like the philosophy "decisions not options."
+2. Would at least 80% of users find it useful? If even a quarter of our users won't use it, it's very likely that the request doesn't fit our product's core goal.

This comment has been minimized.

@MikeMcQuaid

MikeMcQuaid Jun 17, 2016

I might be tempted to bump this number just because people tend to estimate in their own favour 😉

@MikeMcQuaid

MikeMcQuaid Jun 17, 2016

I might be tempted to bump this number just because people tend to estimate in their own favour 😉

This comment has been minimized.

@beardofedu

beardofedu Jun 22, 2016

If you do keep the 80% figure, what methodology is used (or preferred) to identify that a percentage of the users would find the feature useful? I agree with @MikeMcQuaid that people tend to skew numbers in their favour, but if there is a set way to identify that 80% metric it makes it a little harder to fudge the numbers.

@beardofedu

beardofedu Jun 22, 2016

If you do keep the 80% figure, what methodology is used (or preferred) to identify that a percentage of the users would find the feature useful? I agree with @MikeMcQuaid that people tend to skew numbers in their favour, but if there is a set way to identify that 80% metric it makes it a little harder to fudge the numbers.

This comment has been minimized.

@parkr

parkr Jun 22, 2016

Member

what methodology is used (or preferred) to identify that a percentage of the users would find the feature useful

Hm... good point. This is all by gut measurement, really. I really stole the idea of the 80% thing from @benbalter. Ben, do you have a way of identifying this?

@parkr

parkr Jun 22, 2016

Member

what methodology is used (or preferred) to identify that a percentage of the users would find the feature useful

Hm... good point. This is all by gut measurement, really. I really stole the idea of the 80% thing from @benbalter. Ben, do you have a way of identifying this?

This comment has been minimized.

@envygeeks

envygeeks Jun 22, 2016

Contributor

Why not have features weighted by votes? 👍 / 👎 vote period a version behind. So if you are on 3.1 and request a new feature, it will be allowed to be voted on until 3.2 at which time it will either be closed as unwanted by the community or with the community extremely interested and land in 3.3.

The problem here is exposure to these feature tickets.

@envygeeks

envygeeks Jun 22, 2016

Contributor

Why not have features weighted by votes? 👍 / 👎 vote period a version behind. So if you are on 3.1 and request a new feature, it will be allowed to be voted on until 3.2 at which time it will either be closed as unwanted by the community or with the community extremely interested and land in 3.3.

The problem here is exposure to these feature tickets.

This comment has been minimized.

@MikeMcQuaid

MikeMcQuaid Jun 22, 2016

I think gut measurement is the best way to do this. Votes skew towards the type of users who vote on GitHub issue trackers which tend to be a different group to those of all those who use Jekyll. You could argue e.g. that all GitHub Pages users are Jekyll users but they might not even know what Jekyll is!

@MikeMcQuaid

MikeMcQuaid Jun 22, 2016

I think gut measurement is the best way to do this. Votes skew towards the type of users who vote on GitHub issue trackers which tend to be a different group to those of all those who use Jekyll. You could argue e.g. that all GitHub Pages users are Jekyll users but they might not even know what Jekyll is!

This comment has been minimized.

@kytrinyx

kytrinyx Jun 22, 2016

You could argue e.g. that all GitHub Pages users are Jekyll users but they might not even know what Jekyll is!

Does GitHub Pages use Jekyll for something behind the scenes?

@kytrinyx

kytrinyx Jun 22, 2016

You could argue e.g. that all GitHub Pages users are Jekyll users but they might not even know what Jekyll is!

Does GitHub Pages use Jekyll for something behind the scenes?

This comment has been minimized.

@MikeMcQuaid

MikeMcQuaid Jun 22, 2016

@kytrinyx Yeh, basically every GitHub Pages project is implicitly a Jekyll project.

@MikeMcQuaid

MikeMcQuaid Jun 22, 2016

@kytrinyx Yeh, basically every GitHub Pages project is implicitly a Jekyll project.

This comment has been minimized.

@benbalter

benbalter Jun 22, 2016

Contributor

Agree that direct voting will skew results towards power users. Logically, the new user experience would suffer because a potential user doesn't know to / want to vote until they're a user, and even then, have so little invested, that they're all but guaranteed to be underrepresented. See http://ben.balter.com/2016/03/08/optimizing-for-power-users-and-edge-cases/ for some additional context. Specifically:

The silent majority of users

Think about the type of user that's going to take the time to open an issue: they're either going to be power users that have already invested hours in your app and have grown the need for additional functionality that extends the core use case, or new and potential users with unique circumstances that you didn't originally intended to accommodate. In either case, opening an issue takes time, commitment to a cause, and specialized knowledge, and thus, through pure self selection, most issues are going to be opened by the extreme edges of your user base, not by the core of your users (or potential users, for that matter).

Instead of optimizing for the most vocal, feature requests should be evaluated through the lens of "what would benefit 80% of users?" — the silent majority, to borrow a phrase from politics. I suspect, for most projects, the exact opposite is sure, that 80% of feature requests come from just 20% of users (or as few as 1%). Combined with the fact that these requests often represent the "fun" problems, this 80/20 rule means that in many cases, absent your intervention, the majority of features implemented, will provide little or no benefit to the majority of users.

We can however, (in a subsequent pass) do a better job of identifying our core users and more narrowly defining our core use case, to help move things a bit from "gut" to "this doesn't fit our core use case" a bit more concretely and objectively.

@benbalter

benbalter Jun 22, 2016

Contributor

Agree that direct voting will skew results towards power users. Logically, the new user experience would suffer because a potential user doesn't know to / want to vote until they're a user, and even then, have so little invested, that they're all but guaranteed to be underrepresented. See http://ben.balter.com/2016/03/08/optimizing-for-power-users-and-edge-cases/ for some additional context. Specifically:

The silent majority of users

Think about the type of user that's going to take the time to open an issue: they're either going to be power users that have already invested hours in your app and have grown the need for additional functionality that extends the core use case, or new and potential users with unique circumstances that you didn't originally intended to accommodate. In either case, opening an issue takes time, commitment to a cause, and specialized knowledge, and thus, through pure self selection, most issues are going to be opened by the extreme edges of your user base, not by the core of your users (or potential users, for that matter).

Instead of optimizing for the most vocal, feature requests should be evaluated through the lens of "what would benefit 80% of users?" — the silent majority, to borrow a phrase from politics. I suspect, for most projects, the exact opposite is sure, that 80% of feature requests come from just 20% of users (or as few as 1%). Combined with the fact that these requests often represent the "fun" problems, this 80/20 rule means that in many cases, absent your intervention, the majority of features implemented, will provide little or no benefit to the majority of users.

We can however, (in a subsequent pass) do a better job of identifying our core users and more narrowly defining our core use case, to help move things a bit from "gut" to "this doesn't fit our core use case" a bit more concretely and objectively.

docs/triaging-an-issue.md
+
+### Supported Platform
+
+Is the author using a supported platform? We support the latest versions of macOS, most common distributions of Linux. If the author is using Windows, you can use the `@jekyll/windows` team to help you investigate, but ultimately it is not a supported platform.

This comment has been minimized.

@MikeMcQuaid

MikeMcQuaid Jun 17, 2016

On Homebrew we tend to close these issues out but still encourage discussion in the closed issue. Not saying you should do that but just thought you might be interested.

@MikeMcQuaid

MikeMcQuaid Jun 17, 2016

On Homebrew we tend to close these issues out but still encourage discussion in the closed issue. Not saying you should do that but just thought you might be interested.

This comment has been minimized.

@parkr

parkr Jun 21, 2016

Member

Historically, I have received a lot of anger from users on Windows for not supporting Windows and for closing their issue (making it "undiscoverable"?). What would be a good way to respond to this, you think? Just being firm and assuring the user that closing it doesn't make it "undiscoverable"?

@parkr

parkr Jun 21, 2016

Member

Historically, I have received a lot of anger from users on Windows for not supporting Windows and for closing their issue (making it "undiscoverable"?). What would be a good way to respond to this, you think? Just being firm and assuring the user that closing it doesn't make it "undiscoverable"?

docs/triaging-an-issue.md
+
+### What they wanted vs. what they got
+
+An issue without a clear explanation of what the user got and what they were expecting to get is not an issue we can accurately respond to. If the user doesn't provide this information, please ask for clarification. This information helps us build test cases such that we do not break the behaviour again in the future.

This comment has been minimized.

@kytrinyx

kytrinyx Jun 20, 2016

Is there a tag that marks something as waiting for more information from the original author?

Is there a timeframe for closing issues that haven't gotten more information?

@kytrinyx

kytrinyx Jun 20, 2016

Is there a tag that marks something as waiting for more information from the original author?

Is there a timeframe for closing issues that haven't gotten more information?

This comment has been minimized.

@parkr

parkr Jun 21, 2016

Member

There is a tag, yes! We keep a pending-feedback label that @jekyllbot unassigns once the issue/PR author leaves a comment.

I mention this somewhere, but yes, we give one month until the issue is marked as stale, then another month in which the user can respond before it is automatically closed. Also a @jekyllbot thing.

@parkr

parkr Jun 21, 2016

Member

There is a tag, yes! We keep a pending-feedback label that @jekyllbot unassigns once the issue/PR author leaves a comment.

I mention this somewhere, but yes, we give one month until the issue is marked as stale, then another month in which the user can respond before it is automatically closed. Also a @jekyllbot thing.

This comment has been minimized.

@kytrinyx

kytrinyx Jun 22, 2016

Nice. I can't decide if I think it would be helpful to say something about it here. Probably not, since it's an automatic thing.

@kytrinyx

kytrinyx Jun 22, 2016

Nice. I can't decide if I think it would be helpful to say something about it here. Probably not, since it's an automatic thing.

@kytrinyx

This comment has been minimized.

Show comment
Hide comment
@kytrinyx

kytrinyx Jun 20, 2016

The triage docs look really good.

You might also get some good ideas from Steve Klabnik's blog post "Open Source Gardening" http://words.steveklabnik.com/how-to-be-an-open-source-gardener

The triage docs look really good.

You might also get some good ideas from Steve Klabnik's blog post "Open Source Gardening" http://words.steveklabnik.com/how-to-be-an-open-source-gardener

docs/avoiding-burnout.md
+
+# 1. Use Jekyll
+
+Maintainers of Homebrew should be using it regularly. This is partly because you won't be a good maintainer unless you can put yourself in the shoes of our users but also because you may decide to stop using Jekyll and at that point you should also decide not to be a maintainer and find other things to work on.

This comment has been minimized.

@victorsolis

victorsolis Jun 21, 2016

It still says Homebrew here.

@victorsolis

victorsolis Jun 21, 2016

It still says Homebrew here.

This comment has been minimized.

@parkr

parkr Jun 21, 2016

Member

Good catch. 😄

@parkr

parkr Jun 21, 2016

Member

Good catch. 😄

parkr added some commits Jun 21, 2016

docs/triaging-an-issue.md
+
+### Supported Platform
+
+Is the author using a supported platform? We support the latest versions of macOS, and most common distributions of Linux.

This comment has been minimized.

@parkr

parkr Jun 21, 2016

Member

@envygeeks what distributions of Linux do we want to support?

@parkr

parkr Jun 21, 2016

Member

@envygeeks what distributions of Linux do we want to support?

This comment has been minimized.

@envygeeks

envygeeks Jun 22, 2016

Contributor

Ubuntu, Debian, CentOS, Fedora, Arch. IMO.

@envygeeks

envygeeks Jun 22, 2016

Contributor

Ubuntu, Debian, CentOS, Fedora, Arch. IMO.

docs/triaging-an-issue.md
+
+**This guide is for maintainers.** These special people have **write access** to one or more of Jekyll's repositories and help merge the contributions of others. You may find what is written here interesting, but it’s definitely not for everyone.
+
+Here are some key things to remember when evaluating an issue.

This comment has been minimized.

@beardofedu

beardofedu Jun 22, 2016

This might be beyond the scope of your typical use case, but perhaps include something along the lines of:

Before evaluating an issue, it is important to identify if it is a feature request or a bug. For the Jekyll project the following definitions are used to identify a feature or a bug:
Feature - A feature is defined as a request that adds functionality to Jekyll outside of its current capabilities.
Bug - A bug is defined as an issue that identifies an error that a user (or users) encounter when using current Jekyll functionalities.

@beardofedu

beardofedu Jun 22, 2016

This might be beyond the scope of your typical use case, but perhaps include something along the lines of:

Before evaluating an issue, it is important to identify if it is a feature request or a bug. For the Jekyll project the following definitions are used to identify a feature or a bug:
Feature - A feature is defined as a request that adds functionality to Jekyll outside of its current capabilities.
Bug - A bug is defined as an issue that identifies an error that a user (or users) encounter when using current Jekyll functionalities.

This comment has been minimized.

@envygeeks

envygeeks Jun 22, 2016

Contributor

@beardofedu the ticket template takes care of that in asking the user within the first few lines to let us know if this is a feature request or otherwise. Sometimes it's hard for a user to tell if something is a feature or a bug, a fundamental concept left out is a bug to some but a feature to others. Solving an edge might be a feature to us but a bug to them.

@envygeeks

envygeeks Jun 22, 2016

Contributor

@beardofedu the ticket template takes care of that in asking the user within the first few lines to let us know if this is a feature request or otherwise. Sometimes it's hard for a user to tell if something is a feature or a bug, a fundamental concept left out is a bug to some but a feature to others. Solving an edge might be a feature to us but a bug to them.

parkr added some commits Jun 22, 2016

@parkr

This comment has been minimized.

Show comment
Hide comment
@parkr

parkr Jul 13, 2016

Member

@jekyllbot: merge +dev

Member

parkr commented Jul 13, 2016

@jekyllbot: merge +dev

@jekyllbot jekyllbot merged commit f55cf34 into master Jul 13, 2016

1 check was pending

continuous-integration/travis-ci/pr The Travis CI build is in progress
Details

@jekyllbot jekyllbot deleted the maintainer-docs branch Jul 13, 2016

jekyllbot added a commit that referenced this pull request Jul 13, 2016

@parkr

This comment has been minimized.

Show comment
Hide comment
@parkr

parkr Jul 13, 2016

Member

🚢 'd it. Thanks to everyone who helped review this! Huge for us to be able to point to written resources on how we work.

Member

parkr commented Jul 13, 2016

🚢 'd it. Thanks to everyone who helped review this! Huge for us to be able to point to written resources on how we work.

stevecheckoway added a commit to stevecheckoway/jekyll that referenced this pull request Jul 24, 2016

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment