Help open-source maintainers stay sane

[balupton]

Currently, as an open-source maintainer of popular projects, we are greeted with screens like this each day:

Which just makes us want to explode due to the instant feelings of anxiety, guilt, and shame such screens promote. Which then left untreated, results in burnout.

Now it's not just me suffering from this, but it seems most open-source maintainers I've talked to about it suffer from it to one degree or another. @fat gave a talk about this here, watching it at the time, I was like, I suffered this with History.js, but not with DocPad, so in your face! Turns out, in hindsight, with the current resources available to us, it's just a matter of time before those feelings catch up with us... and me.

It would be nice if GitHub catered its interface to supporting maintainers of popular open-source projects a bit better, especially as the GitHub community is often the only resources and support that independent open-sourcers like myself have.

However, in the meantime, let us brainstorm and share ideas for what's worked for us, and what could work for others. Let us share the techniques, resources, and support we've used before and those we use today to overcome these feelings of anxiety, guilt, and shame our roles as open-source maintainers are up against.

---

Backstory:

• Free culture is a lot harder to maintain that it's made out to be
• Is there anyone else that would like a meetup about dealing with life as a open-source maintainer?
• Seems the connotation of "tearing hair out" is becoming increasingly popular in the tech scene...
• Anyone know a way to solve the "signal vs noise" ratio problem with github issues?
• A person is only as good as what their tools enable them to become

---

Problems of the _incomplete_ puzzle so far:

• People post issues for everything
  • A well done CONTRIBUTING.md helps with this, but perhaps we need a check to make sure they've actually read it
• Maintenance often requires more time than the authors have
  • Partially solved by the pull request contributor technique, but that doesn't guarantee contributors will stick around or make the evolution to a maintainer
• Projects are rarely maintained by their author, as it already satisfies the authors requirements, then forks happen that create fragmentation
  • Solved by the pull request contributor technique
• Creation is an intrinsic motivator, maintenance usually requires extrinsic motivation
  • Solved by the pull request contributor technique
• GitHub notifications are not easy to reply to, requires a lot of clicking, switching windows, and contexts
  • Perhaps solved by the feedly suggestion
• GitHub notifications are often not relevant to you, they may already be closed, or attended to by a team member, and may not require your response
  • You can unwatch a project, but that means you miss out on everything
  • Perhaps reduced by the feedly suggestion
• There is no easy way for maintainers to cipher through the noise to find the signal
  • Perhaps assisted by view counts on issues
  • Perhaps assisted by outside-github techniques such as project managers and management skills
  • More suggestions needed
• Developers get lonely working by themselves with no face to face interaction
  • Interconnect aims to solve this, not yet developed yet, could do with your help
  • Weekly Google Hangouts On Air helps with this, but not that much as it provides no camaraderie, just more meetings
  • Sqwiggle would be good, but doesn't compute with open-source teams, hence why Interconnect was created

---

Solutions to the _incomplete_ puzzle so far:

• Reduce the need for support requests by having great documentation
  • +1 from @balupton
• Reduce the barrier of entry of maintenance and contributions by building your monolithic context-sensitive app as several independent modular components that are trivial for people to get their head around completely
  • +1 from @balupton
  • Resources:
    • Many things by @substack
• Reduce the need of bug reports by having great comprehensive tests
  • +1 from @balupton
• Reduce the need of you being the bottleneck by trusting others and delegating things others can do to those inspired to do them
  • +1 from @balupton, @felixge
  • Resources:
    • The pull request hack by @felixge
• Reduce the distance between others expectations of you and your capabilities by being vulnerable against the super-hero image open-source maintainers are given
  • +1 from @balupton
  • Resources:
    • Listening to shame by Brene Brown
• Get financial sponsors for your open-source projects to pay your bills
  • +1 from @balupton
  • Notes:
    • @balupton: takes time to do, and when a sponsor pulls out, it's emergency time
• Raise a few million dollars and hire some helpful friends
  • +1 from @isaacs
  • Notes:
    • @balupton: very difficult to do, especially in investment poor communities
• If/Once you've identified a better workflow than what GitHub provides, leverage the GitHub APIs to build your own tools
  • +1 from @eddieajau
  • Instances:
    • http://issues.joomla.org
    • http://waffle.io
    • http://huboard.com
• Improve GitHub to make it less stressing
  • @balupton: be more like feedly
• Address people, not projects
  • Resources:
    • Programming is terrible—Lessons learned from a life wasted.
    • Getting Things Done
    • Choose the life you want, the mindful way to happiness

[eddieajau]

rather than having GitHub cater more to those of the closed-sourced teams that pay GitHub's bills that already have the support and resources available to deal with such issues.

Just for your information I use github.com and Github Enterprise and the problems we face at work on GE are exactly the same as you do on the outside. We don't have a red phone to Github command for ordering new features. Updates to Github Enterprise are actually behind github.com so don't complain to hard - the Open Source community gets to try all the new toys first :p

That said I agree with the premise of your comment but I don't see specific ideas to fix it (in this post). Github, I thought, was looking at ways of having a better dialogue with Open Source communities so you might want to try to contact them (via the support form) to get your name on that list. I don't know when that's going to happen, only that they were looking into it.

[balupton]

Just for your information I use github.com and Github Enterprise and the problems we face at work on GE are exactly the same as you do on the outside. We don't have a red phone to Github command for ordering new features. Updates to Github Enterprise are actually behind github.com so don't complain to hard - the Open Source community gets to try all the new toys first

Interesting. That's a good point to raise, as it is definitely the case that GitHub provides us the same stuff, and even the OS community first.

Perhaps a better way of putting it, is that of independently backed open-sourcers versus company backed open-sourcers:

With independently backed open-sourcers, they generally only have the support and resources of themselves.

With company backed open-sourcers (like that of Google, Mozilla, etc), they generally have at least access to management from project managers and camaraderie from daily co-workers within their company to help keep them sane, then if their lucky access to team resources as well like other personal they can utilise or bounce ideas off within their company, and if their really lucky, counselling, guidance, and personal development resources as well.

I've updated the original post to reflect this.

That said I agree with the premise of your comment but I don't see specific ideas to fix it (in this post).

Agreed, I've added a listing to the bottom of the initial post to serve as the cumulation of the pieces of the puzzle here to solve — or at least what we as a community can do, without this being addressed in the GitHub product specifically.

[eddieajau]

those company driven open-sourcers have the benefit of the support and resources that their own company provides

They do, and that does help sometimes, but it's not a silver bullet and project priorities more often than not mean you have to make the best of what you have.

I think the key here is to get more collaboration on tools using the Github API rather than relying on Github to build things into the Github UI. This is exactly what we've done here http://issues.joomla.org (and it's still early days).

[balupton]

I think the key here is to get more collaboration on tools using the Github API rather than relying on Github to build things into the Github UI. This is exactly what we've done here http://issues.joomla.org (and it's still early days).

Cool added to the list.

[balupton]

For what GitHub can do, perhaps having notifications stream in the way Feedly works would be great.

For instance, with GitHub notifications, I instantly get stressed out seeing the vomit of every single notification I have to attend to at once, and that stress amplifies itself dependening on how many notifications there are. It is a place for me to get interrupted by everyones demeands, and to feel shame and guilt about how many things I have to attend to, and anxiety about how long clicking each one, reading each one, replying to reach one is actually going to take, knowing there are soo many more left. All this makes the process of working through notifications gut wrenching.

However, with Feedly, I don't get any of those negative feelings, even with 1000+ things in my newsfeed. I think this is precisely due to the inline reading abilities of Feedly, the auto-mark-as-read as I read through them by scrolling, and only showing you one page of items at a time, rather than every single item at once. It actually makes the working through news articles fun.

It would be really nifty to see this same technique applied to GitHub notifications, as it would certainly remove the major cause of my negative feelings towards GitHub as such an initiative streamlines the entire process of reading through, replying, and checking of the items as I go through them.

[yoshuawuyts]

+1 for better notifications, something that's sorely needed.

[dominictarr]

I think the biggest problem with github notifications is that clicking on one marks it as read,
so If you don't respond to it, it's now off the radar - which makes the other person think you are ignoring them. Otherwise you have open tabs lurking around, which is volatile.
Normally, when I look at github notifications some times I have hours to spare, some times only minutes. If a issue looks like it will easy to deal with, I shoot off a response - at least then it's in their court.

What would be very handy is a way to put something back into the notification queue, but with an estimate about how long it will take to deal with. If it looks like an hour to debug that, or 20 minutes to think something through, it would at least be good be able to respond to the poster.

Also, one of the problems with the internet is that nobody knows how much stuff you actually have to deal with. It would make a big difference if other people could see that you had a big queue of incoming stuff. @felixge speculated about this : is there a pull request hack for email?

Sometimes, when I need @substack to merge something I go through browserify issues and respond to things, I know substack pretty well, so I can often tell them what he'd say anyway.
Usually it's stuff like "we will not merge this feature, make it a module" or "can you describe your usecase". Quite a lot of OS maint. is just helping people to post better issues...

[balupton]

@mitsuhiko did you end up coming up with a solution to http://lucumr.pocoo.org/2013/11/28/emotional-programming/ ?

[humidair1999]

RE: "Free culture is a lot harder to maintain that it's made out to be"

I can't stress enough how frequently I see the "working alone is lonely"/"days start becoming the same" bullet points come up in my day-to-day. These sentiments aren't only shared by open-source contributors, either; I find that programmers and developers in general tend to have more frequent gripes about these things than most other career paths.

I think it's absolutely imperative that we leverage the technology available to us to create communities that allow us to communicate and feel connected with one another.

We've seen experiments along the lines of "collaborative coding" and devs video-casting their workdays, but I think we can go further. I've toyed with the idea of a network based on the concept of "e-co-working;" that is, video-based chatrooms that you'd participate in on a day-to-day basis, in which you'd be paired with other developers working with/on loosely the same technology/projects, the idea being that even if you're located remotely from major hotspots of developer activity in your real life, you'd get some sort of face-to-face connection on a daily basis with other devs that could potentially contribute to your projects, help you with questions and answers, and, above all else, provide some much-needed human interaction.

Aside from that, I took particular notice of two other bullet points:

• Reduce the need for support requests by having great documentation

I've seen a massive decrease in the quality of documentation available in relation to new npm modules over the past year or so; I've come across many modules recently with documentation so sparse that my only recourse was to post an Issue (and often, submit a PR with additional documentation that I'd written myself). I strongly dislike doing this, as conceptually, an "Issue" should be an actual issue with the code itself, not questions about how to use it.

Clearly, the onus lies on the maintainer(s) to provide reasonable documentation up-front; even more so than solving issues related to reducing the bottleneck of developing and resolving Issues, if no one can figure out how to use your goddamn lib because the documentation is so bad, you can't reasonably expect more people to be willing (or able, really) to contribute.

• Reduce the need of you being the bottleneck by trusting others and delegating things others can do to those inspired to do them

I found the "pull request hack" quite brilliant, actually. I've definitely noticed (and contributed to) an unfortunate rise in using Issues as a Q&A section, rather than actual issues with the code. By lowering the barrier to contribution by treating PRs more as "requests to contribute," I could absolutely see heightened contribution, reduced time to turnaround, and increased sense of responsibility for maintainers. I'd love to see a more established process for treating PRs differently in this way, though I don't have any brilliant ideas on how that'd be accomplished.

Thanks for putting this together, @balupton.

[balupton]

@jkymarsh

I think it's absolutely imperative that we leverage the technology available to us to create communities that allow us to communicate and feel connected with one another.

We've seen experiments along the lines of "collaborative coding" and devs video-casting their workdays, but I think we can go further. I've toyed with the idea of a network based on the concept of "e-co-working;" that is, video-based chatrooms that you'd participate in on a day-to-day basis, in which you'd be paired with other developers working with/on loosely the same technology/projects, the idea being that even if you're located remotely from major hotspots of developer activity in your real life, you'd get some sort of face-to-face connection on a daily basis with other devs that could potentially contribute to your projects, help you with questions and answers, and, above all else, provide some much-needed human interaction.

I completely agree. I've actually come to the same conclusions about this as you have. I've set out to create bevry/interconnect to solve this, would be cool to get your feedback on Interconnect over on its repos etc :)

[indexzero]

Thank you for this. Issues and pull-requests do take their fair share of mental wear and tear, but the problem lies not just in the tools. As I've said before "if you're going to do Open Source you should be ready to give up your Friday nights to work on it if you want it to succeed." It is a labor of love, and with any love there is turmoil when you feel it is not respected or cared for properly.

First we need to examine the culture of "open source consumerism" which is how the developers who are using libraries use (or frankly, abuse) Github issues. It is also (frankly) our fault as maintainers. How many times have you said?

"Have a problem? Open an issue on Github!"

This is great. It's simple, repeatable and often is the right thing to do. Many times, we should be saying:

"Have a problem? Read the documentation and FAQ because 100 people have asked that question before!"

There of course are always more problem to solve, but there are three domains that could probably solve 80% of the problem.

1. Identifying audience size of issues and changes.

We are now getting Traffic Analytics from Github, but this is specific to the repository. What if we could get analytics on issues to see how many times they have been viewed? Tell me who is lurking on my project so I can better identify what is important.

2. Help others from previous discussions

What is an FAQ? It is one more thing for us to maintain. What if you could "pin" issues somewhere? I've often used an "FAQ" tag for issues on a couple of popular modules to help people find them, but making that a better part of the information browsing experience for end users would make finding those common "gotchas" much easier. Github has done a good job with pull-requests and CONTRIBUTING.md files, another possibility is something similar for FAQ.md

3. Identify what you're really behind on

The concept of "aging" issues and pull-requests is immensely appealing. The reason that the screen from @balupton is so overwhelming is that there is no organization beyond what is newest. What about:

• What is oldest?
• What is oldest but has the most recent comment?
• What has the most references in other issues?
• ... and so on.

Each one of these is views on the open issues tell part of the story, and together they could help us as maintainers actually figure out what to do next and why.

[dominictarr]

@indexzero great points. Okay I'm about to go all scifi here, but maybe the problem is that we even expect each project to have a maintainer. What if there was a way to just have a "river of forks" that flowed in the same direction... Because currently the reward for having a cool idea is you get lots of email. But you also get situations where you wrote a thing for fun... that other people use in production. That is really kinda a weird situation that could never happen in another field.

I think one thing here is that we can't rely on github to solve our problems, because they are already good at being github, and that is a slightly different thing. Building github is more similar to using github in a team enviroment - but using github as opensource is a very different kind of team. A swarm, not a team.

[balupton]

@dominictarr

Okay I'm about to go all scifi here, but maybe the problem is that we even expect each project to have a maintainer. What if there was a way to just have a "river of forks" that flowed in the same direction... Because currently the reward for having a cool idea is you get lots of email. But you also get situations where you wrote a thing for fun... that other people use in production. That is really kinda a weird situation that could never happen in another field.

Agreed. Very quickly a project you wrote for fun, becomes a project you maintain for others instead of you. The pull request hack does solve this somewhat. However, it doesn't help with dealing with the signal vs noise ratio of notifications. For instance, if someone else is maintaing a repo for me, and I'm watching that repo, even if they've taken care of the issues, I still get notifications.

[balupton]

@indexzero

Great points too.

What is an FAQ? It is one more thing for us to maintain. What if you could "pin" issues somewhere? I've often used an "FAQ" tag for issues on a couple of popular modules to help people find them, but making that a better part of the information browsing experience for end users would make finding those common "gotchas" much easier. Github has done a good job with pull-requests and CONTRIBUTING.md files, another possibility is something similar for FAQ.md

This is quite relevant, maintaining documentation is a job in itself. That said, documentation like a FAQ.md has a clear advantage of conciseness and clarity over GitHub Issues. We've found this with DocPad, we use to link to GitHub issues directly for certain "more information" sections, but then people would be like "uh... how is this relevant? It's just a bunch of people arguing about something". In which case it prompts us to do a FAQ entry, with more clarity and relevance.

First we need to examine the culture of "open source consumerism" which is how the developers who are using libraries use (or frankly, abuse) Github issues. It is also (frankly) our fault as maintainers

This is fascinating, and a great read that rings true for me too. I find the trick to staying motivated with having the dog (as used in your metaphor) is to keep the intrinsic or extrinsic motivation going. Luckily, sometimes you get sponsorship or consulting and it provides extrinsic motivation to maintain your dog. However, if your sponsorship or consulting runs out, or you don't have any, then it is all up to intrinisic motivation, which can quickly be destroyed if the project is no longer fun for you anymore, or you want to work on other things, or whatever else.

[balupton]

Considering the feedback here. I've updated the original post to include a listing of the problems we've identified with GitHub, and the problems we've identified with open-source maintenance in general.

[balupton]

I wonder what @DamonOehlman's feedback on this is, as to avoid the open-source consumerism issue, he actually hosts new projects and moved a lot of existing projects over to BitBucket now instead of GitHub, to keep their popularity crippled to avoid hitting the maintenance issues that popular projects have: DamonOehlman/damonoehlman.github.io#22

[DamonOehlman]

I'm not sure that the experiment of moving stuff over to bitbucket has quite worked the way I wanted, as there is seems that there is just no ability for a "project meritocracy" to play out there at this stage. So while it does address the direct problem you are talking about here, it's not really a solution.

I think there has been a lot of good ideas outlined from what github could do to make things better for maintainers in terms of tooling (thanks for collating the suggestions @balupton), and also ideas or examples of external tools which make life easier too.

While I hadn't seen @felixge's write up on the PR hack, I've seen it put into good practice like @rvagg with excellent results. I also like @dominictarr's illustration of a "river of forks" so maybe it's time that we moved to thinking of ourselves as "project initiators" and ignore the "Project Owner" label that github gives us when we start a project.

It definitely needs to be ok to start something, and also ok to move on. I'm tired of living in fear that one day I'll write something that will be used as much as you lot and not get as much time to spend with my wife and kids :)

[dominictarr]

Realistically, this is gonna have to be something we do for our selves - or someone makes another startup for it - github already has their hands full with a nice fat cash cow - maybe this isn't even just about opensource, in a way, this is maybe just a new kind of todo list - a todo list that other people can add to, yet that is optimized for low stress.
(in this it reminds me of https://en.wikipedia.org/wiki/Getting_Things_Done)

[balupton]

Github sent me an email saying they're going to look into it, but alas I do still agree we should figure out what we can do by ourselves.

Things like GTD are definitely a component to this, I'll add it and things like it to the suggestions above, thanks for raising that! Updated, added.

[patcon]

Hey @balupton, this could use a better title that's more descriptive of the ask :)

[patcon]

Also, in regards to solving the problem without github:

https://www.zenhub.io/
#128

[ahdinosaur]

hi, i love everything about this issue, thanks everyone for sharing their mind. :)

i don't have a good answer to the interface / workflow / development issues, only some related background material on the "organism" of open source developers, in the hopes that maybe it is useful to someone here.

• i think collaborative "swarms" are well-modeled as nested holons
  • a really great academic paper i found on holonic systems is here [PDF].
• an interesting open source project management tool was BetterMeans
• an awesome collaborative decision-making tool is Loomio, i highly recommend it for any developer networks that need to make decisions
• i think funding / gifting / sharing money among open source developers is a good idea

@derekrazo

[balupton]

So a few major things have happened since I posted this that have helped tremendously well:

• Do daily Google Hangouts on Air with your community
  • This has helped tremendously well, it's great to actually offload everything you've been thinking about into the community for easier digestion and able to sort it out. I guess it is like having a thousand bricks on your shoulders, but as soon as you start sharing those concerns with the community, now everyone can carry a very manageable brick, and they are happy to do so
• Stop caring about GitHub Issues and Tweets
  • This has stemmed from the suggestions above about open-source consumerism, I use to feel a need to reply to every tweet, and issue, and fix every bug that I've ever written in any of my projects, as otherwise people will have bad experiences, and that is my fault. In reality, through an experience not by choice, I started to stress out too much, and took 2-4 weeks completely away from the community while I got my head into gear, with a huge fear the community would collapse. It didn't, people got by. Haters moved on. Lovers stayed. Stress reduced.
• Create a meta github repository for your organisation where you can talk about concens relating to big picture stuff
  • I've created http://github.com/bevry/meta and that's helped a lot with figuring out the difficult issues and just in general how to move things forward on a more meta level. Often at times, it isn't the tasks that needs fixing, it's the actual system you use to go about things. The meta repository has helped with that, as it is all about fixing the problems with the system and improving the system to take it to the next level.

[Mithgol]

Closed issues should really be displayed differently in the list of notifications. It helps to prioritize.

[jakesylvestre]

Zenhub anyone?

[Mithgol]

I've noticed that closed issues are now displayed differently in the list of notifications.

(Probably for a month or two already.)