Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Already on GitHub? Sign in to your account

Investigate switching away from GitHub #5205

Closed
nzakas opened this Issue Feb 10, 2016 · 170 comments

Comments

Projects
None yet
Owner

nzakas commented Feb 10, 2016

As ESLint has continued to grow, we've started to outgrow the GitHub ecosystem. Team members spend hours each day triaging issues, many of which have incomplete information or are otherwise unclear. As such, we spend a lot of time just asking for more information ("what do you mean?", "what version are you using?", etc.). This has become increasingly frustrating for everyone on the team and ultimately takes time away from being able to contribute code to the project.

Additionally, it's nearly impossible to keep up with what are the most important issues and whether or not people are following up on issues. In short, everything that Dear GitHub mentioned is severely hurting the team now. As ESLint's popularity continues to grow and there are more and more issues filed, this is quickly becoming a scalability problem.

The team has discussed investigating alternatives to GitHub to see if we can find a home that is better suited to open source projects with our level of scale. We strongly feel that the code and issue tracker need to live in the same location to make it easier to manage and give people one location to visit for all of their ESLint-related needs (so simply moving to a different issue tracker and keeping the code on GitHub is not an alternative).

Requirements

  • Must host the repo along with the related tools
  • Must be able to run automated tests on pull requests
  • Must allow contributions from anyone
  • Must have a way to setup issue templates prescribing what fields are required
  • Must have ways to organize issues other than labeling (milestones, priorities, etc.)
  • Must import existing GitHub issues and pull requests

Alternatives to Investigate

  1. GitLab
  2. Phabricator

Concerns

  1. Will people find us in another location?
  2. How high is the barrier to entry for new contributors if we're not on GitHub?
  3. What about the Gitter chatroom? What happens to that? Do we need to find another chat location?
  4. Others?
Owner

ilyavolodin commented Feb 10, 2016

Just to add a few more concerns:

  • Integration with other tools. We are currently use Travis, AppVeyor (both fall under need to build after each issue), but we also integrate with Coveralls (not really a deal breaker, but nice to have).
  • Even if people do find us in another location, would they be willing to create an account on the new service just to contribute to one project, or create an issue?

Maybe it's worth checking with other people who maintain large projects on GitHub and signed "dear GitHub" letter to see what their plan is? Coordinated move to the same alternative service would have higher chance of success.

Owner

ilyavolodin commented Feb 10, 2016

One other (pretty major) thing that I forgot is the ability to easily host/deploy site with support for markdown, as well as everything connected to that (plugins for markdown, custom domain name support, etc). I guess we could always go the route of separate website hosting, but it would mean more involved releases, and might be harder to maintain.

Owner

nzakas commented Feb 10, 2016

We could also keep the website hosting on GitHub if that's a big concern.

I'm not opposed to doing separate hosting, either. My personal blog is a Jekyll site that I update automatically. It would be pretty easy to setup the same thing for eslint.org.

@nzakas you can just move issues to Phabricator as babel did, keeping all popular projects in one place is really cool.

As for organizing issues - can't you use ZenHub or alternatives so that repo would stay on Github? (doesn't solve the issue template though, but might be a good start and it's free for OSS)

Personally, I find it much easier to track all popular open-source projects in one place (that's pretty much the only reason I moved from Bitbucket to Github couple of years ago) and ability to participate in any of them through one UI, cross-links between projects etc.

Projects that move elsewhere raise the barrier (again, just a personal feeling / opinion here) to something like what projects on Google Code felt before its recent discontinuation or SourceForge feels like. Note - I'm not talking about UI, as other platforms can surely have features which are more convenient for maintainers, but rather talking specifically about the existing ecosystem and low barrier for participating in new projects within it as for newcomers as well for regular OSS contributors.

Contributor

captbaritone commented Feb 11, 2016

I would like to echo @RReverser's concern about keeping the barrier to participation low. Participation comes not just in the form of commits and filing issues, but also staying abreast of what those issues are, and how they are being handled. I have an app on my phone that allows me to follow GitHub repos. I see all the PRs, all the issues, and all the responses. I see new commits and tagged releases. I'm sure there are many other people who are passively involved in the project via this or similar systems (including the builtin GitHub email notifications). The barrier to entry can be as low as clicking the "Watch" button on the GitHub page.

In another ecosystem, I'm guessing my best bet would email email alerts, and I have no interest in flooding my inbox with that number of emails.

Is it possible that we could take some work of the maintainers' plates another way? Maybe we could have some lower-lever volunteers who can handle the more mundane tasks?

I would really hate to see this project leave GitHub.

Contributor

afahim commented Feb 11, 2016

As a fairly nascent developer, I think one of the things that I / people with my experience level care about is establishing credibility / getting credit for open source contributions, and whether we like it or not, github has become the first place to look whenever someone is trying to judge someone's contributions. It shows contribution activity loud and clear on profile pages, and this type of attribution is really helpful / important for new contributors to prove their mettle in the open source world. I feel that not having this kind of attribution would not provide enough motivation / demoralize a lot of people who would want to contribute, and I think this kind of lack of motivation and enthusiasm can be to the detriment of the project in the long term.

Given those considetations, I would really hate to see this project leave Github.

Owner

mysticatea commented Feb 11, 2016

Honestly, I love GitHub. I hope to not leave from here.

Owner

alberto commented Feb 11, 2016

👎 to moving away, unless there are very clear benefits.

What is making us move? Is it some or all of the points of the "Dear Github" open letter? Anything else? Can we solve it without moving away?

Contributor

ljharb commented Feb 11, 2016

Not being on Github is a huge barrier to entry. Babel's move to Phabricator was perhaps a clear win for the maintainers, but I believe it's a huge downside for everyone else.

While it's important to make sure maintainers are able to sustainably maintain a project, it would be a very sad thing if users had to bear a burden because maintainers wanted to lighten theirs.

I know the issues on Github are terrible. But the benefits of the Github community are great. Please keep in mind there is a barrier for new users - everybody is familiar with Github which is not the case for Gitlab nor Phabricator.

What about bitbucket?

Bitbucket seems to have a lot of downtime, I would suggest following their Twitter account to check it's not an issue for you

Member

BYK commented Feb 11, 2016

As a long-time Phabricator user, it looks very compelling to me but the above-mentioned barrier to entry is quite high, especially for new comers. Right now GitHub even allows submitting a PR purely from the web interface which may help people get started with quick documentation fixes. This is not possible with Phabricator.

Idea: move everything to Phabricator, including bug reporting but keep a GitHub mirror and only allow pull requests. This may address most of the complaints with minimal contributor impact. Since Phabricator supports many third-party logins, including GitHub, that shouldn't be a big deal.

It does support CI integration but may require some extra work to make the actual integration work. I'm willing to help with this if we decide to chose Phabricator.

Member

BYK commented Feb 11, 2016

travis-ci/travis-ci#2143 is the ticket for TravisCI - Phabricator integration. Looks like it won't happen soon unless someone tries to do it themselves. travis-ci/travis-ci#2143 (comment) shows a viable alternative.

I think following Babel example and moving to Phabricator might be a good solution. Perhaps @kittens could share how easy/difficult was to move to Phabricator.

I'd like to reiterate @RReverser's comment about Zenhub. I'm not associated with it, and I don't use it for any serious organization yet, but I surveyed it for use in our company and it solves a lot of the problems brought up in "dear github".

Note that one thing that it doesn't "solve" is that, since it's used primarily as a browser extension, viewers of your repo who don't have it installed won't be able to use any of the sweet features. For example, a user creating a new issue won't be presented with any configurable fields (e.g. "what did you expect to happen/what happened").

Still, you should give it a look and see if it might fit your flow to reduce "paperwork" costs.

Owner

nzakas commented Feb 11, 2016

I hear everyone's concerns about moving off of GitHub, it's not something I want to do. There's a clear barrier to participation elsewhere.

However, what we are doing right now is unsustainable. It's not just that the GitHub experience is suboptimal, it's that it aggressively undermines development. There's just no way we can keep up with the pace of issues anymore, especially when most of them lack enough information to be useful. The tooling isn't good enough for the onslaught we receive.

Really, it comes down to this: something has to change. It seems like GitHub won't change, or even respond to these concerns, so we have to do something.

Another option is that ESLint users step up and help us manage issues. I made that request in #5083 and got zero responses. If you're one of the people who commented about not wanting us to leave GitHub, are you willing to help out to make dealing with GitHub easier on the team?

Regarding ZenHub - looks interesting, though it doesn't solve our worst problem, which is issues filed without enough information. Also, I'd be looking at $50/month for the whole team, and more if we add more people to the team. It's not really economically feasible without sponsorship.

Owner

nzakas commented Feb 11, 2016

Oops, looks like ZenHub is free for open source: https://www.zenhub.io/open-source

Zenhub is free for OSS

Edit: You beat me to it @nzakas lol

Have you considered a bot that would triage issues for you? Have it do some checks on the issue and respond with pre-configured responses and apply labels based on that? (edit: nevermind I just noticed eslintbot)

I highly support the dear-github movement, but it's a catch 22. Leaving GitHub and staying with GitHub both bring their own pros and cons. Unfortunately, all the cons of leaving outweighs the pros in my eyes.

Owner

ilyavolodin commented Feb 11, 2016

@joshmanders We already have a bot that does basic request for additional information. Most of the people simply ignore it:-(

Contributor

MoOx commented Feb 11, 2016

Maybe working on the bot is not a bad idea. You could even auto close an issue if nobody responded to the bot request?

@ilyavolodin Have the bot pass through again after a time period and close the issue with a message.

Member

BYK commented Feb 11, 2016

Working on the bot also sounds like a good idea. It would automatically close the issue if certain values cannot be matched in the original issue message such as ESLint version etc.

@MoOx I have plans of creating a bot that does a lot of triaging and that for issues and help maintainers, unfortunately I haven't gotten farther than registering an account for it and start planning, but my goal is to create a bot that does a lot of offloading and providing it to communities to either host themselves, or remotely hosted.

I just wish I had more time to dedicate to it, because I know a lot of people are feeling the pains and could utilize it heavily and most people have their own bots that do a few things and that's it.

Owner

ilyavolodin commented Feb 11, 2016

We could beef up our bot to be much smarter, true. But in all honesty, in that case, we are going to be spending time developing bot's code, not ESLint:-( The other problem is that we already get complains from people thinking that our requirements are too strict. Having their issues auto-closed will only make it worse. I know that's not a good argument, but never the less, that's probably what's going to happen. And it does not really help team's productivity to see tweets saying how horrible and unaccommodating ESLint's team is.

Contributor

MoOx commented Feb 11, 2016

Bot can send sweet and kind request multiples times with different "degree" warnings. Automating what can be made is always a good idea. For another organization we are going to work on a small bot. I am sure this initiative could be shared for a lot of organization.

Have you guys considered how the Ionic team does things too? Seems helpful. http://blog.ionic.io/how-ionic-uses-github-better/

I wonder how other (even bigger) projects managed to stay productive on Github. Perhaps it would be worth getting in touch with core maintainers of bootstrap/node/jquery/etc to see how they managed to overcome the issues you mention above?

ariya commented Feb 11, 2016

@ilyavolodin Users don't need to create a GitLab account to e.g. file an issue. They can sign in using their Github credential (done via OAuth).

image

Contributor

ljharb commented Feb 11, 2016

Phabricator allows the same thing, but "sign up with Github", while a prerequisite for usability, does not eliminate the friction of having to switch sites.

One other big problem is the auto-linking of issues/commits/PRs that Github allows that would simply not be possible if eslint was hosted elsewhere. It's incredibly helpful to be able to visually see on an issue/PR of mine that a linked issue is closed, or a linked PR is merged.

Team members spend hours each day triaging issues, many of which have incomplete information or are otherwise unclear. As such, we spend a lot of time just asking for more information ("what do you mean?", "what version are you using?", etc.).

This has nothing to do with GitHub, but I see your point about lack of action/response from them.
ZenHub is great for sure as well. In fact, I keep a repo just for issues about GitHub, with a ZenHub-able label. Almost everything I want falls in there.

https://github.com/kevinSuttle/github/labels/Zenhub-able

Member

hzoo commented Feb 11, 2016

I think the familiarity of github is super important, but it's obvious that with a large project github issues just can't scale.

Since Babel/Phabricator was mentioned, I'll say that we could of handled the move to it better and are probably losing some contributors by doing so. It's really nice to have things like issue templates and all that it provides (there's probably a lot that is yet to be explored). It definitely looks like the quality of issues is much higher (a lot of useful information is provided upfront and there's less back and forth) and easier to debug/followup on, but yeah still a social/people issue where we want users to be welcomed, feel like they are helping, etc.

It's easier to manage, people aren't complaining about the switch as much from what I can tell, but we aren't getting like more PRs or same contributors back so I don't think it can solve everything just switching? We do have instructions in the readme, our contributing.md was telling people not to do support in the issues, and there's links on the website, but sometimes there's still issues that are created about using babel that are created on the website repo since they can't find the issues on github, or people will say things are broken on twitter or start emailing us.

It's a hard problem, and it can be a struggle to think about the "wants" of the maintainers while keeping a low barrier of entry to contribute (in any way). Users are still who we're supposed to be serving and helping? There's probably a lot that can be improved on our side that we aren't thinking about - maybe we should focus on making it easier to contribute and giving others responsibility.

Don't know if that was useful at all.

thasmo commented Feb 11, 2016

It was mentioned before; what about https://github.com/kentcdodds/issue-template as a first step or so.

Owner

nzakas commented Feb 11, 2016

As far as I can tell, here's what would likely happen with any move (regardless of location):

Pros

  1. Much easier to triage issues by enforcing or at least suggesting proper information to include.
  2. Easier to track and manage ongoing issues.
  3. Better tools for managing code and releases.
  4. More time spent on code, less time spent on issues.

Cons

  1. Lose visibility/discoverability.
  2. Lose casual contributors (likely forever).
  3. Lose some bug reports because people don't want to sign up for Yet Another Account.
  4. Lose some feature requests for the same reason.
  5. Less friendly to beginners.
  6. Lose ecosystem compatibility (Travis? Coveralls? BountySource? Bots?)

Analysis

What the community ultimately wants is for ESLint to continue to grow and be developed. What the team wants is to be able to continue to grow and develop ESLint. The community believes that this can happen while staying on GitHub, the team does not. So there's got to be something that gives.

If we remain on GitHub for, let's say, the next year, it's likely that the project will stagnate under the weight of feedback in the form of issues. Everyone on the team is doing this in their spare time, and as the project starts requesting more and more of that free time, we risk losing team members, which would cause the project to further stagnate. If we got more team members, that would temporarily relieve the pressure, but people tend to come and go so there's no guarantee of anything. In all likelihood, we'll have to start ignoring issues just to make progress on actual code, and that would result in community disappointment.

If we move off GitHub, we will immediately eliminate the random noise we get through issues. Increasing the barrier to file an issue slightly, both by requiring another login and by having issue templates, decreases the amount of issues we'll have to triage. It's possible we'll miss an important bug because of this, but I have to believe that if something is really problematic, someone is going to eventually break through and file the issue. At the same time, we'll eliminate drive-by pull requests that also need reviewing. We'll likely be able to hold onto all the team members because the workload will remain manageable. We'll also get fewer new contributors because they'll be less familiar with the system, which means the team growth will be slowed. However, all of this means a much more stable and predictable path forward for ESLint, which means more time to work on the code and less fussing over issue management.

Owner

nzakas commented Feb 11, 2016

@thasmo that only works if you can funnel people through a different link to file issues, which is virtually impossible. We've tried using different "Report bug" links and such, and no one ever clicks on them.

How about recruiting people to the team who's sole job is to just triage issues? I mean saying "Sure help out, tackle those issues" isn't as inviting, but saying "Anyone who is part of this group of people, will be official team members, just like core contributors, and will be recognized as an important part of the team." could be a little more inviting.

@joshmanders it's 2016

@knowbody whatever that's supposed to mean.

That we have computers to solve those problems.

Well clearly it's not helping. I'm just throwing ideas out there. The biggest issue seems to be triaging issues. Leaving Github I feel will hinder the project more than help it.

Owner

nzakas commented Feb 11, 2016

@joshmanders what is that opinion based on? How would it hinder the project?

Owner

platinumazure commented Feb 11, 2016

@nzakas Let's suppose (for discussion's sake) that ESLint decided to stay on GitHub, or at least was open to it iff folks in the community could help with triaging issues. What could people in the community do to help make that easier for you?

I didn't mean to be rude sorry. I meant that we can either build something or use existing tools. Can't imagine having ppl to just dig through issues.

Contributor

ljharb commented Feb 11, 2016

I think it's unfortunate that "working on the code" is considered more important than "issue management". Just like when a company grows, as your responsibility increases, your individual contribution decreases. I think adding more collaborators, specialized in specific skills like issue triage and PR review, is analogous to the proven solution of growing a company by hiring similarly.

kittens commented Feb 11, 2016

@ljharb I'm not sure why you got that impression from this issue because I got the opposite one. Babel moved from GitHub to Phabricator precisely so we could make managing issues more effective and scalable.

@nzakas discoverability is huge, and as easy as people think it'd be to just up and move to say GitLab, that's basically a death sentence for anyone hoping to receive help from the community. Take me for instance, I understand the usages of all those other services, but 99.999% of all of the things I use and help with are on GitHub, 1-3 projects leaving Github isn't going to get me to go "welp, time to check yet another site to help fix issues with the tools I use" no, I'll just find another project that's on github. It's a catch 22. Sure, you'll have less issues triaging issues and that, but that's at the fact that nobody will post any.

I feel developers are too deep into the toolsets to realize that leaving a standard and heavily used by EVERYONE resource like Github is going to make their end users just go "Off to gitlab we go"

kittens commented Feb 11, 2016

Also to the people saying discoverability is going to be halted, when projects reach a certain size user growth is less of an issue than maintaining the current user base. Once you reach critical mass and have something that's useful the location of the code and issues is less important. Most ESLint users will be using the websites documentation and this wont change because the repo shifts from one website to another.

Owner

nzakas commented Feb 11, 2016

@ljharb Unlike with a company, no one is getting paid to do this. It's easy to say, "just get more people," but very hard to do in practice.

@joshmanders ESLint doesn't have a discoverability issue at this point. In the last 30 days, ESLint was downloaded 1.6 million times. Discoverability is a problem for new projects, not established ones.

Owner

platinumazure commented Feb 11, 2016

@nzakas Is it possible to add users who have some collaborator permissions but not others? E.g., can label issues, can't push?

@nzakas Sorry discoverability was the wrong word. I mean support. As @kittens says Babel moved to phabircator, I have an account there and commented on one issue, and as heavily as I rely on Babel, I am not compelled to go to Phabricator to read issues or participate in them. It sucks that issues and code are separated. I'm personally just too lazy to go that extra step. I'm certain many are. I could be wrong, just speaking for personal experiences.

I have visited ESLint's site like 3 times, honestly. But most of the time I come to the github repo and read stuff, including code, because you can't count on outside resources being up-to-date.

Owner

nzakas commented Feb 11, 2016

@platinumazure not on GitHub.

@joshmanders what you're describing with Babel is the "casual contributors" I mentioned in my previous comment. Yes, we lose that, but in some ways, that's also what we're looking to filter out. People are no longer just going to happen upon ESLint and randomly decide to open an issue. Instead, they'll measure if their request is important enough to be worth the extra work of opening an issue. That alone will reduce the amount of noise. You may not spend time perusing issues anymore, but the people who are working on the project every day will be.

Contributor

inikulin commented Feb 11, 2016

I found it painful to navigate through and fill issues to Babel after it's move to Phabricator. Maybe it's just me, but I was not able to do some basic stuff in the search (like sorting issues by date). Filling issues was not fun as well: GH markdown is really handy comparing to what is present in Phabricator. The overall impression was that transition made issues management easier for the maintainers, but not for users. And I agree with all other points above: GH is not just a code hosting, but unique social network that enables many collaboration/tracking/automation scenarios which are not possible in the isolated codebase.

Contributor

jmm commented Feb 11, 2016

I can't speak for anyone else, but I've been underwhelmed with the Phabricator experience since Babel issues moved there. Perhaps it has a lot of potential to improve things, but I also don't know how much work that would take or who's interested enough in figuring out Phabricator to do it.

Having a template for new issues is probably a great way to try to encourage higher quality reports, but it doesn't stop people from filling it out with garbage. There's still plenty of low quality reports coming in to the Babel Phabricator. Although not providing repro code is a problem, dumping a bunch of irrelevant code into an issue report is also a problem. Example: I dealt with an issue on the Babel Phabricator recently where the reporter first didn't provide sample input at all (paraphrasing):

If the source is needed I can supply it but it's been happening with multiple files...

Then through back and forth I gradually coaxed them to provide a working repro and then still had to trim their sample input from 620 bytes to 26 to get to a minimal repro without so much noise.

Owner

nzakas commented Feb 11, 2016

@jmm I don't think anyone expects all problems to go away, I do expect, however, that a certain level of noise will be eliminated.

Contributor

jmm commented Feb 11, 2016

@nzakas Maybe it will. I'm not convinced that's been the case with Babel. And meanwhile I find the Phabricator interface confusing and clunky. And since moving there reporters have been able to do crazy stuff like assign issues and set priority level themselves.

larkin commented Feb 12, 2016

What you do is aggressively close issues that are poorly written or unhelpful. If people won't follow the instructions in your CONTRIBUTING.md, then you don't need to give them the light of day. You're not here to hand-hold junior developers-- they need to learn to investigate and get to the root cause of issues. If they do a poor job of bug reporting, and they feel they should get more respect than a closed issue, well, they can write up a new, better issue, and submit that.

Taking a firm stance should cut down on time spent working issues. People who need support or have found a legitimate bug will work hard to demonstrate that-- they will put in the work to write a good issue, and others will not come back.

Expect more of your developers and your community-- expect them to be smarter and hard working. And they will be. For those who won't put in the time to work with you, you don't need to put in the time to work with them.

my 2c.

dcposch commented Feb 12, 2016

Just an observation: if Node itself is happily using Github, then a JS linting tool should def be able to do so as well.

It just requires disciplined organization.

Owner

platinumazure commented Feb 12, 2016

Think I'm in agreement with @larkin. There's a difference between having expectations of developers and being "mean". We can't afford to "be nice" to junior developers when it drains so much time and productivity.

kittens commented Feb 12, 2016

@dcposch

Just an observation: if Node itself is happily using Github, then a JS linting tool should def be able to do so as well.

It just requires disciplined organization.

You're forgetting that Node has a very large contributor base relative to other large open source projects and it even has people paid fulltime to work on it.

Member

IanVS commented Feb 12, 2016

It seems React Native had some pain around issue management and prioritization, which drove the introduction of Product Pains. Their bot leaves a comment on GitHub issues asking users to submit bugs and feature requests on Product Pains (example comment).

It would be interesting to hear from someone on the RN team about their experience triaging issues and whether Product Pains has helped to ease their own pains.

JaKXz commented Feb 12, 2016

I don't know if this has been suggested yet (long thread and I'm on my phone): waffle.io?

vjeux commented Feb 12, 2016

@IanVS for React Native, what we've been trying to solve is how to prioritize what the community needs. At this point, there are way too many issues for our team to handle and most of them are very low value and would take far more time and energy to fully figure out than actually working on the project itself.

We've tried a bunch of things to extract actionable things from issues like tagging all of them by area and counting the most issues per area, sorting issues by comments... While they gave some sense of what the community wanted, it was very noisy.

The big push that we've done was to setup Product Pains. The goal is to be able to have a ranked list of what people building react native struggled with the most. It has been very successful, the first page is full of very actionable and impactful content. We are using this in order to influence our roadmap.

At this point, we have pretty much given up on managing issues and aren't closing them anymore. Closing issues is causing a lot of drama because people take it as an attack so it's safer to just leave them open.

The way the team (fb and community core contributors) communicate is via a secret facebook group. This is a space where we can have high signal communication. Having this kind of communication in a public space where everyone can comment quickly turns into big discussions with too many cooks in the kitchen and is ultimately not productive.

Hope that it gives some context. Let me know if you have any more questions, happy to follow up

aloisdg commented Feb 12, 2016

An update: I had a nice conversation with one of the folks at GitHub. It sounds like they do understand our concerns and have some plans to address them.

All problems wont be here if GitHub was a FOSS. New problems will rise and you cant do nothing about it. If you found an issue on Phabricator or GItLab you can report or even fix it. You are confined and limited on GitHub. We cant participate to all of this. It's manage by GitHub, only...

jlukic commented Feb 12, 2016

Thanks @vjeux I find your commentary insightful. I think I've been feeling the same things lately. It's unfortunate for the public/private divide, but it doesn't seem like theirs much other recourse for maintainers.

jasom commented Feb 12, 2016

Note that phabricator allows you to still use github for source-control; phabricator is perfectly happy with the repository being somewhere else.

jmurzy commented Feb 12, 2016

@larkin You beat me to it.

GitLab FTW!

@rrjanbiah rrjanbiah referenced this issue in RestyaPlatform/board Feb 12, 2016

Open

Address few GitHub pain points #241

nothing can beat JIRA here

Marak commented Feb 12, 2016

Well.

It seems your main issue is dealing with a large number of Github Issues.

As someone who has spent a lot of time triaging 1000s of issues across 100s of github projects for the past six years I can offer a small amount of advice no one else seems to be touching on.

If your users are filing too many support issues, that is a direct result of your software and documentation not offering enough meaningful information to the user. You can't blame your users for this.

For example, if I search on your Github README, I can find NO information of how to file a support issue. There are rules for filing support issues? Where you have clearly expressed this to your users? You have a bot which automatically responds to Github Issues? That is reactive support. You need to implement proactive support. I did finally find the rules for submitting a Support Issue, after clicking into the very small "Contributing" link. The thing is, I'm not a contributor. I'm a user with a support issue. I'm not going to click "Contributing". I'm going to immediately file a support issue without reading anything about "Contributing".

You need to establish clear channels for support. I see a one line section at the bottom of your readme titled, "Where to ask for help?". How am I suppose to ask for help on these mediums? I'm confused. No instructions have been given to me as a user so I don't know what to do. I'll probably open a support issue on Github, get an automated response from the bot, and give up.

Another thing you can do is to temporarily ignore any issues which aren't pertinent or immediately pressing. If a user asks a not-so-smart question, you shouldn't stop what you are doing and immediately respond. Let the issue sit for a few days...maybe they will resolve it themselves...maybe another community member ( who is not a core contributor ) will step in and help resolve the issue.

Try to remember you are running an open-source project. Responding to Github Issues is not bringing you revenue, so you should be careful in how you spend your most limited resource: time.

You seem to want to fortify yourself away from the users so you can be shielded from their unbridled contributions and communications. If you feel that is the best way to run your project, that is your choice. Personally, I would rather have the additional signal and not worry too much about filtering out the extra noise.

The way i see it:

  1. Will people find us in another location?
  • No probably not. But you can bet your ass that somebody is going to clone the repo and dump a github one right back.
  1. How high is the barrier to entry for new contributors if we're not on GitHub?
  • High, plus you don't see any/many other projects make the step. Got to facepalm when i saw the headline.
  1. What about the Gitter chatroom? What happens to that? Do we need to find another chat location?
  • Main downside is that it doesn't have a W10M app and is locked to 25 users (if used free).
  1. Others?
  • How about Slack? Seems to be more popular amongst other communities and there are various solutions to get registration in order (just look up on some free Heroku and other solutions to handle registrations)

And instead of just blaming Github and be annoyed about the issues. How about a new structure to work around that? Or still keep the Github for release candidates and normal releases but skip the normal commits and beta's?

Bounga commented Feb 12, 2016

@nzakas Bitbucket seems to fulfill all you requirements. I use it since 2008 and it is very good! They are very concern with their users issues. And even if it's not very important for you, you'll have unlimited private repos. You really should give it a try, just to see what it does.

@ilyavolodin You can login to bitbucket using your github account.

Github is not the only viable solution out there, it's more of a trend. Some well known developers and projects helped a lot Github to become mainstream but there's other players around that does a good job also (maybe a better job in fact).

Contributing won't be harder on another host. Bitbucket for example will allow the same workflow (fork / create a branch, push, send PR). You've a nice interface with button to fork, create PR and so on.

I don't get the point of developers saying: «You need to be on Github or your project will never take off». That's not the host that make the project interesting. It's the project itself, its qualities, its contributors, etc

Contributor

glenjamin commented Feb 12, 2016

@ilyavolodin You can login to bitbucket using your github account.

This is getting "phased out" - https://bitbucket.org/account/signin/?next=/#social-auth

Bounga commented Feb 12, 2016

@glenjamin Oh yes you're right. So use your google account, or just create an account (that's what? 2 or 3 clicks). If you don't have the motivation to create an account, I'm pretty sure you won't have the motivation to create a good PR or a good issue with all the relevant info.

If self-hosting is not out of the game, maybe pagure could be an option.

It's all python, it provides quite a bit of the features github offers and it's entirely FOSS.
On the cons side, it's relatively new/young so there are likely some features that are missing but I am sure upstream would be very happy to see RFE being filled on the tracker :)

petetnt commented Feb 12, 2016

I want to chime in and agree with what @Marak said.

If the biggest issue here is low quality issues, why on earth is the bug reporting guidelines stashed away under Contributing -> Bug Reporting -> Actual bug reporting guidelines page. Why not on the README.md itself right under the Usage and Configuration sections.

It might not be the prettiest place for it, but it seems to bring much more trouble to the eslint repo itself than say people having trouble seeing what the sponsors or the team behind it are. Not to say that information is invaluable.

kittens commented Feb 12, 2016

fwiw for Babel we had literally a 3 line README.mda with the one notable in bold "don't post support questions here", we even had a massive H1 heading in bold in CONTRIBUTING.md saying not to do this. Want to guess what happened? Nobody read it and people opened issues anyway. People don't read the README.md, even if it's super tiny and people certainly don't read CONTRIBUTING.md.

GitHub has encouraged a culture where people just create issues willy nilly. It's super cheap for users to open issues but it's extremely expensive for maintainers to close them.

DjebbZ commented Feb 12, 2016

The Clojure Community led by Rich Hickey decided to use Github for code but use JIRA as an issue tracker. They also require contributors to sign a CLA and to send patches in JIRA issues, not PR. So the barrier of entry is very high, but all contributions/issues are meaningful. I'm not suggesting to do the same, just pointing to another successful OSS project that don't use Github. JIRA is by far a superior issue tracking system.

As @kittens just said,

GitHub has encouraged a culture where people just create issues willy nilly. It's super cheap for users to open issues but it's extremely expensive for maintainers to close them.

So don't use Github issues if that's the problem.

Also, @nzakas and ESLint people, have you already investigated distributed issues tracking system ? They're text-based and committed to the repo. My small investigation tells me that Bugs Everywhere (OSS, written in Python) is the most feature complete, that even comes with a web-based interface. Never used such a system but I find them very interesting in the principle.

My 2c. Hope this helps the discussion !

petetnt commented Feb 12, 2016

@kittens This is just hindsight 20/20 but if you look at how the 3 line page version (one of them, at least) looked at a certain point of time:

https://github.com/babel/babel/blob/f4a3b2f01ae17acddb78b00cee7fb5cbd4a1524f/README.md

Babel is a compiler for writing next generation JavaScript.

For questions and support please visit the gitter room or StackOverflow. The Babel issue tracker is exclusively for bug reports and feature requests.

Issues without instructions to reproduce will be immediately closed.

For documentation and website issues please visit the babel.github.io repo.

While it's short, it's also very vague, especially for the demographic that contribute with bad issues. For example:

  • What is gitter? What is StackOverflow? Why would I want to use those? Why can't I just ask the question here?
  • What's the difference between questions, support, bug reports, feature requests?
  • Issues without instructions to reproduce? Reproduce how?

Obviously some of that tie in to the things in Dear Github-letter. But just having more labels doesn't solve any of them. I think that the Discussions-tab, for example, would be very neat addition but it alone wouldn't solve anything either. If you cannot close an issue by just stating "Hi @ username, this issue doesn't contain enough information for us to proceed" or "Please ask support questions at SO" then it might be an issue with the delivery. Popular projects attract people of all kinds of skillsets and language barriers.

AFAIU, the issue with moving to another platform like Phabricator is that it hampers support/discoverability/increases barrier to entry of contributors/etc; but if in a parallel universe, these problems weren't present, ESLint from that universe would have happily moved away to another issue tracker. Right?

If so... I suggest moving to another issue tracker in this very universe, and patch the aforesaid problems with a bot:

  • keep the issue tracker on GH open
  • Have the bot mirror all new issues on the new platform
  • Have the bot close the GH issue instantly (possibly even lock it to collaborators)
  • Have the bot leave a polite message explaining the situation on the closed issue, with a link to the mirrored issue and request to have all further issue opening, and conversation on the new platform only

This ensures that you guys get to hear everyone's views. People who care enough to follow up will likely follow the links given as well. And you get to enjoy another platform.

agilob commented Feb 12, 2016

Give gitlab a try. I wrote a post on my blog trying to show I like the most about gitlab. I'm in no way affiliated with them.

For what it's worth, GitHub responded to dear-github; dear-github/dear-github#115

Bounga commented Feb 12, 2016

@agilob If Github doesn't work for them, how Gitlab would? I'm using it every day at work and it's more or less a Github clone. The only benefit is it's self-hostable and free.

Contributor

jmm commented Feb 12, 2016

@nzakas

@jmm got it working: #5217

Awesome! Thanks. Is there info anywhere on how your bot is setup?

agilob commented Feb 12, 2016

@agilob If Github doesn't work for them, how Gitlab would? I'm using it every day at work and it's more or less a Github clone. The only benefit is it's self-hostable and free.

@Bounga Because it covers their requirements in 100%

Must have a way to setup issue templates prescribing what fields are required

Supported, configured in repo settings, for both issues and merge requests.

Must have ways to organize issues other than labeling (milestones, priorities, etc.)

All supported by GitLab CE (gitlab.com). Issues also have weights, votes and belong to milestones, have labels and users can vote on issues.

Must import existing GitHub issues and pull requests

Not sure about pull requests, but issues and wiki are supported.

Contributor

MoOx commented Feb 12, 2016

Pull request on GitLab are called "merge request" (which is the appropriate name according to the action) :)

@sarunast already posted.

Some great points on this subject from the folks behind successful open source projects such as Rails and Elixir (@josevalim): "Tips for keeping your Open Source Software issues tracker tidy"

Might help providing some insights.

We struggle with this too @ Basho. We are using JIRA and GH, which is kind of a pain. We have product development, custom development, and project management in JIRA while GH gets most of the bug & community stuff. We have a service that links the two, but the problem is that we are bouncing around, getting double notifications for things and it can create a confusing user experience for users in GH when an issue is updated automatically from JIRA because our PM changed something.

What we want is something like Waffle.IO that has PM capabilities + the ability to create private tasks / projects that don't get pushed to GH + ability to have a project that spans multiple repos. Haven't really found something like it yet.

First off, thank all of you for working on ESLint!

I've worked on a bunch of open source and closed source software projects, using GitHub, Trac, Phabricator, Bugzilla, Mantis, FogBugz, and other issue tracking systems. Yes, none of them are perfect. GitHub has less customizability than, for instance, Bugzilla. I wouldn't be surprised if you do decide GitHub doesn't work for the particular workflow you want. (Are you using free GitHub hosting or have you paid for extra features?)

Bots and automation can help triage bugs, but in my experience there is no complete substitute for thoughtful people using their brains to address each one. Once someone gets the hang of it they can move pretty fast, and yes, this absolutely is an opportunity for new contributors. The Wikimedia "how to triage document" is a useful model here -- if you can add stuff like that to the "working on issues" HOWTO it'll help new people start as bug triagers. Wikimedia trained an intern as a bug triager and the Outreachy internships are a good match for this kind of work. Someone who starts by triaging bugs is often able to help a lot with release management, prioritizing features, communication with important upstreams and downstreams, and similar work.

I see some folks earlier in this thread suggesting better bug reporting guidelines and suggesting helping users with templates for required information. That is pretty helpful. I see some other folks using wording that implies that people submitting incomplete bug reports are stupid, lazy, or undeserving of help. That, in my experience, is an approach that leads to a bunch of less optimal outcomes. Every bug reporter is, by definition, running into a problem they don't know how to solve themselves; it's unlikely you're running into them on the best day of their life. I've usually found that a little human kindness and "could you go into more detail on points x, y, and z so we can have a better chance at solving this problem?" leads to bug information that helps developers make much more robust software.

Having a good short set of required data fields is useful. But it pays to be careful in the user experience around that. In my experience, it comes across as hostile to go with the "you didn't add required data, so we're auto-closing this, re-open if you think your new report is good enough" approach. That approach makes users more likely to think of maintainers as aloof and unresponsive; then, when a bug comes up that you really wish you'd known about, they don't bother telling you. A good middle ground is to ask for more data, and wait a few days, and then leave a kind explanatory message when closing the issue.

Hope this helps.

I'm with @Nejuf and @JaKXz to suggest you waffle.io

waffle

For using it on a daily basis, it's pretty ergonomic and allows you to benefit the hosting power of Github without pain (or, at least, less pain).
I haven't tested it with a larger project as yours, but it deserve to be tested IMO.

Owner

nzakas commented Feb 12, 2016

@Marak Thanks for that, these suggestions are very useful. We can definitely make it clearer what is expected and how to request help.

You seem to want to fortify yourself away from the users so you can be shielded from their unbridled contributions and communications.

This is incorrect. We want a system that allows us better control of the contributions and communications. A side effect is that we'd lose some of it, but as I've said previously in this thread, I believe what we lose would be mostly low-level noise.

Owner

nzakas commented Feb 12, 2016

One thing I want to point out to a few people who have said things along the lines of "projects much bigger than you seem to be doing okay on GitHub."

Projects much bigger than us tend to have full-time developers and/or much larger teams. Node.js has not only a large number of contributors, but paid developers and the support of a foundation. npm has a company backing it. React/ReactNative have paid developers. The list goes on.

ESLint is run by volunteers working nights and weekends (and, to be fair, we also sneak in during the day to do some stuff when we probably should be focusing on our full-time jobs). The number of people who consistently make contributions is fairly low (< 10) and the number of people who actively help triage issues is even lower.

As I've mentioned earlier, part of the solution could be for more people to contribute. I've publicly requested help for issue triaging and had zero responses. Running an open source project doesn't mean contributors magically appear to help with everything, there's a lot of begging and pleading. (ESLint is certainly not the first project to face this problem.)

So while others have been able to make things work on GitHub, different constraints and resource availability mean that it's not a one-to-one comparison.

mkurz commented Feb 12, 2016

GitHub responded to the "Dear GitHub" open letter: dear-github/dear-github#115
New features are coming to issues soon! Maybe wait for if them and give GitHub a second chance before switching to some other tool?

Saw this on HN today: http://gitmagic.io/

Might be helpful.

aldanor commented Feb 12, 2016

if GitHub indeed rolls out new issue features in the coming weeks, maybe that would be sufficient?

brodock commented Feb 12, 2016

GitLab have "Page Hosting" support like GitHub Pages. There is a WIP Merge Request to have CNAME and SSL supported for next release: https://gitlab.com/gitlab-org/gitlab-ee/merge_requests/173.

You can read more about GitLab Pages here: https://about.gitlab.com/2015/12/22/gitlab-8-3-released/

Owner

nzakas commented Feb 12, 2016

@aldanor yes, it could be enough to keep us around. I want to see what improvements are made before committing to anything, though. In the meantime, we're still going to be exploring other options so we're better informed.

The real question: is this truly a technical issue, or is it a community issue? I.e. are there ways to grow the community by using your governance and docs to draw in new contributors who can do useful work, not just submit incomplete tickets or the like?

Just a thought from the peanut gallery. If you're having this big a discussion on the details of tools that you use for the work, perhaps it's time to re-examine with a fresh eye how the social governance and interaction really works.

(Yes, I know, it doesn't always help, sometimes it's really hard to get new users to become contributors - but that's the only real way open source communities can grow over the long haul.)

Good luck!

Qix- commented Feb 13, 2016

/quote

  • Must host the repo along with the related tools
  • Must be able to run automated tests on pull requests
  • Must allow contributions from anyone
  • Must have a way to setup issue templates prescribing what fields are required
  • Must have ways to organize issues other than labeling (milestones, priorities, etc.)
  • Must import existing GitHub issues and pull requests

/endquote

Seriously? What is on this list that Github can't do? You literally described all of the core features of Github.

What problems are you facing with Github, specifically?

Maybe we should consider BitBucket.

No, you shouldn't. If you think Github has abandoned its product, then you should see Atlassian's complete disregard for its product(s).


This is going to come across as being a jerk. Take it for what it is: direct, and subjective.

ESLint's governance is stiff and rigid. You guys have insanely high bars for what are considered "good" contributions. Maybe this is a good thing, maybe not - either way, your backlog isn't due to any sort of technical shortcoming, but a lack of involvement of collaborators (or maybe the number of collaborators?).

Another issue you're facing is an issue that most large OSS projects face - writing a PR for a feature/bug takes way longer than opening an issue, and most people are not willing to take the time.

I personally speculate that this is because OSS development has been gamified a bit too much. But that's just an opinion.

Switching to another tool is just going to derail this project beyond any sort of meaningful use. You're going to lose a buttload of contributors, watchers, experts, veterans and users alike.

You're also going to force contributors to learn a new product. However symmetrical Github and XYZ product are, you're going to be forcefully introducing human error. Isn't that what you're trying to "fix" in this issue to begin with?


👎 for this. If you do it, you'll lose quite a few users, contributions will diminish, and another, better alternative to your project will emerge.

Until Github itself dies, this is the best place for it to be. Github doesn't care if you leave. You're not sending a message by taking ESLint off of Github.

To me, this is trying to find drama where there is none and completely disregarding the practicality of migrating a project off of an established website considered 'home' since its conception.

That is my opinion.

So GitHub reached out and talked to me after responding to the dear-github letter earlier today. It sounds like the letter did more good than I expected it to, and all I'm going to say is that I'm looking forward to the things that they'll be doing in the next few weeks or so.

@nzakas I would hold off on making a decision for now. And you might want to consider locking this issue because it's turned into an exercise on who can write the most exhaustively long comment stating their uninformed opinion as fact.

@cpaulik cpaulik referenced this issue in syl20bnr/spacemacs Feb 13, 2016

Open

[META] Incremental process improvements #5037

Awesome news, and great work @thejameskyle!

Qix- commented Feb 13, 2016

@thejameskyle that's exciting. Agreed on all points.

Hi,
what about having the bot sending a nice email directly to the bug reporter, asking him to respond the bot with a provided template ? It would then be posted to Github.
It won't be a silver-bullet, but it may be coded quickly and hopefully reduce at least a bit the unactionnable issues.

GitHub has added issue templates silently. dear-github/dear-github#125

All you have to do is add a issue_template.md file to your repo and when a new issue is created it should prefill in.

@joshmanders It doesn't work with mobile access (Chrome on my Nexus 6P) or ForkHub on Android though, but it's definitely a start.

@brianjking indeed, just checked on my iPhone 6, it don't work. But agreed. Great start.

Owner

nzakas commented Feb 17, 2016

@thejameskyle as I mentioned earlier, we are continuing to investigate alternatives. While I'm happy about issue templates, I don't see any reason to stop looking into what other alternatives are out there. I've been having some great conversations with the folks at GitLab about their plans, and I'll post more about that when I have a second.

However, I am locking this issue at this point, not because I see any offensive comments (in fact, I'm thankful for the mature conversation that has take place on this issue), but because I feel like we have all the data from the community that we need. We know that moving off of GitHub would upset some, and that's why we're hopeful we don't have to do that. The team will continue posting updates here as things progress.

I want to especially thank the folks who also run large open source projects for their suggestions and sharing their insights. It's been invaluable to us.

@nzakas nzakas locked and limited conversation to collaborators Feb 17, 2016

Owner

nzakas commented Feb 22, 2016

I just wanted to add some data I got from GitLab:

  1. They'd be willing to give us an Enterprise Edition license so we could run it ourselves
  2. They already have an easy way to import a project from GitHub. The only thing that doesn't currently work is importing milestones: https://gitlab.com/gitlab-org/gitlab-ce/issues/13421
  3. They're working on a unified way to look at issues across repos: https://gitlab.com/gitlab-org/gitlab-ce/issues/2425
  4. The performance issues they mentioned on gitlab.com are now mostly resolved.
Owner

nzakas commented Apr 21, 2016

I want to thank everyone once again for the good discussion here. Since the time we opened this issue, GitHub has made a lot of improvements to issues and we've been better able to handle the volume of new issues we get. As such, we've decided that staying on GitHub is the right choice.

@nzakas nzakas closed this Apr 21, 2016

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.