Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Already on GitHub? Sign in to your account

Investigate switching away from GitHub #5205

Closed
nzakas opened this Issue Feb 10, 2016 · 170 comments

Comments

Projects
None yet
Owner

nzakas commented Feb 10, 2016

As ESLint has continued to grow, we've started to outgrow the GitHub ecosystem. Team members spend hours each day triaging issues, many of which have incomplete information or are otherwise unclear. As such, we spend a lot of time just asking for more information ("what do you mean?", "what version are you using?", etc.). This has become increasingly frustrating for everyone on the team and ultimately takes time away from being able to contribute code to the project.

Additionally, it's nearly impossible to keep up with what are the most important issues and whether or not people are following up on issues. In short, everything that Dear GitHub mentioned is severely hurting the team now. As ESLint's popularity continues to grow and there are more and more issues filed, this is quickly becoming a scalability problem.

The team has discussed investigating alternatives to GitHub to see if we can find a home that is better suited to open source projects with our level of scale. We strongly feel that the code and issue tracker need to live in the same location to make it easier to manage and give people one location to visit for all of their ESLint-related needs (so simply moving to a different issue tracker and keeping the code on GitHub is not an alternative).

Requirements

  • Must host the repo along with the related tools
  • Must be able to run automated tests on pull requests
  • Must allow contributions from anyone
  • Must have a way to setup issue templates prescribing what fields are required
  • Must have ways to organize issues other than labeling (milestones, priorities, etc.)
  • Must import existing GitHub issues and pull requests

Alternatives to Investigate

  1. GitLab
  2. Phabricator

Concerns

  1. Will people find us in another location?
  2. How high is the barrier to entry for new contributors if we're not on GitHub?
  3. What about the Gitter chatroom? What happens to that? Do we need to find another chat location?
  4. Others?
Owner

ilyavolodin commented Feb 10, 2016

Just to add a few more concerns:

  • Integration with other tools. We are currently use Travis, AppVeyor (both fall under need to build after each issue), but we also integrate with Coveralls (not really a deal breaker, but nice to have).
  • Even if people do find us in another location, would they be willing to create an account on the new service just to contribute to one project, or create an issue?

Maybe it's worth checking with other people who maintain large projects on GitHub and signed "dear GitHub" letter to see what their plan is? Coordinated move to the same alternative service would have higher chance of success.

Owner

ilyavolodin commented Feb 10, 2016

One other (pretty major) thing that I forgot is the ability to easily host/deploy site with support for markdown, as well as everything connected to that (plugins for markdown, custom domain name support, etc). I guess we could always go the route of separate website hosting, but it would mean more involved releases, and might be harder to maintain.

Owner

nzakas commented Feb 10, 2016

We could also keep the website hosting on GitHub if that's a big concern.

I'm not opposed to doing separate hosting, either. My personal blog is a Jekyll site that I update automatically. It would be pretty easy to setup the same thing for eslint.org.

@nzakas you can just move issues to Phabricator as babel did, keeping all popular projects in one place is really cool.

As for organizing issues - can't you use ZenHub or alternatives so that repo would stay on Github? (doesn't solve the issue template though, but might be a good start and it's free for OSS)

Personally, I find it much easier to track all popular open-source projects in one place (that's pretty much the only reason I moved from Bitbucket to Github couple of years ago) and ability to participate in any of them through one UI, cross-links between projects etc.

Projects that move elsewhere raise the barrier (again, just a personal feeling / opinion here) to something like what projects on Google Code felt before its recent discontinuation or SourceForge feels like. Note - I'm not talking about UI, as other platforms can surely have features which are more convenient for maintainers, but rather talking specifically about the existing ecosystem and low barrier for participating in new projects within it as for newcomers as well for regular OSS contributors.

Contributor

captbaritone commented Feb 11, 2016

I would like to echo @RReverser's concern about keeping the barrier to participation low. Participation comes not just in the form of commits and filing issues, but also staying abreast of what those issues are, and how they are being handled. I have an app on my phone that allows me to follow GitHub repos. I see all the PRs, all the issues, and all the responses. I see new commits and tagged releases. I'm sure there are many other people who are passively involved in the project via this or similar systems (including the builtin GitHub email notifications). The barrier to entry can be as low as clicking the "Watch" button on the GitHub page.

In another ecosystem, I'm guessing my best bet would email email alerts, and I have no interest in flooding my inbox with that number of emails.

Is it possible that we could take some work of the maintainers' plates another way? Maybe we could have some lower-lever volunteers who can handle the more mundane tasks?

I would really hate to see this project leave GitHub.

Contributor

afahim commented Feb 11, 2016

As a fairly nascent developer, I think one of the things that I / people with my experience level care about is establishing credibility / getting credit for open source contributions, and whether we like it or not, github has become the first place to look whenever someone is trying to judge someone's contributions. It shows contribution activity loud and clear on profile pages, and this type of attribution is really helpful / important for new contributors to prove their mettle in the open source world. I feel that not having this kind of attribution would not provide enough motivation / demoralize a lot of people who would want to contribute, and I think this kind of lack of motivation and enthusiasm can be to the detriment of the project in the long term.

Given those considetations, I would really hate to see this project leave Github.

Owner

mysticatea commented Feb 11, 2016

Honestly, I love GitHub. I hope to not leave from here.

Owner

alberto commented Feb 11, 2016

👎 to moving away, unless there are very clear benefits.

What is making us move? Is it some or all of the points of the "Dear Github" open letter? Anything else? Can we solve it without moving away?

Contributor

ljharb commented Feb 11, 2016

Not being on Github is a huge barrier to entry. Babel's move to Phabricator was perhaps a clear win for the maintainers, but I believe it's a huge downside for everyone else.

While it's important to make sure maintainers are able to sustainably maintain a project, it would be a very sad thing if users had to bear a burden because maintainers wanted to lighten theirs.

I know the issues on Github are terrible. But the benefits of the Github community are great. Please keep in mind there is a barrier for new users - everybody is familiar with Github which is not the case for Gitlab nor Phabricator.

What about bitbucket?

Bitbucket seems to have a lot of downtime, I would suggest following their Twitter account to check it's not an issue for you

Member

BYK commented Feb 11, 2016

As a long-time Phabricator user, it looks very compelling to me but the above-mentioned barrier to entry is quite high, especially for new comers. Right now GitHub even allows submitting a PR purely from the web interface which may help people get started with quick documentation fixes. This is not possible with Phabricator.

Idea: move everything to Phabricator, including bug reporting but keep a GitHub mirror and only allow pull requests. This may address most of the complaints with minimal contributor impact. Since Phabricator supports many third-party logins, including GitHub, that shouldn't be a big deal.

It does support CI integration but may require some extra work to make the actual integration work. I'm willing to help with this if we decide to chose Phabricator.

Member

BYK commented Feb 11, 2016

travis-ci/travis-ci#2143 is the ticket for TravisCI - Phabricator integration. Looks like it won't happen soon unless someone tries to do it themselves. travis-ci/travis-ci#2143 (comment) shows a viable alternative.

I think following Babel example and moving to Phabricator might be a good solution. Perhaps @kittens could share how easy/difficult was to move to Phabricator.

I'd like to reiterate @RReverser's comment about Zenhub. I'm not associated with it, and I don't use it for any serious organization yet, but I surveyed it for use in our company and it solves a lot of the problems brought up in "dear github".

Note that one thing that it doesn't "solve" is that, since it's used primarily as a browser extension, viewers of your repo who don't have it installed won't be able to use any of the sweet features. For example, a user creating a new issue won't be presented with any configurable fields (e.g. "what did you expect to happen/what happened").

Still, you should give it a look and see if it might fit your flow to reduce "paperwork" costs.

Owner

nzakas commented Feb 11, 2016

I hear everyone's concerns about moving off of GitHub, it's not something I want to do. There's a clear barrier to participation elsewhere.

However, what we are doing right now is unsustainable. It's not just that the GitHub experience is suboptimal, it's that it aggressively undermines development. There's just no way we can keep up with the pace of issues anymore, especially when most of them lack enough information to be useful. The tooling isn't good enough for the onslaught we receive.

Really, it comes down to this: something has to change. It seems like GitHub won't change, or even respond to these concerns, so we have to do something.

Another option is that ESLint users step up and help us manage issues. I made that request in #5083 and got zero responses. If you're one of the people who commented about not wanting us to leave GitHub, are you willing to help out to make dealing with GitHub easier on the team?

Regarding ZenHub - looks interesting, though it doesn't solve our worst problem, which is issues filed without enough information. Also, I'd be looking at $50/month for the whole team, and more if we add more people to the team. It's not really economically feasible without sponsorship.

Owner

nzakas commented Feb 11, 2016

Oops, looks like ZenHub is free for open source: https://www.zenhub.io/open-source

Zenhub is free for OSS

Edit: You beat me to it @nzakas lol

Have you considered a bot that would triage issues for you? Have it do some checks on the issue and respond with pre-configured responses and apply labels based on that? (edit: nevermind I just noticed eslintbot)

I highly support the dear-github movement, but it's a catch 22. Leaving GitHub and staying with GitHub both bring their own pros and cons. Unfortunately, all the cons of leaving outweighs the pros in my eyes.

Owner

ilyavolodin commented Feb 11, 2016

@joshmanders We already have a bot that does basic request for additional information. Most of the people simply ignore it:-(

Contributor

MoOx commented Feb 11, 2016

Maybe working on the bot is not a bad idea. You could even auto close an issue if nobody responded to the bot request?

@ilyavolodin Have the bot pass through again after a time period and close the issue with a message.

Member

BYK commented Feb 11, 2016

Working on the bot also sounds like a good idea. It would automatically close the issue if certain values cannot be matched in the original issue message such as ESLint version etc.

@MoOx I have plans of creating a bot that does a lot of triaging and that for issues and help maintainers, unfortunately I haven't gotten farther than registering an account for it and start planning, but my goal is to create a bot that does a lot of offloading and providing it to communities to either host themselves, or remotely hosted.

I just wish I had more time to dedicate to it, because I know a lot of people are feeling the pains and could utilize it heavily and most people have their own bots that do a few things and that's it.

Owner

ilyavolodin commented Feb 11, 2016

We could beef up our bot to be much smarter, true. But in all honesty, in that case, we are going to be spending time developing bot's code, not ESLint:-( The other problem is that we already get complains from people thinking that our requirements are too strict. Having their issues auto-closed will only make it worse. I know that's not a good argument, but never the less, that's probably what's going to happen. And it does not really help team's productivity to see tweets saying how horrible and unaccommodating ESLint's team is.

Contributor

MoOx commented Feb 11, 2016

Bot can send sweet and kind request multiples times with different "degree" warnings. Automating what can be made is always a good idea. For another organization we are going to work on a small bot. I am sure this initiative could be shared for a lot of organization.

Have you guys considered how the Ionic team does things too? Seems helpful. http://blog.ionic.io/how-ionic-uses-github-better/

I wonder how other (even bigger) projects managed to stay productive on Github. Perhaps it would be worth getting in touch with core maintainers of bootstrap/node/jquery/etc to see how they managed to overcome the issues you mention above?

ariya commented Feb 11, 2016

@ilyavolodin Users don't need to create a GitLab account to e.g. file an issue. They can sign in using their Github credential (done via OAuth).

image

Contributor

ljharb commented Feb 11, 2016

Phabricator allows the same thing, but "sign up with Github", while a prerequisite for usability, does not eliminate the friction of having to switch sites.

One other big problem is the auto-linking of issues/commits/PRs that Github allows that would simply not be possible if eslint was hosted elsewhere. It's incredibly helpful to be able to visually see on an issue/PR of mine that a linked issue is closed, or a linked PR is merged.

Team members spend hours each day triaging issues, many of which have incomplete information or are otherwise unclear. As such, we spend a lot of time just asking for more information ("what do you mean?", "what version are you using?", etc.).

This has nothing to do with GitHub, but I see your point about lack of action/response from them.
ZenHub is great for sure as well. In fact, I keep a repo just for issues about GitHub, with a ZenHub-able label. Almost everything I want falls in there.

https://github.com/kevinSuttle/github/labels/Zenhub-able

Member

hzoo commented Feb 11, 2016

I think the familiarity of github is super important, but it's obvious that with a large project github issues just can't scale.

Since Babel/Phabricator was mentioned, I'll say that we could of handled the move to it better and are probably losing some contributors by doing so. It's really nice to have things like issue templates and all that it provides (there's probably a lot that is yet to be explored). It definitely looks like the quality of issues is much higher (a lot of useful information is provided upfront and there's less back and forth) and easier to debug/followup on, but yeah still a social/people issue where we want users to be welcomed, feel like they are helping, etc.

It's easier to manage, people aren't complaining about the switch as much from what I can tell, but we aren't getting like more PRs or same contributors back so I don't think it can solve everything just switching? We do have instructions in the readme, our contributing.md was telling people not to do support in the issues, and there's links on the website, but sometimes there's still issues that are created about using babel that are created on the website repo since they can't find the issues on github, or people will say things are broken on twitter or start emailing us.

It's a hard problem, and it can be a struggle to think about the "wants" of the maintainers while keeping a low barrier of entry to contribute (in any way). Users are still who we're supposed to be serving and helping? There's probably a lot that can be improved on our side that we aren't thinking about - maybe we should focus on making it easier to contribute and giving others responsibility.

Don't know if that was useful at all.

thasmo commented Feb 11, 2016

It was mentioned before; what about https://github.com/kentcdodds/issue-template as a first step or so.

Owner

nzakas commented Feb 11, 2016

As far as I can tell, here's what would likely happen with any move (regardless of location):

Pros

  1. Much easier to triage issues by enforcing or at least suggesting proper information to include.
  2. Easier to track and manage ongoing issues.
  3. Better tools for managing code and releases.
  4. More time spent on code, less time spent on issues.

Cons

  1. Lose visibility/discoverability.
  2. Lose casual contributors (likely forever).
  3. Lose some bug reports because people don't want to sign up for Yet Another Account.
  4. Lose some feature requests for the same reason.
  5. Less friendly to beginners.
  6. Lose ecosystem compatibility (Travis? Coveralls? BountySource? Bots?)

Analysis

What the community ultimately wants is for ESLint to continue to grow and be developed. What the team wants is to be able to continue to grow and develop ESLint. The community believes that this can happen while staying on GitHub, the team does not. So there's got to be something that gives.

If we remain on GitHub for, let's say, the next year, it's likely that the project will stagnate under the weight of feedback in the form of issues. Everyone on the team is doing this in their spare time, and as the project starts requesting more and more of that free time, we risk losing team members, which would cause the project to further stagnate. If we got more team members, that would temporarily relieve the pressure, but people tend to come and go so there's no guarantee of anything. In all likelihood, we'll have to start ignoring issues just to make progress on actual code, and that would result in community disappointment.

If we move off GitHub, we will immediately eliminate the random noise we get through issues. Increasing the barrier to file an issue slightly, both by requiring another login and by having issue templates, decreases the amount of issues we'll have to triage. It's possible we'll miss an important bug because of this, but I have to believe that if something is really problematic, someone is going to eventually break through and file the issue. At the same time, we'll eliminate drive-by pull requests that also need reviewing. We'll likely be able to hold onto all the team members because the workload will remain manageable. We'll also get fewer new contributors because they'll be less familiar with the system, which means the team growth will be slowed. However, all of this means a much more stable and predictable path forward for ESLint, which means more time to work on the code and less fussing over issue management.

Owner

nzakas commented Feb 11, 2016

@thasmo that only works if you can funnel people through a different link to file issues, which is virtually impossible. We've tried using different "Report bug" links and such, and no one ever clicks on them.

How about recruiting people to the team who's sole job is to just triage issues? I mean saying "Sure help out, tackle those issues" isn't as inviting, but saying "Anyone who is part of this group of people, will be official team members, just like core contributors, and will be recognized as an important part of the team." could be a little more inviting.

@joshmanders it's 2016

@knowbody whatever that's supposed to mean.

That we have computers to solve those problems.

Well clearly it's not helping. I'm just throwing ideas out there. The biggest issue seems to be triaging issues. Leaving Github I feel will hinder the project more than help it.

Owner

nzakas commented Feb 11, 2016

@joshmanders what is that opinion based on? How would it hinder the project?

Member

platinumazure commented Feb 11, 2016

@nzakas Let's suppose (for discussion's sake) that ESLint decided to stay on GitHub, or at least was open to it iff folks in the community could help with triaging issues. What could people in the community do to help make that easier for you?

I didn't mean to be rude sorry. I meant that we can either build something or use existing tools. Can't imagine having ppl to just dig through issues.

Contributor

ljharb commented Feb 11, 2016

I think it's unfortunate that "working on the code" is considered more important than "issue management". Just like when a company grows, as your responsibility increases, your individual contribution decreases. I think adding more collaborators, specialized in specific skills like issue triage and PR review, is analogous to the proven solution of growing a company by hiring similarly.

kittens commented Feb 11, 2016

@ljharb I'm not sure why you got that impression from this issue because I got the opposite one. Babel moved from GitHub to Phabricator precisely so we could make managing issues more effective and scalable.

@nzakas discoverability is huge, and as easy as people think it'd be to just up and move to say GitLab, that's basically a death sentence for anyone hoping to receive help from the community. Take me for instance, I understand the usages of all those other services, but 99.999% of all of the things I use and help with are on GitHub, 1-3 projects leaving Github isn't going to get me to go "welp, time to check yet another site to help fix issues with the tools I use" no, I'll just find another project that's on github. It's a catch 22. Sure, you'll have less issues triaging issues and that, but that's at the fact that nobody will post any.

I feel developers are too deep into the toolsets to realize that leaving a standard and heavily used by EVERYONE resource like Github is going to make their end users just go "Off to gitlab we go"

kittens commented Feb 11, 2016

Also to the people saying discoverability is going to be halted, when projects reach a certain size user growth is less of an issue than maintaining the current user base. Once you reach critical mass and have something that's useful the location of the code and issues is less important. Most ESLint users will be using the websites documentation and this wont change because the repo shifts from one website to another.

Owner

nzakas commented Feb 11, 2016

@ljharb Unlike with a company, no one is getting paid to do this. It's easy to say, "just get more people," but very hard to do in practice.

@joshmanders ESLint doesn't have a discoverability issue at this point. In the last 30 days, ESLint was downloaded 1.6 million times. Discoverability is a problem for new projects, not established ones.

Member

platinumazure commented Feb 11, 2016

@nzakas Is it possible to add users who have some collaborator permissions but not others? E.g., can label issues, can't push?

@nzakas Sorry discoverability was the wrong word. I mean support. As @kittens says Babel moved to phabircator, I have an account there and commented on one issue, and as heavily as I rely on Babel, I am not compelled to go to Phabricator to read issues or participate in them. It sucks that issues and code are separated. I'm personally just too lazy to go that extra step. I'm certain many are. I could be wrong, just speaking for personal experiences.

I have visited ESLint's site like 3 times, honestly. But most of the time I come to the github repo and read stuff, including code, because you can't count on outside resources being up-to-date.

Owner

nzakas commented Feb 11, 2016

@platinumazure not on GitHub.

@joshmanders what you're describing with Babel is the "casual contributors" I mentioned in my previous comment. Yes, we lose that, but in some ways, that's also what we're looking to filter out. People are no longer just going to happen upon ESLint and randomly decide to open an issue. Instead, they'll measure if their request is important enough to be worth the extra work of opening an issue. That alone will reduce the amount of noise. You may not spend time perusing issues anymore, but the people who are working on the project every day will be.

Contributor

inikulin commented Feb 11, 2016

I found it painful to navigate through and fill issues to Babel after it's move to Phabricator. Maybe it's just me, but I was not able to do some basic stuff in the search (like sorting issues by date). Filling issues was not fun as well: GH markdown is really handy comparing to what is present in Phabricator. The overall impression was that transition made issues management easier for the maintainers, but not for users. And I agree with all other points above: GH is not just a code hosting, but unique social network that enables many collaboration/tracking/automation scenarios which are not possible in the isolated codebase.

Contributor

jmm commented Feb 11, 2016

I can't speak for anyone else, but I've been underwhelmed with the Phabricator experience since Babel issues moved there. Perhaps it has a lot of potential to improve things, but I also don't know how much work that would take or who's interested enough in figuring out Phabricator to do it.

Having a template for new issues is probably a great way to try to encourage higher quality reports, but it doesn't stop people from filling it out with garbage. There's still plenty of low quality reports coming in to the Babel Phabricator. Although not providing repro code is a problem, dumping a bunch of irrelevant code into an issue report is also a problem. Example: I dealt with an issue on the Babel Phabricator recently where the reporter first didn't provide sample input at all (paraphrasing):

If the source is needed I can supply it but it's been happening with multiple files...

Then through back and forth I gradually coaxed them to provide a working repro and then still had to trim their sample input from 620 bytes to 26 to get to a minimal repro without so much noise.

Owner

nzakas commented Feb 11, 2016

@jmm I don't think anyone expects all problems to go away, I do expect, however, that a certain level of noise will be eliminated.

Contributor

jmm commented Feb 11, 2016

@nzakas Maybe it will. I'm not convinced that's been the case with Babel. And meanwhile I find the Phabricator interface confusing and clunky. And since moving there reporters have been able to do crazy stuff like assign issues and set priority level themselves.

@ilyavolodin @nzakas we have Gitlab support for Coveralls nearing completion -- probably a matter of weeks for release!

And since moving there reporters have been able to do crazy stuff like assign issues and set priority level themselves.

@jmm that's largely just because we haven't setup permissions properly.

phated commented Feb 11, 2016

My experience is with babel moving to phabricator is that I now contact the devs directly through other channels because the UI/UX barrier is too high. I would expect the tweets/irc/whatever messages to dramatically increase if you move to phabricator.

@phated I don't see how that's not a good thing. Issues are not for support-like requests, I'd rather have those discussions in other channels.

I think GitLab's interface is definitely superior to Phabricator's, and I think feels a lot more familiar to GitHub users. I don't like that Phabricator has unclear names and icons for everything, makes it a pain to use compared to the Issues/Pull Requests (or Merge Requests)/Wiki vocabulary that's used by GitHub and GitLab .

With regards to GitLab.com's loadtimes, I should mention they're making huge strides in that area. If you're intending to host it yourselves, that won't be as much of a problem.

Contributor

jmm commented Feb 11, 2016

@thejameskyle

@jmm that's largely just because we haven't setup permissions properly.

I understand, but as I mentioned earlier, an important consideration in thinking about that switch is if there are people who can and will take the time to figure out and actually do those kinds of things in Phabricator. Personally I have close to zero interest in spending time developing expertise with it.

Contributor

inikulin commented Feb 11, 2016

@thejameskyle

I don't see how that's not a good thing. Issues are not for support-like requests, I'd rather have those discussions in other channels.

You can get the same workflow on github then, just by closing question issues and keeping only bug reports.

Anecdotal evidence for the difficulty of switching, so it may not be worth all that much, but: I've started working on GitLab in my spare time over the past two weeks or so. There was a bit of an initial confusion with how the page was supposed to be navigated, but I could get started using it pretty quickly. When the Wikimedia Foundation switched to Phabricator it took a lot longer for me to become comfortable with it, and even then it was difficult to use.

Again, anecdotal evidence.

You can get the same workflow on github then, just by closing question issues and keeping only bug reports.

We did do that, and it was a massive annoyance which lead us to leave GitHub

Happy to provide help or answer questions as needed. We're in #phabricator on freenode (clittle or epriestley).

egtann commented Feb 11, 2016

It's worth considering that perhaps you don't need a web interface (Github, GitLab, etc.) at all. Git has all the tools to support distributed development built in, so a simple server with a --bare repo should suffice.

Google Groups or any other email group service would enable everyone to use whatever POP/IMAP client they choose (whether that's Gmail, Outlook, Mutt, etc), and thus sort/filter/search issues, patches, and discussions according to their own preferences. You could simply accept and sign-off on patches submitted via Git to the email group, which has the bonus side-effect of effortlessly keeping a clean history.

Linux has been doing this successfully for a while. No reason it can't be applied here.

Contributor

inikulin commented Feb 11, 2016

Google Groups or any other email group service would enable everyone to use whatever POP/IMAP client they choose (whether that's Gmail, Outlook, Mutt, etc), and sort/filter/search issues, patches, and discussions according to their own preferences.

It doesn't scale from the user perspective. It would be good if eslint is the only project you are interested in/working on.

egtann commented Feb 11, 2016

It doesn't scale from the user perspective. It would be good if eslint is the only project you are interested in/working on.

Of course it does! In fact it scales quite beautifully! :)

I'll use Gmail as an example, since that's probably the most common use case here. Simply have the message board forward you emails with the format of youremail+label@gmail.com and Gmail will automatically label the incoming messages. In this case, you could easily set up an ESLint label alongside, e.g. a Linux label, and then search by label.

Most of us already have our email clients open (or at least get push notifications), so all discussions, issues, and patches are sent to contributors (with labels) in a workflow that nearly every contributor here already understands and uses every day.

tubbo commented Feb 11, 2016

I'm a big fan of what MongoDB has done with their repos, consolidating issue tracking to https://jira.mongodb.org/ but keeping code hosting, pull requests, etc. available on GitHub. GitHub is a great code hosting platform, and provides rudimentary issue tracking capabilities, but JIRA is a much more complete means of tracking tickets.

Just wanted to respond to @afahim: At wikimedia, we have our own code hosting setup, but we replicate all repositories to github (@wikimedia). This takes care of the 'but people look in one place only!' issue

Contributor

inikulin commented Feb 11, 2016

I'll use Gmail as an example, since that's probably the most common use case here. Simply have the message board forward you emails with the format of youremail+label@gmail.com and Gmail will automatically label the incoming messages. In this case, you could easily set up an ESLint label alongside, e.g. a Linux label, and then search by label.

I'm trying my best to figure out how it's easier than what we have currently.

ariya commented Feb 11, 2016

Lose some bug reports because people don't want to sign up for Yet Another Account.

Will this be the case since both GitLab and Phabricator allows a sign-in using the user existing's Github/Twitter/Google account?

I do however agree with the (minor) "site switching" friction as pointed out by @ljharb.

phated commented Feb 11, 2016

@thejameskyle I reached out to you like a month ago on twitter about a potential issue, which I was able to track down and work around, but I still think it is an issue and haven't filed it due to phabricator. That's a major problem.

Will this be the case since both GitLab and Phabricator allows a sign-in using the user existing's Github/Twitter/Google account?

Phabricator supports multiple auth sources. You can login with GitHub credentials.

Phabricator supports multiple auth sources. You can login with GitHub credentials.

You still have the issue of jumping around between sites.

Owner

nzakas commented Feb 11, 2016

I don't see "site switching" as a very high cost, especially if you can login with GitHub credentials. On a daily basis, I "site switch" numerous times a day between Twitter, Gitter, GitHub, Gmail, Google Docs (literally have these all open right now), sometimes talking about the same things.

+1 for gitlab

You still have the issue of jumping around between sites.

We have "Nuance" on our roadmap for 2016, an "at the source" triage tool that will pull reports in from any source you choose, so people file bugs via Twitter, Stack Overflow, GitHub, or Twitch.

Contributor

inikulin commented Feb 11, 2016

@nzakas But you don't get notified about PR/issue status on Twitter and don't read tweets in Google Docs. Each site has it's focus and purpose.

dosire commented Feb 11, 2016

For more information on how GitLab addresses issues encountered by large open source projects please see https://about.gitlab.com/2016/01/15/making-gitlab-better-for-large-open-source-projects/

For all your CI, code quality and deployment needs please have a look at https://about.gitlab.com/gitlab-ci/

We're very open to requests to add additional functionality, see https://about.gitlab.com/direction/ for our current plans.

psion commented Feb 11, 2016

I've started using BitBucket recently for myself and organization. As a small organization, I find that having to pay for one or two private repos a bit hard since I don't make that kind of revenue just yet. Being on BitBucket lets me keep my team small and have private repos. Plus, integration of issues with Jira is really nice.

Okay so we've identified that issues are a problem. Drive by issue creation that don't adhere to any specific format with helpful information. We've suggested the use of bots to triage as much as possible to handle these, and were retorted that they don't pay attention... So my question here is, if switching to gitlab is an idea, they currently only offers issue templates, if you set a template via new_issue.md like they suggest, what is stopping the user from selecting the template contents hitting delete, and doing the same thing they do with an empty issue form?

That leaves the barrier of switching to another service as the sole reason to alleviate the pressure of having so many issues coming in and having to spend development time triaging them... Is that really a good solution? I mean, yeah you can put a waist high wall directly in front of the doors of your building to make the barrier of entry more complicated for drive by users. But is that really the message you want to send to your users?

I offered the solution of having dedicated issue triaging users on the official team, there's tons of people always talking about getting started in open source and this sounds like something right up the alley of this. Let newer users who want to get started in open source triage your issues. So your argument here is not having anyone to do it... On the Hacker News thread, one specific user even said he has students always looking for ways to get in contributing to OSS, so instead of waiting for people to start doing it, why not start a campaign to get users on board? Give them read access to repositories, which then gives them the collaborator tag in the org and allows them to assign labels to issues.

I honestly 100% believe that leaving GitHub is not the solution to this problem.

Trevoke commented Feb 11, 2016

I'm with @egtann ... If you have reached such size that this is a problem, consider the model that the Linux kernel uses.

xorgy commented Feb 11, 2016

You could use the GitHub API to automatically close or delete issues which do not include required information. Maybe.

I would say GitLab is your best bet as far as replacements go. A further benefit is that if something doesn't currently work as you would expect, you could conceivably add the feature to GitLab CE. GitLab CE is continuously deployed as the public site. Assuming the desired features have general appeal, they will be gladly merged.

For what it's worth, I think a move to GitLab would be fine. It looks pretty nice, you can login with GitHub creds, and I like that they responded to the Dear GitHub and seem to have a lot of it already covered.

I wouldn't worry too much about visibility for regular users; type "eslint" into Google and the top two results are the website. It could do better, as this issue points out, but essentially it has all the info you need as a user.

In terms of filing issues, the user base is now so large I don't think you're going to miss a real bug even if 4/5 of people who would have gone to file it give up, either because they won't sign in to GitLab or because they won't fill out a proper issue report.

The website itself could stay on GitHub, although I do fear if that was the only repo under the org you might just get lots of people mistakenly filing eslint issues against it.

Ultimately, if you move to GitLab tomorrow and then in three months think "that was a mistake" for whatever reason, you could just move back to GitHub, right? So, might as well give it a shot before team members start dropping off.

  1. Stay on GitHub. Keep the barrier to entry for participating in the project low.
  2. Consider disabling issues on the github repo(s) and using a separate service to track issues. Have clear documentation about where and how to create issues.

For what it's worth, I do a lot of work with Drupal. The issue queues on Drupal.org are very complete in terms of the metadata that's tracked with each issue (version, subsystem, severity, etc). However, you would not believe how much time and energy people spend bickering about the "right" metadata for an issue. It seems like a very nice thing to have a lot of extra metadata like that, but speaking as someone who has experienced it first hand for the better part of a decade, I wouldn't recommend it.

That said, if you really want to do it, I'd recommend putting an issue tracker up somewhere, and leave the code hosting + pull requests in place on Github. PRs are wonderful things, and it lowers the barrier to entry for new contributors to give something to the project (since they don't have to learn a new tool/process specifically for your project). You could even set up a bot to watch for new issues on the Github repo, and when a new one is created, have the bot create a corresponding issue on your new issue tracker, and close the GH issue with a message thanking the user for their bug report/feature request/whatever and directing them to the real issue tracker (which they can hopefully log in to with their Github account via oauth).

Alternatively, you can get semi-structured data for your issues by auto-closing anything opened through the Github UI, and directing users to a form on eslint.org. When a user fills out this form, they will be required to select a version and a subsystem (or whatever other data you need to collect), and fill out a specific template that will help you reproduce the problem or otherwise understand what they're saying. When they click save, a bot will create an issue for them on GH issues with a label for version + a label for subsystem (and whatever else you want to track). If they don't give you the info you need, they can't submit the issue.

This seems nice in theory, but consider that not all users will have the know-how to properly describe the issue or how to consistently reproduce it. If you require this information, you may be left unaware of bugs that someone else might be able to solve.

In addition, this leaves you yet-another-thing to deal with as a project. Who will be responsible for keeping the bot running? Or the issue tracker that you decide to use? Would that time be better spent working directly on eslint? Or simply triaging issues on github?

On the question of bots, Google uses Github for Kubernetes (a project with 25k commits), and I know that they had a lot of complaints with the way Github handles issues and code reviews. They have built a bot to handle their issue flow:

https://github.com/kubernetes/contrib/tree/00f87a5fc4e01f5479c0b90f0cb5596d2090453c/mungegithub

This might be overly specific to their usecase, but maybe there's something you can port or commonize.

dosire commented Feb 11, 2016

@nickmerwin good to hear that Coveralls support fot GitLab is coming soon as you said in #5205 (comment)

Have you guys seen this tool from @fat ? https://github.com/fat/haunt

Haunt helps you keep your github issues under control. It does this by allowing you to run unit tests against github issues and pull-requests, then make contextual decisions about closing, sorting, tagging, and commenting.

Hi,
I am part of the rails committer team, and we usually face similar problems. Specially around people opening issues without enough info to reproduce them, or issues that were already fixed and so on. We have a big amount of issues coming in every day. I would like to tell you how we solve those problems, and if you are interested I might be able to help out you to integrate them in here.

  • First, we have an "issues team", which are people that are willing to help the project, and also had done a few contributions to the code base too. Pretty much the Issues team is the first entrance point to the Rails team. People on the issues team can merge doc changes, and close issues. So they usually are the ones that reply first on the issues asking for more info, and if they fell like they should close the issue thats what they do. I see above that you already tried to recruited people for doing that.
  • https://github.com/rails/rails-bot : We have a rails-bot that posts to every issue/pr to make sure people read the contributions guide and provide the needed information.
  • https://github.com/rafaelfranca/rails-tools/blob/master/bin/close_issues : is a script that @rafaelfranca runs on his machine to mark as stale inactive issues, and close the stale ones.

I've got a say that this is not a perfect setup, but it helps a lot, and allow us to focus on a few issues.
I hope that helps.

Nejuf commented Feb 11, 2016

If issue organization and management are the main problems, Waffle.io may be what you're looking for. Sorry, I haven't used it in project, so I can't provide much feedback on it.

It'd be nice to ask to someone at docker team how the handle these situations. I believe they're one of the largest and most active projects on Github.

This is an OSS project run by volunteers who have real jobs, real families, and limited time. My thoughts on this matter is to push back crappy issues onto the users.

Here is how to manage this:

  1. Create a set of requirements for new issues. This is hard to enforce on Github where there is not a form (with required fields) that you can create, but can be documented well enough using a wiki page on the project's Github wiki.
  2. When an issue is incredibly weak comment as such and link to that shiny new wiki page. Slap a label on the issue Requesting Additional Information. You can filter the default issue list to exclude issues with specific labels.
  3. Once a month review the list of open issues with that label and close any issues that have not been updated or reengaged in more than 2 months.

A good idea for a wiki issue reporting page is something that can be copy and pasted, so that all the desired markdown formatting is already in place and the user just has to fill in the blanks with their required details. Here are some details to consider:

  • ESLint version number
  • OS
  • Environment: browser, node, other
  • Environment version
  • Additionally required rules/components/modules/dependencies
  • Steps to reproduce
  • Error message text

I know that when somebody reports a defect you want to jump on it immediately. Each reported defect brings a sense of despair that some critical failure is spawning in production. There is nothing worse than knowing your project is somehow defecting and not having the details to reproduce the problem, but you don't have the time or resources to read minds and guess at obscure breadcrumbs. Or worse still is when you spend time investigating a reported defect only to discover critical details are missing after you are knee-deep in the code and 30 minutes has gone by.

One thing that also really helps is an automated config/stacktrace dump. A feature that when run produces a self-diagnostic. Atom-Beautify has this, here is an example: https://gist.github.com/carter-lavering/2cf0611a47d03edfea11

Contributor

jmm commented Feb 11, 2016

@prettydiff

Create a set of requirements for new issues. This is hard to enforce on Github where there is not a form (with required fields) that you can create, but can be documented well enough using a wiki page on the project's Github wiki.

When an issue is incredibly weak comment as such and link to that shiny new wiki page. Slap a label on the issue Requesting Additional Information

I'm 👍 to these ideas generally and have (previously) implemented them with Browserify (which needs to institute a policy of no support requests in the issue tracker, but that's another story). I wish it were easier to link to a wiki page with some kind of shorthand and have it automatically populate the URL and title. Another approach is to just close low quality issues and say to open a new issue with more information if it's still a problem. Most of those people never come back. Sometimes the other maintainers on the project feel like being more indulgent though and will re-open the issue.

A good idea for a wiki issue reporting page is something that can be copy and pasted, so that all the desired markdown formatting is already in place and the user just has to fill in the blanks with their required details.

That's not a bad idea, that I never thought of.

Contributor

jmm commented Feb 11, 2016

@prettydiff

Create a set of requirements for new issues. This is hard to enforce on Github where there is not a form (with required fields) that you can create, but can be documented well enough using a wiki page on the project's Github wiki.

When an issue is incredibly weak comment as such and link to that shiny new wiki page. Slap a label on the issue Requesting Additional Information

I'm 👍 to these ideas generally and have (previously) implemented them with Browserify (which needs to institute a policy of no support requests in the issue tracker, but that's another story). I wish it were easier to link to a wiki page with some kind of shorthand and have it automatically populate the URL and title. Another approach is to just close low quality issues and say to open a new issue with more information if it's still a problem. Most of those people never come back. Sometimes the other maintainers on the project feel like being more indulgent though and won't close issues like that, or even re-open an issue someone else has closed for that reason.

A good idea for a wiki issue reporting page is something that can be copy and pasted, so that all the desired markdown formatting is already in place and the user just has to fill in the blanks with their required details.

That's not a bad idea, that I never thought of.

pygy commented Feb 11, 2016

Life is better with Rust's community automation <== a talk about the Rust github bots, and how it improves their morale (skipping the beginning, which is not related to the issue at hand).

It would help if @eslintbot was more anthropomorphic.

Also, this HN comment is insighful:

Seems like the issue is... issues. Some projects get so popular that it's a full time job just to respond to issues that lack information.

I've been in software for a long time. There's no tool that solves this problem. People who don't know your software don't know how to report issues. They don't know the keywords or nomenclature to search for pre-existing error messages either.

Switching to another tool won't solve this problem they're having. Typical engineers - they think issue templates will help. No. I'll just fill in whatever gets me past the validation cos I'm frustrated.

This might be unpopular, but you're under no obligation to answer those issues or even have a public issue tracker at all. Turn the issue tracker off if it's frustrating, and use a private issue tracker for the core team. Or find some students that want to break into open source and have them be 1st level triage for your issues. It's a people problem, not a technology problem.

Too many developers forget that software development, even open-source, is 20% code and 80% people. Documentation and support are a huge part of software. And software only goes so far to solving that in my experience.

ESLint user here who has contributed/owns a few popular projects. Please don't move to Phabricator. It was one of the worst bug reporting experiences I've encountered other than default old RedMine.

After two crappy experiences with babel's phabricator I simply won't be submitting bugs to them, it's not worth the effort. I'd rather fork and write a hack to fix the bug locally.

xixixao commented Feb 11, 2016

It seems to me that the main problem are issues with bad quality. So I suggest to close them + label them as missing information. That should be a single keyboard shortcut. It seems that you don't want to do this (or just ignore the issues) because you don't want to piss off the community. But moving somewhere else is probably going to piss off / shut off the same people.

If the problem is that it takes too long to figure out that an issue is missing crucial information then your problem is purely of scale of the contributor team and moving elsewhere will just offset the problem for a little while.

Contributor

jmm commented Feb 11, 2016

Re: the wiki idea with instructions on issue reporting...another idea I forgot to mention is to create a unique label for that situation and when you run into one of those issues, just apply the label and do nothing else (or maybe close), and have a bot that posts a generic reply to those issues that explains why it was closed and has a link to the "how to report" issues instructions (and closes if you want to bot to do that instead of even doing that manually).

jlukic commented Feb 11, 2016

For large projects the amount of issues that aren't correctly filed can end up being the majority of the projects issues. I'm currently sitting at around 800 unread notifications that have been plaguing me.

As maintainers time ebbs and flows the ability to keep issues filed at "net zero" becomes more and more difficult, especially when most new issue triaging is really just informing people of incorrectly filed issues.

It's hard not to commiserate with users because GitHub doesn't provide dedicated channels for community managed support, so most people look for it through GitHub issues which they can count on to be actively watched by maintainers.

Owner

nzakas commented Feb 11, 2016

An update: I had a nice conversation with one of the folks at GitHub. It sounds like they do understand our concerns and have some plans to address them. Nothing set in stone as of yet, but it does seem that issue templates are on the roadmap. I've opened a dialogue with them that will continue to see if we can get to a resolution that will allow ESLint to stay on GitHub.

@joshmanders FWIW, all other issues aside, it's not realistic to have the least-in-the-know contributors in charge of deciding what issues are addressed by the core maintainers.

Owner

nzakas commented Feb 11, 2016

@arthurnn thanks for the info. I've tried to rangle an issues team but haven't had any success. Do you have any insights as to how to build such a team? (I'll take a look at the other links as well, thanks!)

@prettydiff those are some good suggestions, and we can look into that. The only downside is that it still requires humans to start out the process, and that's what I'm trying to minimize.

@jmm that's a pretty good idea. I think our bot can probably do that pretty easily.

@pygy thanks for the link, that's very helpful!

@nzakas I'm glad to hear they have been responsive. I think doubling down on this service and helping them iterate through the growing pains is the best possible course for all concerned, though of course it's predicated on their willingness to get a roadmap in place.

Contributor

glenjamin commented Feb 11, 2016

One thing I think might be useful in this disucssion is to talk about the types of issues that are currently being received, and what should be happening to them.

eg.
Questions -> ideally on stackoverflow?
Bugs -> needs enough info to repro
New rules -> push out to plugins?
New features -> attempt to avoid?
Support for new JS language features -> probably need? Maybe push out to 3rd party parsers?

One potential avenue to reduce the number of incoming issues is to reduce the project scope by recommending moving more stuff into eslint plugins - the plugin architecture is really solid these days, and having more stuff in userspace building on this I would hope would reduce the burden on core maintainers.

@colinmegill Nothing to do with being in the know and all to do with a few select users who's whole job is to just triage issues and respond to invalid issues, close them, add tags, etc. Obviously if they have access to do so, they will be somewhat-in-the-know on how to handle things. Not like I'm suggesting randomly giving some github user access to close and tag issues.

GitHub isn't all or nothing. Why stop using their git hosting when their issue tracker is where your problems are? Just find a better issue tracker and keep your source code here. There are plenty of good issue trackers with GitHub integration.

Contributor

jmm commented Feb 11, 2016

@nzakas

@jmm that's a pretty good idea. I think our bot can probably do that pretty easily.

If you get that up and running I'd be interested to see it in action / hear how it's working out.

One thing participants of social networks often find is that metcalfe value massively exceeds tool value

There's a reason you don't use DARCShub

Owner

nzakas commented Feb 12, 2016

@jmm got it working: #5217

Owner

nzakas commented Feb 12, 2016

@rmcardle this was discussed way back at the top. No one wants to switch between one place for code and one for issues. That's a nonstarter for us.

larkin commented Feb 12, 2016

What you do is aggressively close issues that are poorly written or unhelpful. If people won't follow the instructions in your CONTRIBUTING.md, then you don't need to give them the light of day. You're not here to hand-hold junior developers-- they need to learn to investigate and get to the root cause of issues. If they do a poor job of bug reporting, and they feel they should get more respect than a closed issue, well, they can write up a new, better issue, and submit that.

Taking a firm stance should cut down on time spent working issues. People who need support or have found a legitimate bug will work hard to demonstrate that-- they will put in the work to write a good issue, and others will not come back.

Expect more of your developers and your community-- expect them to be smarter and hard working. And they will be. For those who won't put in the time to work with you, you don't need to put in the time to work with them.

my 2c.

dcposch commented Feb 12, 2016

Just an observation: if Node itself is happily using Github, then a JS linting tool should def be able to do so as well.

It just requires disciplined organization.

Member

platinumazure commented Feb 12, 2016

Think I'm in agreement with @larkin. There's a difference between having expectations of developers and being "mean". We can't afford to "be nice" to junior developers when it drains so much time and productivity.

kittens commented Feb 12, 2016

@dcposch

Just an observation: if Node itself is happily using Github, then a JS linting tool should def be able to do so as well.

It just requires disciplined organization.

You're forgetting that Node has a very large contributor base relative to other large open source projects and it even has people paid fulltime to work on it.

Member

IanVS commented Feb 12, 2016

It seems React Native had some pain around issue management and prioritization, which drove the introduction of Product Pains. Their bot leaves a comment on GitHub issues asking users to submit bugs and feature requests on Product Pains (example comment).

It would be interesting to hear from someone on the RN team about their experience triaging issues and whether Product Pains has helped to ease their own pains.

JaKXz commented Feb 12, 2016

I don't know if this has been suggested yet (long thread and I'm on my phone): waffle.io?

vjeux commented Feb 12, 2016

@IanVS for React Native, what we've been trying to solve is how to prioritize what the community needs. At this point, there are way too many issues for our team to handle and most of them are very low value and would take far more time and energy to fully figure out than actually working on the project itself.

We've tried a bunch of things to extract actionable things from issues like tagging all of them by area and counting the most issues per area, sorting issues by comments... While they gave some sense of what the community wanted, it was very noisy.

The big push that we've done was to setup Product Pains. The goal is to be able to have a ranked list of what people building react native struggled with the most. It has been very successful, the first page is full of very actionable and impactful content. We are using this in order to influence our roadmap.

At this point, we have pretty much given up on managing issues and aren't closing them anymore. Closing issues is causing a lot of drama because people take it as an attack so it's safer to just leave them open.

The way the team (fb and community core contributors) communicate is via a secret facebook group. This is a space where we can have high signal communication. Having this kind of communication in a public space where everyone can comment quickly turns into big discussions with too many cooks in the kitchen and is ultimately not productive.

Hope that it gives some context. Let me know if you have any more questions, happy to follow up

aloisdg commented Feb 12, 2016

An update: I had a nice conversation with one of the folks at GitHub. It sounds like they do understand our concerns and have some plans to address them.

All problems wont be here if GitHub was a FOSS. New problems will rise and you cant do nothing about it. If you found an issue on Phabricator or GItLab you can report or even fix it. You are confined and limited on GitHub. We cant participate to all of this. It's manage by GitHub, only...

jlukic commented Feb 12, 2016

Thanks @vjeux I find your commentary insightful. I think I've been feeling the same things lately. It's unfortunate for the public/private divide, but it doesn't seem like theirs much other recourse for maintainers.

jasom commented Feb 12, 2016

Note that phabricator allows you to still use github for source-control; phabricator is perfectly happy with the repository being somewhere else.

jmurzy commented Feb 12, 2016

@larkin You beat me to it.

GitLab FTW!

@rrjanbiah rrjanbiah referenced this issue in RestyaPlatform/board Feb 12, 2016

Open

Address few GitHub pain points #241

nothing can beat JIRA here

Marak commented Feb 12, 2016

Well.

It seems your main issue is dealing with a large number of Github Issues.

As someone who has spent a lot of time triaging 1000s of issues across 100s of github projects for the past six years I can offer a small amount of advice no one else seems to be touching on.

If your users are filing too many support issues, that is a direct result of your software and documentation not offering enough meaningful information to the user. You can't blame your users for this.

For example, if I search on your Github README, I can find NO information of how to file a support issue. There are rules for filing support issues? Where you have clearly expressed this to your users? You have a bot which automatically responds to Github Issues? That is reactive support. You need to implement proactive support. I did finally find the rules for submitting a Support Issue, after clicking into the very small "Contributing" link. The thing is, I'm not a contributor. I'm a user with a support issue. I'm not going to click "Contributing". I'm going to immediately file a support issue without reading anything about "Contributing".

You need to establish clear channels for support. I see a one line section at the bottom of your readme titled, "Where to ask for help?". How am I suppose to ask for help on these mediums? I'm confused. No instructions have been given to me as a user so I don't know what to do. I'll probably open a support issue on Github, get an automated response from the bot, and give up.

Another thing you can do is to temporarily ignore any issues which aren't pertinent or immediately pressing. If a user asks a not-so-smart question, you shouldn't stop what you are doing and immediately respond. Let the issue sit for a few days...maybe they will resolve it themselves...maybe another community member ( who is not a core contributor ) will step in and help resolve the issue.

Try to remember you are running an open-source project. Responding to Github Issues is not bringing you revenue, so you should be careful in how you spend your most limited resource: time.

You seem to want to fortify yourself away from the users so you can be shielded from their unbridled contributions and communications. If you feel that is the best way to run your project, that is your choice. Personally, I would rather have the additional signal and not worry too much about filtering out the extra noise.

The way i see it:

  1. Will people find us in another location?
  • No probably not. But you can bet your ass that somebody is going to clone the repo and dump a github one right back.
  1. How high is the barrier to entry for new contributors if we're not on GitHub?
  • High, plus you don't see any/many other projects make the step. Got to facepalm when i saw the headline.
  1. What about the Gitter chatroom? What happens to that? Do we need to find another chat location?
  • Main downside is that it doesn't have a W10M app and is locked to 25 users (if used free).
  1. Others?
  • How about Slack? Seems to be more popular amongst other communities and there are various solutions to get registration in order (just look up on some free Heroku and other solutions to handle registrations)

And instead of just blaming Github and be annoyed about the issues. How about a new structure to work around that? Or still keep the Github for release candidates and normal releases but skip the normal commits and beta's?

Bounga commented Feb 12, 2016

@nzakas Bitbucket seems to fulfill all you requirements. I use it since 2008 and it is very good! They are very concern with their users issues. And even if it's not very important for you, you'll have unlimited private repos. You really should give it a try, just to see what it does.

@ilyavolodin You can login to bitbucket using your github account.

Github is not the only viable solution out there, it's more of a trend. Some well known developers and projects helped a lot Github to become mainstream but there's other players around that does a good job also (maybe a better job in fact).

Contributing won't be harder on another host. Bitbucket for example will allow the same workflow (fork / create a branch, push, send PR). You've a nice interface with button to fork, create PR and so on.

I don't get the point of developers saying: «You need to be on Github or your project will never take off». That's not the host that make the project interesting. It's the project itself, its qualities, its contributors, etc

Contributor

glenjamin commented Feb 12, 2016

@ilyavolodin You can login to bitbucket using your github account.

This is getting "phased out" - https://bitbucket.org/account/signin/?next=/#social-auth

Bounga commented Feb 12, 2016

@glenjamin Oh yes you're right. So use your google account, or just create an account (that's what? 2 or 3 clicks). If you don't have the motivation to create an account, I'm pretty sure you won't have the motivation to create a good PR or a good issue with all the relevant info.

If self-hosting is not out of the game, maybe pagure could be an option.

It's all python, it provides quite a bit of the features github offers and it's entirely FOSS.
On the cons side, it's relatively new/young so there are likely some features that are missing but I am sure upstream would be very happy to see RFE being filled on the tracker :)

petetnt commented Feb 12, 2016

I want to chime in and agree with what @Marak said.

If the biggest issue here is low quality issues, why on earth is the bug reporting guidelines stashed away under Contributing -> Bug Reporting -> Actual bug reporting guidelines page. Why not on the README.md itself right under the Usage and Configuration sections.

It might not be the prettiest place for it, but it seems to bring much more trouble to the eslint repo itself than say people having trouble seeing what the sponsors or the team behind it are. Not to say that information is invaluable.

kittens commented Feb 12, 2016

fwiw for Babel we had literally a 3 line README.mda with the one notable in bold "don't post support questions here", we even had a massive H1 heading in bold in CONTRIBUTING.md saying not to do this. Want to guess what happened? Nobody read it and people opened issues anyway. People don't read the README.md, even if it's super tiny and people certainly don't read CONTRIBUTING.md.

GitHub has encouraged a culture where people just create issues willy nilly. It's super cheap for users to open issues but it's extremely expensive for maintainers to close them.

DjebbZ commented Feb 12, 2016

The Clojure Community led by Rich Hickey decided to use Github for code but use JIRA as an issue tracker. They also require contributors to sign a CLA and to send patches in JIRA issues, not PR. So the barrier of entry is very high, but all contributions/issues are meaningful. I'm not suggesting to do the same, just pointing to another successful OSS project that don't use Github. JIRA is by far a superior issue tracking system.

As @kittens just said,

GitHub has encouraged a culture where people just create issues willy nilly. It's super cheap for users to open issues but it's extremely expensive for maintainers to close them.

So don't use Github issues if that's the problem.

Also, @nzakas and ESLint people, have you already investigated distributed issues tracking system ? They're text-based and committed to the repo. My small investigation tells me that Bugs Everywhere (OSS, written in Python) is the most feature complete, that even comes with a web-based interface. Never used such a system but I find them very interesting in the principle.

My 2c. Hope this helps the discussion !

petetnt commented Feb 12, 2016

@kittens This is just hindsight 20/20 but if you look at how the 3 line page version (one of them, at least) looked at a certain point of time:

https://github.com/babel/babel/blob/f4a3b2f01ae17acddb78b00cee7fb5cbd4a1524f/README.md

Babel is a compiler for writing next generation JavaScript.

For questions and support please visit the gitter room or StackOverflow. The Babel issue tracker is exclusively for bug reports and feature requests.

Issues without instructions to reproduce will be immediately closed.

For documentation and website issues please visit the babel.github.io repo.

While it's short, it's also very vague, especially for the demographic that contribute with bad issues. For example:

  • What is gitter? What is StackOverflow? Why would I want to use those? Why can't I just ask the question here?
  • What's the difference between questions, support, bug reports, feature requests?
  • Issues without instructions to reproduce? Reproduce how?

Obviously some of that tie in to the things in Dear Github-letter. But just having more labels doesn't solve any of them. I think that the Discussions-tab, for example, would be very neat addition but it alone wouldn't solve anything either. If you cannot close an issue by just stating "Hi @ username, this issue doesn't contain enough information for us to proceed" or "Please ask support questions at SO" then it might be an issue with the delivery. Popular projects attract people of all kinds of skillsets and language barriers.

AFAIU, the issue with moving to another platform like Phabricator is that it hampers support/discoverability/increases barrier to entry of contributors/etc; but if in a parallel universe, these problems weren't present, ESLint from that universe would have happily moved away to another issue tracker. Right?

If so... I suggest moving to another issue tracker in this very universe, and patch the aforesaid problems with a bot:

  • keep the issue tracker on GH open
  • Have the bot mirror all new issues on the new platform
  • Have the bot close the GH issue instantly (possibly even lock it to collaborators)
  • Have the bot leave a polite message explaining the situation on the closed issue, with a link to the mirrored issue and request to have all further issue opening, and conversation on the new platform only

This ensures that you guys get to hear everyone's views. People who care enough to follow up will likely follow the links given as well. And you get to enjoy another platform.

agilob commented Feb 12, 2016

Give gitlab a try. I wrote a post on my blog trying to show I like the most about gitlab. I'm in no way affiliated with them.

For what it's worth, GitHub responded to dear-github; dear-github/dear-github#115

Bounga commented Feb 12, 2016

@agilob If Github doesn't work for them, how Gitlab would? I'm using it every day at work and it's more or less a Github clone. The only benefit is it's self-hostable and free.

Contributor

jmm commented Feb 12, 2016

@nzakas

@jmm got it working: #5217

Awesome! Thanks. Is there info anywhere on how your bot is setup?

agilob commented Feb 12, 2016

@agilob If Github doesn't work for them, how Gitlab would? I'm using it every day at work and it's more or less a Github clone. The only benefit is it's self-hostable and free.

@Bounga Because it covers their requirements in 100%

Must have a way to setup issue templates prescribing what fields are required

Supported, configured in repo settings, for both issues and merge requests.

Must have ways to organize issues other than labeling (milestones, priorities, etc.)

All supported by GitLab CE (gitlab.com). Issues also have weights, votes and belong to milestones, have labels and users can vote on issues.

Must import existing GitHub issues and pull requests

Not sure about pull requests, but issues and wiki are supported.

Contributor

MoOx commented Feb 12, 2016

Pull request on GitLab are called "merge request" (which is the appropriate name according to the action) :)

@sarunast already posted.

Some great points on this subject from the folks behind successful open source projects such as Rails and Elixir (@josevalim): "Tips for keeping your Open Source Software issues tracker tidy"

Might help providing some insights.

We struggle with this too @ Basho. We are using JIRA and GH, which is kind of a pain. We have product development, custom development, and project management in JIRA while GH gets most of the bug & community stuff. We have a service that links the two, but the problem is that we are bouncing around, getting double notifications for things and it can create a confusing user experience for users in GH when an issue is updated automatically from JIRA because our PM changed something.

What we want is something like Waffle.IO that has PM capabilities + the ability to create private tasks / projects that don't get pushed to GH + ability to have a project that spans multiple repos. Haven't really found something like it yet.

First off, thank all of you for working on ESLint!

I've worked on a bunch of open source and closed source software projects, using GitHub, Trac, Phabricator, Bugzilla, Mantis, FogBugz, and other issue tracking systems. Yes, none of them are perfect. GitHub has less customizability than, for instance, Bugzilla. I wouldn't be surprised if you do decide GitHub doesn't work for the particular workflow you want. (Are you using free GitHub hosting or have you paid for extra features?)

Bots and automation can help triage bugs, but in my experience there is no complete substitute for thoughtful people using their brains to address each one. Once someone gets the hang of it they can move pretty fast, and yes, this absolutely is an opportunity for new contributors. The Wikimedia "how to triage document" is a useful model here -- if you can add stuff like that to the "working on issues" HOWTO it'll help new people start as bug triagers. Wikimedia trained an intern as a bug triager and the Outreachy internships are a good match for this kind of work. Someone who starts by triaging bugs is often able to help a lot with release management, prioritizing features, communication with important upstreams and downstreams, and similar work.

I see some folks earlier in this thread suggesting better bug reporting guidelines and suggesting helping users with templates for required information. That is pretty helpful. I see some other folks using wording that implies that people submitting incomplete bug reports are stupid, lazy, or undeserving of help. That, in my experience, is an approach that leads to a bunch of less optimal outcomes. Every bug reporter is, by definition, running into a problem they don't know how to solve themselves; it's unlikely you're running into them on the best day of their life. I've usually found that a little human kindness and "could you go into more detail on points x, y, and z so we can have a better chance at solving this problem?" leads to bug information that helps developers make much more robust software.

Having a good short set of required data fields is useful. But it pays to be careful in the user experience around that. In my experience, it comes across as hostile to go with the "you didn't add required data, so we're auto-closing this, re-open if you think your new report is good enough" approach. That approach makes users more likely to think of maintainers as aloof and unresponsive; then, when a bug comes up that you really wish you'd known about, they don't bother telling you. A good middle ground is to ask for more data, and wait a few days, and then leave a kind explanatory message when closing the issue.

Hope this helps.

I'm with @Nejuf and @JaKXz to suggest you waffle.io

waffle

For using it on a daily basis, it's pretty ergonomic and allows you to benefit the hosting power of Github without pain (or, at least, less pain).
I haven't tested it with a larger project as yours, but it deserve to be tested IMO.

Owner

nzakas commented Feb 12, 2016

@Marak Thanks for that, these suggestions are very useful. We can definitely make it clearer what is expected and how to request help.

You seem to want to fortify yourself away from the users so you can be shielded from their unbridled contributions and communications.

This is incorrect. We want a system that allows us better control of the contributions and communications. A side effect is that we'd lose some of it, but as I've said previously in this thread, I believe what we lose would be mostly low-level noise.

Owner

nzakas commented Feb 12, 2016

One thing I want to point out to a few people who have said things along the lines of "projects much bigger than you seem to be doing okay on GitHub."

Projects much bigger than us tend to have full-time developers and/or much larger teams. Node.js has not only a large number of contributors, but paid developers and the support of a foundation. npm has a company backing it. React/ReactNative have paid developers. The list goes on.

ESLint is run by volunteers working nights and weekends (and, to be fair, we also sneak in during the day to do some stuff when we probably should be focusing on our full-time jobs). The number of people who consistently make contributions is fairly low (< 10) and the number of people who actively help triage issues is even lower.

As I've mentioned earlier, part of the solution could be for more people to contribute. I've publicly requested help for issue triaging and had zero responses. Running an open source project doesn't mean contributors magically appear to help with everything, there's a lot of begging and pleading. (ESLint is certainly not the first project to face this problem.)

So while others have been able to make things work on GitHub, different constraints and resource availability mean that it's not a one-to-one comparison.

mkurz commented Feb 12, 2016

GitHub responded to the "Dear GitHub" open letter: dear-github/dear-github#115
New features are coming to issues soon! Maybe wait for if them and give GitHub a second chance before switching to some other tool?

Saw this on HN today: http://gitmagic.io/

Might be helpful.

aldanor commented Feb 12, 2016

if GitHub indeed rolls out new issue features in the coming weeks, maybe that would be sufficient?

brodock commented Feb 12, 2016

GitLab have "Page Hosting" support like GitHub Pages. There is a WIP Merge Request to have CNAME and SSL supported for next release: https://gitlab.com/gitlab-org/gitlab-ee/merge_requests/173.

You can read more about GitLab Pages here: https://about.gitlab.com/2015/12/22/gitlab-8-3-released/

Owner

nzakas commented Feb 12, 2016

@aldanor yes, it could be enough to keep us around. I want to see what improvements are made before committing to anything, though. In the meantime, we're still going to be exploring other options so we're better informed.

The real question: is this truly a technical issue, or is it a community issue? I.e. are there ways to grow the community by using your governance and docs to draw in new contributors who can do useful work, not just submit incomplete tickets or the like?

Just a thought from the peanut gallery. If you're having this big a discussion on the details of tools that you use for the work, perhaps it's time to re-examine with a fresh eye how the social governance and interaction really works.

(Yes, I know, it doesn't always help, sometimes it's really hard to get new users to become contributors - but that's the only real way open source communities can grow over the long haul.)

Good luck!

Qix- commented Feb 13, 2016

/quote

  • Must host the repo along with the related tools
  • Must be able to run automated tests on pull requests
  • Must allow contributions from anyone
  • Must have a way to setup issue templates prescribing what fields are required
  • Must have ways to organize issues other than labeling (milestones, priorities, etc.)
  • Must import existing GitHub issues and pull requests

/endquote

Seriously? What is on this list that Github can't do? You literally described all of the core features of Github.

What problems are you facing with Github, specifically?

Maybe we should consider BitBucket.

No, you shouldn't. If you think Github has abandoned its product, then you should see Atlassian's complete disregard for its product(s).


This is going to come across as being a jerk. Take it for what it is: direct, and subjective.

ESLint's governance is stiff and rigid. You guys have insanely high bars for what are considered "good" contributions. Maybe this is a good thing, maybe not - either way, your backlog isn't due to any sort of technical shortcoming, but a lack of involvement of collaborators (or maybe the number of collaborators?).

Another issue you're facing is an issue that most large OSS projects face - writing a PR for a feature/bug takes way longer than opening an issue, and most people are not willing to take the time.

I personally speculate that this is because OSS development has been gamified a bit too much. But that's just an opinion.

Switching to another tool is just going to derail this project beyond any sort of meaningful use. You're going to lose a buttload of contributors, watchers, experts, veterans and users alike.

You're also going to force contributors to learn a new product. However symmetrical Github and XYZ product are, you're going to be forcefully introducing human error. Isn't that what you're trying to "fix" in this issue to begin with?


👎 for this. If you do it, you'll lose quite a few users, contributions will diminish, and another, better alternative to your project will emerge.

Until Github itself dies, this is the best place for it to be. Github doesn't care if you leave. You're not sending a message by taking ESLint off of Github.

To me, this is trying to find drama where there is none and completely disregarding the practicality of migrating a project off of an established website considered 'home' since its conception.

That is my opinion.

So GitHub reached out and talked to me after responding to the dear-github letter earlier today. It sounds like the letter did more good than I expected it to, and all I'm going to say is that I'm looking forward to the things that they'll be doing in the next few weeks or so.

@nzakas I would hold off on making a decision for now. And you might want to consider locking this issue because it's turned into an exercise on who can write the most exhaustively long comment stating their uninformed opinion as fact.

@cpaulik cpaulik referenced this issue in syl20bnr/spacemacs Feb 13, 2016

Open

[META] Incremental process improvements #5037

Awesome news, and great work @thejameskyle!

Qix- commented Feb 13, 2016

@thejameskyle that's exciting. Agreed on all points.

Hi,
what about having the bot sending a nice email directly to the bug reporter, asking him to respond the bot with a provided template ? It would then be posted to Github.
It won't be a silver-bullet, but it may be coded quickly and hopefully reduce at least a bit the unactionnable issues.

GitHub has added issue templates silently. dear-github/dear-github#125

All you have to do is add a issue_template.md file to your repo and when a new issue is created it should prefill in.

@joshmanders It doesn't work with mobile access (Chrome on my Nexus 6P) or ForkHub on Android though, but it's definitely a start.

@brianjking indeed, just checked on my iPhone 6, it don't work. But agreed. Great start.

Owner

nzakas commented Feb 17, 2016

@thejameskyle as I mentioned earlier, we are continuing to investigate alternatives. While I'm happy about issue templates, I don't see any reason to stop looking into what other alternatives are out there. I've been having some great conversations with the folks at GitLab about their plans, and I'll post more about that when I have a second.

However, I am locking this issue at this point, not because I see any offensive comments (in fact, I'm thankful for the mature conversation that has take place on this issue), but because I feel like we have all the data from the community that we need. We know that moving off of GitHub would upset some, and that's why we're hopeful we don't have to do that. The team will continue posting updates here as things progress.

I want to especially thank the folks who also run large open source projects for their suggestions and sharing their insights. It's been invaluable to us.

@nzakas nzakas locked and limited conversation to collaborators Feb 17, 2016

Owner

nzakas commented Feb 22, 2016

I just wanted to add some data I got from GitLab:

  1. They'd be willing to give us an Enterprise Edition license so we could run it ourselves
  2. They already have an easy way to import a project from GitHub. The only thing that doesn't currently work is importing milestones: https://gitlab.com/gitlab-org/gitlab-ce/issues/13421
  3. They're working on a unified way to look at issues across repos: https://gitlab.com/gitlab-org/gitlab-ce/issues/2425
  4. The performance issues they mentioned on gitlab.com are now mostly resolved.
Owner

nzakas commented Apr 21, 2016

I want to thank everyone once again for the good discussion here. Since the time we opened this issue, GitHub has made a lot of improvements to issues and we've been better able to handle the volume of new issues we get. As such, we've decided that staying on GitHub is the right choice.

@nzakas nzakas closed this Apr 21, 2016

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.