Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Require analytics opt-in rather than make it a blind default #142

Closed
bcardarella opened this issue Apr 25, 2016 · 87 comments
Closed

Require analytics opt-in rather than make it a blind default #142

bcardarella opened this issue Apr 25, 2016 · 87 comments

Comments

@bcardarella
Copy link

I'm not sure if this was discussed but there are going to be companies and government agencies that have a problem with data being sent out without explicit permission. I understand and appreciate the desire to collect the information but this introduces a problem for some people. Ideally this would have been an opt-in on upgrade and an opt-in on install.

@dunn
Copy link
Contributor

dunn commented Apr 25, 2016

It was discussed, we made it opt-in for about a month during testing then sent out notifications to the mailing list and twitter a week before switching to opt-out.

@albus522
Copy link

I would say the vast majority of your users are neither on your mailing list or pay any attention to your Twitter account. So you have notified a very small percentage of your users that you will be collecting their info. If someone else hadn't pointed out this change, I would never have noticed, and I would wager that is the majority case. Add my vote for reverting this change.

@kroofy
Copy link

kroofy commented Apr 25, 2016

Regardless of how many users that have been made aware of this. The user should still have the option to actively make a decision during an install/update of brew.

This is too sneaky and opaque for the user.

@davesque
Copy link

It kinda blows my mind that this isn't opt-in.

@erikj
Copy link

erikj commented Apr 25, 2016

Seems like it would be best if the interface queried the user and remembered their preference.

@DavidCWGA
Copy link

At the very least, prompt the user before first sending data to Google.

@bfontaine
Copy link
Contributor

@davesque The problem with opt-in is that you don’t get representative data.

@DavidCWGA
Copy link

The advantage of opt-in is that you respect your users' choices and privacy.

@Ianleeclark
Copy link

@kroofy
Incredibly opaque, how dare they link to the two files which directly implement sending off analytics. And sneaky! They've actively managed to keep this information to only two of the largest technical communities: reddit and hackernews--plus their thorough documentation of exactly how it works is not yet thorough enough.

Truly, what could we do to fix these roguish individuals and their wanton disregard for us: the users of a free product who have never contributed to the Homebrew project (with exceptions to dunn and bfontaine, every user above me has never made a contribution, including me). Y'all are ridiculous calm down and just set the envvar.

@jeroenh
Copy link

jeroenh commented Apr 25, 2016

@bfontaine the problem with distributing information is that you can never take it back.

@jasonroelofs
Copy link

@GrappigPanda Insulting people for being concerned doesn't help anyone. Very few users of Homebrew are even going to see this, much less know what's happening or that they are now sending statistics to some random Google Analytics account about what they do on their machine.

There are numerous companies and environments employing tens to hundreds of thousands of people in which security and secrecy is utmost such that any service that sends statistics outside of the walls of the company or group is strictly forbidden. With this kind of change, many people will suddenly start breaking these restrictions with no knowledge of what or why or when.

There's nothing wrong with opt-out, but this should be a message that shows up on brew update that's very explicit, very clear, and with instructions on what Homebrew now wants to do and how to opt out. As it stands right now, this change doesn't respect the privacy and needs of users of this tool.

@codingcampbell
Copy link

When things like these come up, I wish projects would stop defaulting to Google Analytics in particular. If you need some anonymous usage info, fine, but does Google really need another vector into my life?

@bfontaine
Copy link
Contributor

@codingcampbell If you have a good solution that doesn’t need more work on our side (i.e. we don’t want to manage another server), we’re all ears.

@Ianleeclark
Copy link

@jasonroelofs I'm having a genuinely difficult time trying to digest what you've written.

If someone is breaching restrictions on sharing information by sending in an anonymous UUID with no other distinct markings to the data besides an OS version, then going to google.com is genuinely perilous. Your average blog is tracking much more information than homebrew is going to be, so heaven forbid if you ever need to consult the internet to figure out why a MySQL migration tool isn't working, or if you need to figure out how to install bcrypt on OS X, or any other reasonable internet search.

It's such a disingenuous thing to say.

@albus522
Copy link

@GrappigPanda Your package manager is not your web browser. Also you can set your web browser to hide a lot of information.

@lacombar
Copy link

At worst, you should ASK during installation whether or not the user want to collect analytics, with the default set to "NO". Then I would consider enabling it. No matter how transparent you are about this default opt-in, this is unacceptable.

@eknkc
Copy link

eknkc commented Apr 25, 2016

While I think it would be better to have a prompt, defaulting to "yes" (because noone would opt in otherwise) to enable this, why is this a huge deal?

I mean, if you have strict policy about this kind of tracking, you should just firewall google analytics (heck, a hosts line would do it) globally. Other software that you've been using might be tracking you just as well, without announcing / documenting anything. At least we know what homebrew does.

It confuses me because if this is unacceptable, it means you trust every single binary you've been running. Teach me how, please.

@lacombar
Copy link

lacombar commented Apr 25, 2016

@eknkc because when I setup a new machine, I don't want to have yet another environment variable to setup. We all think about it today, but in a year, I'm gonna have forgotten all about this, and analytics are gonna get enabled because of the default opt-in... behind my back.

This is just a really pervasive and sneaky way to behave.

@thecosas
Copy link

@eknkc Prompt on install/upgrade with a default to yes and instructions on how to opt out would probably be the best solution for all parties. Transparency, choice, and useful data for the volunteer developers which is representative.

@albus522
Copy link

@eknkc I don't go around installing software I don't trust, especially software whose sole responsibility is installing other software. I also have software that has asked me for usage info and my answer is always the same. I trusted homebrew when I installed it. This action is a definite breach in trust.

If homebrew wanted every user to know what was changing it would have been part of the update and/or install process. That would have been the trustworthy way, and no one would be here having this conversation.

@NickCraver
Copy link

This is how you turn a trusted project into an untrusted project. The fact that Homebrew is being open about implementation is great, but it's not being that open. When I updated earlier today (for dotnet CLI) I had no idea this was enabled until some outbound firewall logs alerted me. Then I found the Hacker News thread.

If we're saying Homebrew's being honest about it then let's actually be honest: you have very likely informed well under 1% of your userbase of this change. Users installing fresh are not being informed or afforded the chance to opt-out during install. Many users, even comfortable running brew commands they find on websites don't know how to opt out; they don't know how to set an environmental variable. Users upgrading are experiencing broken trust. Something they previously reviewed (as in my case) has, without so much as a console message, starting exporting their information.

I understand your need for analytics, but opting out should be trivial and the collection itself should be advertised to the user so they can do so.

@MikeMcQuaid
Copy link
Member

Hi all I'm the maintainer who added this. I would have responded sooner but I've been on a plane for the last 10 hours. I'll post more ASAP when I get my laptop onto some wifi; on my phone now.

@DomT4
Copy link
Member

DomT4 commented Apr 25, 2016

Can we please chill on the overzealous emoji use to every single comment here. We appreciate you have strong opinions and we're happy to discuss this further, but adding thumbs down emojis to people like Mike who's comment isn't any further than "I'll post later when I'm online, please bear with me" is a bit counterproductive.

Let's keep things civil and as calm as possible. Please bear in mind that Homebrew has a Code of Conduct that applies to how everyone talks to everyone else, whether that's us to you, you to us or you to each other.

If you'll give me two minutes before thumbing this post down to death like I've wandered onto Reddit by accident I'll leave a more personal opinion. Just want to try and keep the tone cool here so it can be discussed. Thanks.

@damieng
Copy link

damieng commented Apr 25, 2016

I don't speak for the brew team but metrics/telemetry are essential for making informed decisions about where to take your tools or product. Being off by default is useless - hardly anyone changes the default.

Something like an occasional (and at first use/upgrade) brew sends anonymous metrics to help improve this software. If you wish to opt-out type brew metrics off at the end of a brew console operation would seem to be a fair trade?

Not everyone can give back code/docs to a project but refusing to even allow telemetry be on by default seems like a very one-sided deal.

@NickCraver
Copy link

@damieng I honestly don't think the metrics themselves are a point of contention. I don't think I've seen anyone object to collecting metrics. The objections center around collecting metrics without consent. brew metrics off would be a great addition - that's much saner than expecting all users to know how to set an environmental variable.

I hope everyone is onboard with showing notices (both in the installer and when upgrading). If someone objects to notifying the user this is happening at all, then we should have a very different discussion.

@gmcmillan
Copy link

@DomT4 In what way is there overzealous emoji usage? People are using those emojies as a form of communication because it is an established feature in github and it is a quick way to show you are for or against a certain idea. It also prevents clutter with dozens/hundreds of people saying the exact same thing in written text. So I don't understand the hostility to the emojis in this case -- people are clearly communicating they don't agree with how this was implemented.

I have no problem with this feature as long as it's communicated to the user correctly and I don't believe it has in this case, which is why you guys are getting so much flack. Most people don't seem to have an issue with the collection, but with how it's communicated and enabled for users. I use homebrew a lot, but I never check homebrew twitter or the mailing list (and I highly doubt most users do) so I would have never known I was opted in to this.

Why wasn't this presented as an option at the command line for when the user next uses homebrew? That is the most obvious place I see where something like this should be presented to the user.

Homebrew collects anonymized statistics about your usage to better prioritize features and bugfixes. Staying opted-in would help us a ton in understanding your usage so we can make a better product for you, but we can understand if you want to opt-out.

Would you like to opt-in to the Homebrew analytics platform (powered by Google)? [yes/no]

@NickCraver
Copy link

Adding links to the original issue and PR as references/background:
Issue: Homebrew/legacy-homebrew#34101
PR: Homebrew/legacy-homebrew#50462

MikeMcQuaid added a commit that referenced this issue Apr 25, 2016
Make sure that users are notified on the first run of `brew update`
after we enabled analytics about how it works and how to opt-out. This
will be shown to all users who have not already seen this message from
`brew update` or through a new Homebrew installation.

References Homebrew/install#42
References #142
@joshmanders
Copy link

I'm gonna go against the grain and say thanks to the Homebrew team for being as transparent as possible about the data collected, and I will be keeping analytics turned on to help improve one of the most important pieces of software in my ecosystem.

@tylerault
Copy link

How to get what you want without being creepazoids:

  • Prompt for opt-in on install/upgrade, defaulting to "in".
  • Start your prompt with something positive, like: "Help us improve Homebrew by..."

You'll get analytics from a majority of users, and appease users with privacy/security concerns. You may get somewhat skewed usage info away from the latter group, but that seems a small price for the respect of your user base.

Side note: I'm genuinely amazed the team is surprised that this move upset people in the programming community. Esp. the way it was executed. After Snowden and the FBI/Apple case... and this is a package manager... we're not exactly the least paranoid crowd.

@MikeMcQuaid
Copy link
Member

Hi all, so we've had some time to think, chat and sleep and here's what's the current state of affairs:

We will:

  • continue to work on the software you use for free in our evenings and weekends on top of our jobs and try to make it better for you (as described by the docs: the analytics are there to make this free tool better)
  • modify the analytics implementation to improve privacy, implementation and messaging where people have reproducible cases or questions. A good example of a request: [Analytics] Do not drop .homebrew_analytics_user_uuid in HOME #145
  • consider moving to another analytics platform not run by Google if people have suggestions for one that is hosted and freely available for Homebrew to use

We will not:

  • be changing analytics to opt-in (in the near future, perhaps ever)
  • be letting people abuse maintainers (this includes name calling) anywhere, spam the repository or generally derail. We will block you from the Homebrew organisation temporarily if that happens and permanently if necessary. This is not censorship; you can say whatever you like on your own GitHub repository or blog: not here.
  • be motivated to work on Homebrew when people are abusive. If we are not motivated to work on Homebrew: Homebrew will not exist any more. The majority of people complaining have made no contributions to Homebrew.

Thanks for using Homebrew but if you feel these are unacceptable for you then there are other OS X package managers you can use instead. If you decide to stop using Homebrew (I notice some people in this thread already have said that on Twitter) please do not post again in this issue.

@bcardarella
Copy link
Author

I sense a hostile fork coming

@joshmanders
Copy link

@bcardarella

I sense a hostile fork coming

Because the maintainers won't tolerate abuse towards them? Good luck.

@bcardarella
Copy link
Author

@joshmanders no, because the collection of analytics is a hard stop for certain use cases.

@damieng
Copy link

damieng commented Apr 26, 2016

But as has been pointed out it's now simple to opt-out. Given the people complaining haven't contributed much (anything?) to brew over it's multi-year history I imagine the fork wouldn't be up to much.

@joshmanders
Copy link

@bcardarella is the work necessary to opt-out so hard you're actually willing to fork and maintain an alternate homebrew? Extra good luck.

@MikeMcQuaid
Copy link
Member

@bcardarella If people want to do so then good luck to them (they will need it).

@davebarkerxyz
Copy link

davebarkerxyz commented Apr 26, 2016

Because the maintainers won't tolerate abuse towards them? Good luck.

I feel like I'm missing something here. Things got particularly heated but I didn't see any outright abuse. Was this off-channel somewhere else?

The majority of people complaining have made no contributions to Homebrew.

For some people, this is their contribution. I haven't contributed to Homebrew otherwise. I am, however, a user. I felt that taking part in the discussion might help make Homebrew better for everyone - the privacy-conscious and those who want more metrics in order to further improve the tool.

be changing analytics to opt-in (in the near future, perhaps ever)

I really hope you'll reconsider. A package manager is a critical piece of code. I has to be completely bulletproof, in terms of both performance and behaviour. It's so vital to most users that it isn't worth tinkering with. I think there would be just as much backlash if Chocolatey, OneGet or apt-get started calling home with anonymised metrics.

I'm not completely against analytics (though I do have reservations). I just don't think it should be a default-on situation. At the very least, a Y/N prompt (as many, many other platforms offer) would be a compromise.

continue to work on the software you use for free in our evenings and weekends on top of our jobs and try to make it better for you

I think that most of us commenting here were doing so in our free time, because we all want to make the tool better.

With the way this was snuck in with no real notification, the acknowledgement that making it opt-in would turn off many users (that's a red flag in itself) and the response here (essentially "my way or the highway") I'm really disappointed.

@MikeMcQuaid
Copy link
Member

MikeMcQuaid commented Apr 26, 2016

Was this off-channel somewhere else?

@davb5 Yes.

I really hope you'll reconsider.

We will not.

A package manager is a critical piece of code. I has to be completely bulletproof, in terms of both performance and behaviour. It's so vital to most users that it isn't worth tinkering with.

I don't understand the point here?

For some people, this is their contribution.
I think that most of us commenting here were doing so in our free time, because we all want to make the tool better.

While I appreciate that: the tool does not get better without pull requests.

With the way this was snuck in with no real notification, the acknowledgement that making it out-in would turn off many users (that's a red flag in itself) and the response here (essentially "my way or the highway") I'm really disappointed.

Please don't use quote marks when you aren't going to quote me. Last night I added a message that every Homebrew user will see on update. If you don't see that as any sort of compromise, I'm really not sure what else to say.

@wlewis-sst
Copy link

[We will not] be motivated to work on Homebrew when people are abusive. If we are not motivated to work on Homebrew: Homebrew will not exist any more. The majority of people complaining have made no contributions to Homebrew.

Not that abuse is justifiable, but have you considered that your own actions may be abusive? Personally, I felt violated when I saw homebrew connecting to Google without notice and without my permission. Now that you have given notice, I still feel abused because you are not asking us for permission. You're assuming it.

Please, please, please ask permission. The default can be yes. But if you don't ask, you're being abusive.

@zmwangx
Copy link
Contributor

zmwangx commented Apr 26, 2016

I wasn't able to reply to a few comments directed at me yesterday after the issue was locked. Since then Mike has made it very clear and final, so I'll make one final comment as a courtesy to people who replied to me, and unsubscribe from this issue.

@ntpd

I'd like to see some examples of it being used in this way

I do. Since I don't use other people's bootstrapping scripts, I can't seem to give a good example right now where Homebrew is installed non-interactively, but git-cloning is definitely an important channel of installation — many of us don't like curl pipe into interpreter, let alone giving it root privilege. If you just want an example of Homebrew installation as part of a bootstrapping process, thoughtbot/laptop is one (I just took a closer look and although they set up /usr/local for Homebrew manually, they still run the installation script interactively, which is rather weird — they could very well just install via git).

but the simple solution is that the brew installation can use a --nointeraction parameter which defaults analytics to the safe option of False when a user is not able to consent.

FYI, you don't need an option. Just run the install script with stdin redirected to /dev/null, and it will skip all the interactive prompts. Still, analytics should be opt-out. The decision has been made anyway.

@davb5

Many people here object to default-on analytics. Let's respect that and leave metric collection off by default.

Many people? I see 115 people upvoting this issue, and even upvoting doesn't mean they object to default-on analytics. It could mean they didn't like how the rollout process was executed, and honestly it could have been better, but (1) it happened; (2) it has already been improved. What do you expect more?

Internet forums have a magnifying effect where a few users complaining (especially passionate ones leaving many comments) could feel like "many people".

@NickCraver

This is not correct. There are proxies, blockers, anonymizers, and even Tor. All remote analytics are easily avoided and to some degree inherently are in large environments due to so many users coming from a single source.

Sure. But how is calling home with an anonymous UUID any different? What's the harm when your UUID behind your anonymized IP can't be matched to you as an individual? Also, server logs can't be erased on your part, but calling home with analytics can be disabled.

@lacombar

you can't avoid NSA spying either, though it doesn't make it right.

This is such a straw man argument on every single level, it's impossible to reply.


Again, this will be my last comment, so even if you come back at me, you won't hear anything from me again on this thread. Sorry.

@davebarkerxyz
Copy link

davebarkerxyz commented Apr 26, 2016

Please don't use quote marks when you aren't going to quote me.

@MikeMcQuaid, If I was quoting verbatim, I'd have used GFM quote syntax. Like this...

if you feel these are unacceptable for you then there are other OS X package managers you can use instead

The "essentially" qualifier there was to signify that I was summing up, not directly quoting. I'm sure absolutely no-one was confused by this.

If people were being abusive off-channel then that's absolutely unacceptable. It's disappointing that some people have resorted to that. I understand the frustration from both sides here.

While I appreciate that: the tool does not get better without pull requests.

Whether creating PRs or not, I think using the tool and encouraging others to do so (to the point that it has become one of the primary package managers on OSX) means that we all have a vested interest in how the platform is developed.

Would a PR asking "is this OK, Y/N" on first interactive run be merged? Or would it be a waste of time?

@damieng
Copy link

damieng commented Apr 26, 2016

I think people should also remember that whenever you install a package today over http(s) via whatever mechanism (brew, fink, apt-get) whoever is running the server hosting that package has your IP address and knows what you downloaded from them.

You don't give them permission and they certainly haven't written any formal policies about what they do with the data but chances are it is logged to disk (80% of the web runs on either Apache, IIS or Nginx and all log access by default) that could be further analyzed.

@davebarkerxyz
Copy link

I think people should also remember that whenever you install a package today over http(s) via whatever mechanism (brew, fink, apt-get) whoever is running the server hosting that package has your IP address and knows what you downloaded from them.

I think the difference is that the party involved here has access to a lot more of our data. It's the "creepy factor" as much as anything else. It's not just the Homebrew admins grepping through server logs, it's our data being shared with a third party without our affirmative consent. Which may indeed be illegal under various jurisdictions data protection legislation.

As @zmwangx has said, I don't think we're getting anywhere further here. People are getting frustrated and upset about this change, the way it was rolled out and communicated, and the way people (on both sides of the debate) have acted.

If the decision has been made and we refuse to budge on it, I think we'll have to just leave it here. It's not worth going back and forth and upsetting each other further.

@nemesit
Copy link

nemesit commented Apr 26, 2016

Just remove this feature entirely, it's not like it'll be ever anonymous (you can trust google all you want but they are the ones who anonymize what they receive), or make it run on your own server.

@bfontaine
Copy link
Contributor

@davb5 If nobody submitted PRs Homebrew would be dead by now, whatever its users count is. There are a lot of examples in OSS of people who didn’t like some piece of software and went to write their own; even Homebrew is one of them (Max Howell didn’t like MacPorts so he wrote Homebrew).

@nemesit We don’t want to manage yet another server.

@agh
Copy link

agh commented Apr 26, 2016

RE: #142 (comment)

@MikeMcQuaid Respect your comments above. Wanted to suggest you look at Piwik for analytics. Honestly doesn't bother me that you're opt-out, or using Google Analytics, I think the adaptations you've made to make it easy to disable are sufficient and satisfactory. Obviously others may disagree. 👍

@nemesit
Copy link

nemesit commented Apr 26, 2016

and your users don't want analytics (most of them) I guess that's also the reason why it is opt out instead of opt in in the first place ;-p

@joshmanders
Copy link

Being opt-out vs opt-in doesn't mean "your users don't want analytics (most of them)" it means that most are satisfied with default configuration and do not go any farther as to customize. I know I don't. So even though I had been alerted to this via this discussion, I would have never known, and most likely had never enabled it despite wanting to help improve the software.

They've fixed some issues regarding it. If running a simple command to disable analytics is too troublesome for you, by all means, uninstall homebrew and manually install all these packages yourself.

I don't understand why you keep arguing about this. The team has expressed it's not going to change, so either accept it and move on, or don't accept it, and maintain your own fork with that one thing removed.

You're making a mountain out of a ant hill.

@nemesit
Copy link

nemesit commented Apr 26, 2016

being opt out means "we don't want to ask our users whether they want to opt in because we know the majority would not opt in"!

@MikeMcQuaid
Copy link
Member

I think we've taken this issue as far as it can. I appreciate all the feedback and thoughts here and will keep it in mind going forward.

@MikeMcQuaid
Copy link
Member

I don't understand why you keep arguing about this. The team has expressed it's not going to change, so either accept it and move on, or don't accept it, and maintain your own fork with that one thing removed.

This.

@clshrock
Copy link

clshrock commented Apr 26, 2016

Glad to find this out now. You guys burned bridges today and yesterday.

Explicitly, convention over configuration embodies the understanding that the default behavior should be the safest and most user-friendly. Your decisions and then your handling of the feedback betray that you value your own motives ahead your users. I hope the future will lead to different outcomes.

@joshmanders
Copy link

Your decisions and then your handling of the feedback betray that you value your own motives ahead your users.

Oh yes, wanting to provide a better product to their users is their own motive. Such a horrible act, have you no shame?!

Seriously, just disable it if you don't want it. It's not hard.

@clshrock
Copy link

clshrock commented Apr 27, 2016

Seriously, just disable it if you don't want it. It's not hard.

I am very glad that you can at least disable it. But like it was posted previously, it is yet another project that values taking what they want by default without asking first.

And now, to continue using homebrew, not only will I have to maintain that setting and verify that it is being honored in all future versions, but you have proven that I now must audit your software for future actions because I cannot trust that you have my best interest at heart.

It is further unfortunate that you refuse to consider the slight against your users that we are communicating has occurred.

You could have decided to save analytics data locally and then ask periodically if the user would like to submit the data to help you out. But no, you force opt-out only internet analytics on everyone and then get incensed when there is push-back from your users.

@MikeMcQuaid
Copy link
Member

And now, to continue using homebrew, not only will I have to maintain that setting and verify that it is being honored in all future versions, but you have proven that I now must audit your software for future actions because I cannot trust that you have my best interest at heart.

If you feel that way: please use another package manager.

On which note it seems I cannot leave this thread open so I'm locking it. Do not open new threads on this topic.

@Homebrew Homebrew locked and limited conversation to collaborators Apr 27, 2016
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests