Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add anonymous stats collection plugin #8291

Merged
merged 2 commits into from Nov 6, 2015
Merged

Conversation

wilsonge
Copy link
Contributor

@wilsonge wilsonge commented Nov 5, 2015

No description provided.

@@ -0,0 +1,7 @@
PLG_SYSTEM_STATS="J!Stats"
PLG_STATS_XML_DESCRIPTION="System Plugin that sends environment statistics to a server controlled by the Joomla! project for statistical analysis. Statistics sent include PHP version, CMS version, Database type and Database version."
PLG_STATS_UNIQUE_ID_DESC="An identifier that allows us to count unique installs of the plugin. This is sent with the statistics back to the server."
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we change "us" to "the Joomla! project

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does the project in Joomla! project get a capital letter?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We have both in core. ;) Joomla! project in the new "update mail text" and Joomla! Project in the Backend Update News (com_postinstall).

@zero-24
Copy link
Contributor

zero-24 commented Nov 5, 2015

@wilsonge Can You update the Copyright and add the package ans subpackage tags to all files. As well As Travis fails.

I would expect that we need testing here? So how Can this be done?

<license>GNU General Public License version 2 or later; see LICENSE.txt</license>
<authorEmail>admin@joomla.org</authorEmail>
<authorUrl>www.joomla.org</authorUrl>
<version>1.0.0</version>
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Version should be 3.5.0 ;)

@dgrammatiko
Copy link
Contributor

I would say just mute https://github.com/joomla/joomla-cms/pull/8291/files#diff-3ed68cf31bdf35dfc6b19cef6e2e7253R137
and put a print_r($data); after that line.
That should do it

@wilsonge
Copy link
Contributor Author

wilsonge commented Nov 5, 2015

sorry @dgt41 i just rebased so not sure what you are referring to anymore


// Uncomment the following line to enable debug mode (stats sent every single time)
// define('PLG_SYSTEM_STATS_DEBUG', 1);

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We miss the default defined call here ;)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What do you mean?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

@dgrammatiko
Copy link
Contributor

@wilsonge I was trying to suggest a testing procedure
mute https://github.com/wilsonge/joomla-cms/blob/stats/plugins/system/stats/stats.php#L137-L157
put a print_r($data); after line 157 and see what is you to be send to the server

@wilsonge
Copy link
Contributor Author

wilsonge commented Nov 5, 2015

You can go here and check your data exists :P http://developer.joomla.org/stats/

@wilsonge
Copy link
Contributor Author

wilsonge commented Nov 6, 2015

OK I just fixed a pretty major bug in the code. Things should work now

@dgrammatiko
Copy link
Contributor

@wilsonge you got my data 😃

@wilsonge
Copy link
Contributor Author

wilsonge commented Nov 6, 2015

Nica

@wilsonge
Copy link
Contributor Author

wilsonge commented Nov 6, 2015

Code style fixes should mean travis is happy

@wilsonge wilsonge added this to the Joomla! 3.5.0 milestone Nov 6, 2015
@infograf768
Copy link
Member

If this is merged, please alpha order the new strings and modify admin en-GB/install.xml

@roland-d roland-d merged commit 3dbf207 into joomla:staging Nov 6, 2015
@wilsonge wilsonge deleted the stats branch November 6, 2015 23:13
@coder4life
Copy link

Although a bit late to the party, regardless
https://groups.google.com/forum/#!topic/joomla-dev-cms/FkdJH74rCqQ

@Chris-Jones-Gill
Copy link

Enabled by default?
Does this mean that data is sent before I have the option to disable?
In which case it is not an opt-out at all.

This may be a good thing for the project - lots of data, yum, we can learn lots - and maybe for the user in the long term. But it is not, in my view, a good thing to be doing without explicit permission.

Because we want the data, and we believe it will be useful, is no reason to auto collect data from users.
Data collection should always be anonymous and with informed consent.
Look at windows 10 and it's data grabbing (and the tech community railing against that), is this really that different.
I would have preferred a post install message that has the data listed (with each piece optional, tick boxes, lots of tick boxes) and a "send to Joomla! to help us improve" button.
The same happens after each update - it is opt in, the user sees the data being sent, can opt out of any individual bit, and then can make a fully informed decision.
There could also be an option to auto-send after each update - but it should be opt-in.
Maybe a message
Don't want to see this again, support Joomla! by auto-sending this data after every update, or
Don't want to see this again, opt out of all data collection, or
Ask me every time [ticked by default]
Three tick boxes, or radio buttons.

I am totally against any auto-collection of data without consent. Saying you can opt-out after it is sent is a bit disingenous (dishonest even). Will that opt-out send a message to the data collection server and delete the data it sent earlier, before the chance to say no? I doubt it.

Opt in is always the best for consumers - if your proposal (offering/product/email) is adding value then people will understand and opt in. If people don't, then whatever you are doing isn't really wanted.

@wilsonge
Copy link
Contributor Author

wilsonge commented Nov 8, 2015

Yet again. I know you compare against Microsoft. But how about we compare against our direct competitors. Wordpress collect this data with any kind of opt out at all. Drupal collect this data by default (and I think (but not 100% from glancing through the ticket) you can also not opt out). So I think that what we have is still a better compromise than any of our direct competitors whilst still giving useful information to the project

@Chris-Jones-Gill
Copy link

Once again, just because others do it is not a good enough reason to copy them.
Project_desire !> User_choice
As for the Microsoft reference - they collect shit loads more, but is the negativity directed against the quantity they collect or the fact that they are collecting it in the first place. Answer, because they are collecting it.

Wordpress collect data like this - which is one of the reasons I don't use it
Drupal collect data like this - which is one of the reasons I don't use it
I will be checking how Drupal8 does this, which is very modular, and see if this has changed.

Would it really be so hard to make the post install message the opt-in question?

A post install message is already generated, so why not use it to get consent?

Something like

To help the Joomla! project we would like to collect the following information about your install.
[x] CMS version: 1.5
[x] Database type: ICL ME29
[x] Database version: 0.0.1
[x] Server OS: VAX/VMS

This will help us improve Joomla! in many ways. The stats are always visible at https://developer.joomla.org/about/stats.html and you can find out more about how we use this information {here}

[ ] Don't want to see this again, support Joomla! by auto-sending this data after every update
[ ] Don't want to see this again, opt out of all data collection
[x] Ask me every time

{submit button}

@wilsonge
Copy link
Contributor Author

wilsonge commented Nov 8, 2015

In itself doing it because others do it isn't a reason to copy them. But when you are referring to what other projects do I think it's reasonable for me to refer to other (in my opinion more relevant) projects in my defence :)

I explained as you know two different reasons why we want this data in this google group post (https://groups.google.com/forum/?fromgroups=#!topic/joomla-dev-cms/FkdJH74rCqQ) so it's not like we are doing this just because other projects are.

@Chris-Jones-Gill
Copy link

I see why it could help both the project and the end user (the bcrypt thing was a nightmare, as were the discussions around raising the minimum supported PHP version), but it still does not justify taking the data without consent.

Ask the question, explain why, gather data with consent.

Project_desire !> User_choice

Maybe I should have left the Microsoft example out of my post (although I still think that collecting data without explicit consent, by them, WordPress and Drupal is wrong).

Telling people you have collected the data after the fact, and telling them it is for their own good, but they can stop it happening again if they really want - when they know it is too late to do anything about the data that was grabbed already - is wrong in so many ways.

What I install and where is my business, my data, my information. You, nor anyone else, has the right to know that. I can share it if I want, but you cannot (should not) ever just take that information without my consent.

That is the core of my objection to this. Consent. Before data is taken, not after when it is too late.

You still have not answered my question regarding the post install message acting as the consent form.

Implementing it as part of the post install message will not be affected by 3rd party installers (like it would if the data collection was an option during install), and would give people the choice.

If people do not consent to sharing the data, then despite the desire of the project, the users are not comfortable with it. This is their data, their information, they own it and can choose to share or not.

Perhaps a compromise, whereby the consent form is displayed as a post install message but if it is not interacted with within (for example) 48hours of the install/update the data is sent. This can be made clear in the message.

This would then catch those who can't be bothered to read and review the messages, those who simply don't care about it (one way or the other), as well as those of us who prefer to have a choice.

@mbabker
Copy link
Contributor

mbabker commented Nov 8, 2015

The post-install system cannot handle the conditions you've requested. It is not designed to give any actionable point beyond a single action button or a dismissal button. Likewise, it is unable to give the specifics of data which you've requested; the system works with static language strings that cannot receive dynamic variables.

@Chris-Jones-Gill
Copy link

Thanks Michael.

The simple, glib, response...

Then change it. Redesign it to cope with this use case, and postpone implementing the data collection until it is done.

The more considered response...

Are there any other existing mechanisms that would be able to implement this use case?

Would having a 48hour delay in sending automatically be doable?
The static post install message could say something along the lines of:

Data about your install (CMS version, database type and version and OS) has been collected and is useful to the Joomla! project. This will be sent to Joomla! 48 hours after your install time unless you disable the plugin.
You can do that by [instructions or link]
This information is very useful to us and can help create a better Joomla! The information will always be freely available {here link}, and you can find out more about how, and why, we use it {here link}.

Again, those that care about choice have one. Those that don't read the message probably don't care about it. The project gets their data (from those who actively consent and from those who do not actively say no).
This is not a perfect solution, and is still based on the premise that unless you opt-out you are happy to send your data (which I believe is the wrong way round). However, it could provide a suitable compromise to both viewpoints.

Let me finish by saying I don't object to the project wanting the data, nor the collection of it, and I see how it could be useful. I just don't like the lack of consent, the lack of choice, of not being asked "Is this OK?"

@Chris-Jones-Gill
Copy link

For non-native English readers...

I used the word "glib" in my previous post.

It means
"said or done too easily or carelessly : showing little preparation or thought"

"readily fluent, often thoughtlessly, superficially, or insincerely so:
a glib talker; glib answers."

It has nothing to do with the Glib project https://wiki.gnome.org/Projects/GLib
"GLib provides the core application building blocks for libraries and applications written in C. It provides the core object system used in GNOME, the main loop implementation, and a large set of utility functions for strings and common data structures."

It just occurred to me that not everybody reading would understand the word, and that a FOSS project of that name exsits (and maybe more people know of it than of the English meaning of the word).

@coder4life
Copy link

So was thinking about this today, and I think I have a great compromise. During install/update generate some sort of time/date info of when the plugin was installed. In the plugin code we can allow a 24 hour delay before data is allowed to be collected based on that time/date. This gives Joomla the ability to enable the plugin by default, but give enough time for people to make an executive decision based on postinstall messages.

@Bakual
Copy link
Contributor

Bakual commented Nov 9, 2015

Already proposed with #8346 which would give a 6hrs delay to first sending data.

@jjsjjs
Copy link

jjsjjs commented Feb 6, 2016

Make sure these plug-ins are disabled by default and are opt-in. And not the other way around like it is now. Even if you do not gather any private information. This is not allowed by several countries. So to apply the laws for such countries set it disabled and change to opt-in. Or you most certain will get law suits. You will not be the first who will be in trouble for such issues. I understand the great values you see in this. But this will be probably a show stopper for me and will change to an other csm who does not have such plugins and/or gather their information like this.

@wilsonge
Copy link
Contributor Author

wilsonge commented Feb 6, 2016

We are the last of the major 3 CMS who doesn't have this feature. Both Wordpress and Drupal already have this with no opt-out's at all. They haven't been given lawsuits against them yet ;)

@jjsjjs
Copy link

jjsjjs commented Feb 6, 2016

Because many people do not know this yet, they feel save to do this.
But it is an illegal practice especially in EU. And because they do it, you could then think about especially to not do it.
When people or official parties get this information that data is being collected, law suits will follow. Or at least you will be asked friendly first to change it asap.
A warned man counts twice.

@wilsonge
Copy link
Contributor Author

wilsonge commented Feb 6, 2016

We have checked the legalities out carefully. We are fine to do it. And by giving the parameter we are still offering better than either Drupal or Wordpress.

@chrisdavenport
Copy link
Contributor

@jjsjjs The EU law that I think you are referring to covers personal information. The intersection of the set of websites and the set of EU citizens very likely contains no members.

@jjsjjs
Copy link

jjsjjs commented Feb 6, 2016

Well if so, then keep the users your friend by not scare them off with this force of ignorance by putting it default enabled with an opt-out. Show them you respect them even if it is save data you collect by setting it disabled by default with an opt-in. Many people do not like the point of view you now show. It's still Beta so you can still reverse it.

@mbabker
Copy link
Contributor

mbabker commented Feb 6, 2016

Have you tested the latest beta package or followed any of the referenced items in this thread? If so you'll note that the issues you've raised here have already been addressed.

@jjsjjs
Copy link

jjsjjs commented Feb 6, 2016

Explicitly not, because of what i've read. In the above thread i don't see it addressed. I only see arguments. For me addressed would mean it is solved for the anctious people so the enabled by default and opt-out are gone.

@mbabker
Copy link
Contributor

mbabker commented Feb 6, 2016

I'd suggest reading then. But to sum it up, the plugin stays enabled by default with a delay before it will start sending data (the plugin is how the user is prompted to send data, turn it off by default and you don't even get the prompt which defeats the purpose of having an opt-in or -out system to begin with).

@brianteeman
Copy link
Contributor

That would be far too easy.
On 6 Feb 2016 8:41 pm, "Michael Babker" notifications@github.com wrote:

Have you tested the latest beta package or followed any of the referenced
items in this thread? If so you'll note that the issues you've raised here
have already been addressed.


Reply to this email directly or view it on GitHub
#8291 (comment).

@brianteeman
Copy link
Contributor

Basically see 8346 as referenced immediately before your first comment

@jjsjjs
Copy link

jjsjjs commented Feb 7, 2016

That would be the best as you say, mbabker. No prompt for it is no problem at all. You could set a text message that there is such an plugin available. Like there is now for other things when first install Joomla. Look for example at the CookieLaw. Most people even did not know about it at all. Many Cookies are harmless. There was no opt-out or opt-in. A minister of the Netherlands came with the Law which turn out to an EU law. And have to be noted up front to have an opt-int and/or opt-out depending on the information gathered. If a cookie is only neccessary for functionality then a notice is enough. But when data collecting is needed then an opt-out is obligated. I see this is similair. And i guess a court will also see this as similair, because you will collect data, non-private or not. It's not needed for the functionality of the website, but for future versions.

@brianteeman
Copy link
Contributor

You are discussing something that has already heappend

On 7 February 2016 at 09:29, jjsjjs notifications@github.com wrote:

That would be the best as you say, mbabker. No prompt for it is no problem
at all. You could set a text message that there is such an plugin
available. Like there is now for other things when first install Joomla.
Look for example at the CookieLaw. Most people even did not know about it
at all. Many Cookies are harmless. There was no opt-out or opt-in. A
minister of the Netherlands came with the Law which turn out to an EU law.
And have to be noted up front to have an opt-int and/or opt-out depending
on the information gathered. If a cookie is only neccessary for
functionality then a notice is enough. But when data collecting is needed
then an opt-out is obligated. I see this is similair. And i guess a court
will also see this as similair, because you will collect data, non-private
or not. It's not needed for the functionality of the website, but for
future versions.


Reply to this email directly or view it on GitHub
#8291 (comment).

Brian Teeman
Co-founder Joomla! and OpenSourceMatters Inc.
http://brian.teeman.net/

@sovainfo
Copy link
Contributor

sovainfo commented Feb 7, 2016

Request for a lock on the.is topic. This is not the right place for that discussion!

@Bakual
Copy link
Contributor

Bakual commented Feb 7, 2016

The point in that law is the "personal data" part. It's not about collecting generic data like server infrastructure (part of which is shared with public on every request).
That's why this plugin is no issue at all and as said multiple times what you're requesting is already there in the current beta. So please first check out the current state of the plugin.

@joomla joomla locked and limited conversation to collaborators Feb 7, 2016
@wilsonge
Copy link
Contributor Author

wilsonge commented Feb 7, 2016

Locked

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Feature Language Change This is for Translators
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet