New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[RFC] Privacy Tool Suite in Core #20281

Closed
mbabker opened this Issue May 2, 2018 · 62 comments

Comments

Projects
None yet
@mbabker
Member

mbabker commented May 2, 2018

I'm about to ask a very loaded and very blunt question, primarily directed toward project leadership but all feedback is welcome. Is there interest (as in should this be a roadmap item for a short term release schedule) in having a "proper" tool suite and API in core to help facilitate privacy (GDPR) related items? In essence this means having some high level APIs in the Joomla\CMS\Privacy namespace and a com_privacy at a minimum with some baseline functionality and a way for site owners to interface with said API.

Why does core need this functionality? For the most part, core itself works with a very limited set of data that might fall under the scope of privacy related laws such as GDPR, however it acts as the framework/facilitator for thousands of extensions to perform actions which do fall under that scope, and I do believe that we as that framework owe it to our userbase to at a minimum do the due diligence to either provide a set of tools to facilitate these types of activities in a consistent manner or to tell users that the core project does not feel it is its responsibility to offer these tools and they should be provided by the ecosystem.

Now, here is my offer if this is indeed something the project will accept and support with resources (that means I need you, casual reader, to assist with this effort; I will not do the entire project myself). Internally a discussion started about having a 3.9 release with privacy/GDPR related functionality added to core based on the many PRs that have already been opened and I offered to coordinate that release; that offer still stands. As it relates to this effort, I will offer to team lead, project manage, or whatever you want to call it, to steer the efforts in a coordinated manner so that we can have a releasable product in a reasonable timeframe (clearly we're already a few months too late to starting this work, so anyone expecting a finished product before May 25 has unrealistic expectations unless you're going to pay me full time to work on this, just being blunt about it).

In return, as I hinted at above, I need commitments from interested parties in working on this tool suite. Coders to help write the code, testers to help make sure we aren't building an unusable mess, writers to help document all the things, etc. etc. I will not take this project on by myself, I already do way too much with too little time and making my time offer here is stretching my available resources for other project tasks.

If agreed upon in principle, we can have a space set up relatively quickly and I will outline specifics of what I would consider the MVP for this effort to be at that time.

@brianteeman

This comment has been minimized.

Contributor

brianteeman commented May 2, 2018

i agree in principle - obviously - but i will absolutely not waste another minute of my time on this if it is going to face continual pushback

@nibra

This comment has been minimized.

Member

nibra commented May 2, 2018

+1
Count me in.

@mbabker

This comment has been minimized.

Member

mbabker commented May 2, 2018

i will absolutely not waste another minute of my time on this if it is going to face continual pushback

Me either. That's why before I put in any effort beyond this item I am basically looking for a "we support this feature in principle" response; I'm not wasting my own or anyone else's time on a feature that isn't going to ship this year and isn't supported by leadership. At this point all I'm asking for is the support and the commitment from other teams and individuals to support this effort as it progresses (dev and getting a release milestone accomplished), as I've made perfectly clear here I'm putting my hat in the ring to do a lot of the heavy lifting as it relates to coordination and release efforts.

@Sandra97

This comment has been minimized.

Sandra97 commented May 2, 2018

I support this idea. I'll do what I'm able to do to help.

@parthlawate

This comment has been minimized.

Contributor

parthlawate commented May 2, 2018

As I've said on Glip I support this wholeheartedly.. and would also go ahead and contribute developer time from the Tekdi and Techjoomla team.

@brianteeman

This comment has been minimized.

Contributor

brianteeman commented May 2, 2018

it doesnt matter what we say. all that matters is that those whose voice matters agree. so far they have either been against adding data privacy code into joomla or have been completely awol.

presumably it is not a big job to do as we already have all the code created for joomla.org

@ot2sen

This comment has been minimized.

Contributor

ot2sen commented May 2, 2018

Will test and provide feedback.
Actually think it would be a good idea of finding a bag of gold coins to speed this up would be well spent money by the project.

@brianteeman

This comment has been minimized.

Contributor

brianteeman commented May 2, 2018

@ot2sen the bag of coins was spent on the code created for joomla.org

@mbabker

This comment has been minimized.

Member

mbabker commented May 2, 2018

Someone in the compliance team has also pointed out the .org code as being a solution that can be built on. If they (or anyone who has already created similar code) are willing to share (the compliance team seems to be on board with the sharing piece) then that does help us move things along. Being so late in the game it's going to be helpful if we can consume/use existing code, but that's a detail I'd like to focus on after getting project approval instead of now.

@jeckodevelopment

This comment has been minimized.

Member

jeckodevelopment commented May 2, 2018

Count me in. I'll do what I'm able to do to help.

@tonypartridge

This comment has been minimized.

Contributor

tonypartridge commented May 2, 2018

+1 and @GeraintEdwards might be able to help too.

@aDaneInSpain

This comment has been minimized.

Contributor

aDaneInSpain commented May 2, 2018

I am very much for this. Especially if it includes some kind of way for 3rd party developers to highlight/manage their personal data as outlined in my issue.

@mbabker

This comment has been minimized.

Member

mbabker commented May 2, 2018

Something like that is already on my mind. It HAS to be pluggable/extensible.

@jomres

This comment has been minimized.

jomres commented May 2, 2018

I'm currently implementing the "download your personal info" side of things for Jomres. It would be really nice to receive a call from J telling Jomres that A the user's requested a download or B the user's requested anonymisation. Right now it has to be done within Jomres and then potentially either I'll have to anonymise the J core users table, or just ignore it and hope that the site admins direct users to another tool for that. Working the other way around would be much more helpful, so receive from J > Anon my data (of which, you can imagine, a booking engine has quite a lot) > wave cheerio to the guest.

@coolbung

This comment has been minimized.

coolbung commented May 2, 2018

We're in. In fact we'd begun some design brainstorms around the architecture of this. A wiki of the brainstorm is available here - https://github.com/techjoomla/user-deboarding/wiki

Some high level items :

  • The core provides an API to store named consents
  • If a user decides to revoke a consent the core triggers an event
  • Each extension implements a method to delete/anonymise data after user deletion.
  • Each extension may implement an API to generate an indexed export (this facilitates a google takeout like service)
@davies401

This comment has been minimized.

davies401 commented May 2, 2018

100% Yes please. It would make more sense than having to find a GDPR component for each site

@JoomliC

This comment has been minimized.

Contributor

JoomliC commented May 2, 2018

+1
Count me in ! (test + code + 3rd party)

Thanks @brianteeman and helpers for inspiring work done here: #20051

@dam-man

This comment has been minimized.

Contributor

dam-man commented May 2, 2018

@coolbung what needs to be done. As there no code attached yet in the repo. Does this become part of the J core?

@Webdongle

This comment has been minimized.

Contributor

Webdongle commented May 2, 2018

@mbabker

Internally a discussion started about having a 3.9 release with privacy/GDPR related functionality added to core

Great idea hope it works

@Ruud68

This comment has been minimized.

Contributor

Ruud68 commented May 2, 2018

This is important for Joomla so count me in; testing, troubleshooting, development work.

@crystalenka

This comment has been minimized.

crystalenka commented May 2, 2018

I don’t have a lot of time to commit to this but would be happy to provide UX-oriented feedback when asked.

@srseale

This comment has been minimized.

srseale commented May 2, 2018

I have been looking for some way to contribute for some time. This sounds like a good thing. I can write documentation, test and I am even willing to help with the code. I'm a newbie at this, totally unfamiliar with Github and this process but I'm willing to learn...
Count me in.

@mbabker

This comment has been minimized.

Member

mbabker commented May 3, 2018

After talking this over with @wilsonge and based on his last comment in #20140 we are moving forward with this.

I've set up a repo at https://github.com/joomla-projects/privacy-framework which has the high level items identified, most of which I would consider needing to be completed to have a MVP for merging back to core. Hopefully what's there so far makes some sense, if not feel free to ask questions.

PRs like #20051 or #20173 are probably good candidates to move into that repo so we can make sure they fit into the overall goals and workflow that's being created.

@WilkoRietveld

This comment has been minimized.

WilkoRietveld commented May 3, 2018

Great initiative, I am available for testing!

@rameshelamathi

This comment has been minimized.

rameshelamathi commented May 3, 2018

+1
Good move. Totally agreed. Count me in

@sakiss

This comment has been minimized.

sakiss commented May 5, 2018

Where can i find specs for that project?

@brianteeman

This comment has been minimized.

Contributor

brianteeman commented May 15, 2018

@f-hamel

This comment has been minimized.

f-hamel commented May 15, 2018

@brianteeman Maybe I misunderstand you.

But I think we really should discuss about this here.
Because the European Court of Justice decided, in C582/14 that IP addresses are personal data because they can be related with an natural person.
Further ecj said collecting and using personal data relating to a user are only allowed
"so far as that the collection and use of that data are necessary to facilitate and charge for the specific use of those services by that user"

So where is a need to anonynise ip addresses and maybe the Privacy Tool Suite is the right place for this.

So if I'm wrong with that, it's ok. This should only be a hint.

@brianteeman

This comment has been minimized.

Contributor

brianteeman commented May 15, 2018

No you misunderstand that court decision. As explained in detail here https://www.whitecase.com/publications/alert/court-confirms-ip-addresses-are-personal-data-some-cases the court said that it "can be personal data" in some circumstances.

The CJEU decided that a dynamic IP address will be personal data in the hands of a website operator if:

  1. there is another party (such as an ISP) that can link the dynamic IP address to the identity of an individual; and
  2. the website operator has a "legal means" of obtaining access to the information held by the ISP in order to identify the individual.

On the facts, if the BRD has the legal power to compel the relevant ISP to disclose sufficient information to identify Mr Breyer, then Mr Breyer's IP address will be personal data in the hands of the BRD.

As a regular site owner does not have that legal power then it is not personal data

@Webdongle

This comment has been minimized.

Contributor

Webdongle commented May 15, 2018

As a regular site owner does not have that legal power then it is not personal data

In addition the Host server would still have the records ... so a regular site owner would not be destroying evidence.

@f-hamel

This comment has been minimized.

f-hamel commented May 15, 2018

Thanks for sharing this @brianteeman
I guess we have a grey area here, like we have many of them in GDPR.
And there are lots of debates going on about if ip addresses are personal data.
IMO they are.
Maybe in the close future we will get clear decision by the European court.

I see there is an issue joomla-projects/privacy-framework#16.
Log rotating withe clearing them would be great solution.

In addition the Host server would still have the records ... so a regular site owner would not be destroying evidence.

@Webdongle As I'm working for a German hosing provider, I can tell you we have to anonymize IP addresses for the access.log, and we will do so. Also we will rotate apache logs away after a few days.
Just because German law and the implement of the GDPR in Germany.
So we no evidence if someone need them, e.g. in case of an attack. So we have big controversy debate about the retention of data and GDPR.

Maybe we get a log rotate to Joomla, which would help to solve the problem that ip addresses are stored a long period and with this unnecessarily.

But I think this discussion will go to far here.
So for me it's no more a part of this issue.

@tonypartridge

This comment has been minimized.

Contributor

tonypartridge commented May 15, 2018

@mbabker

This comment has been minimized.

Member

mbabker commented May 15, 2018

As pointed out in the discussions on the privacy-framework repo, there are legitimate use cases for processing IP addresses that would trump any personal data consent or "right to be forgotten" type of requests, so Joomla can't just blindly anonymize them. What we can do is provide tools for things such as log rotation to ensure logs are deleted in a timely manner, and review core's logging to ensure when IP addresses are logged there is a valid reason to do so. And to me that is the extent that our efforts need to go to, anything further creates too many problems.

@schultz-it-solutions

This comment has been minimized.

Contributor

schultz-it-solutions commented May 15, 2018

Agree with mbabker 100%.

@tonypartridge

This comment has been minimized.

Contributor

tonypartridge commented May 15, 2018

@f-hamel

This comment has been minimized.

f-hamel commented May 15, 2018

So with what @mbabker wrote I agree too.
Saving ip addresses for legitimate use cases in short period, is fine.
A start writing my first post in this issue did not saw the discussion in the privacy-framework repo.

@Wildfigmedia

This comment has been minimized.

Wildfigmedia commented May 25, 2018

Happy to help with testing and reporting

@gwsdesk

This comment has been minimized.

gwsdesk commented May 26, 2018

With GDPR we need a consent option on all forms on a website. This includes the Joomla contact-us forms for instance notwithstanding registration etc. We should not start the entire discussion on GDPR here again (who is right and who is wrong) That makes no sense. We need basic consent/denial/etc options implemented and we are late (no offense) since this is known for a long time. So we need to do urgently something. If you use RSforms/Breezingforms etc you can easy add a consent and your privacy policy (Joomla content) can be adjusted.... You also can use a plugin that asks for a consent/denial/info many are available

However I believe we need to address this globally as Michael mentioned? Do I miss something?

Leo 8)


This comment was created with the J!Tracker Application at issues.joomla.org/tracker/joomla-cms/20281.
@brianteeman

This comment has been minimized.

Contributor

brianteeman commented May 26, 2018

yes you have missed that this work is under way

@gwsdesk

This comment has been minimized.

gwsdesk commented May 26, 2018

Tnx Brian much appreciated. We have a link to "work in progress"?

@mbabker

This comment has been minimized.

Member

mbabker commented May 26, 2018

The same https://github.com/joomla-projects/privacy-framework repo that's been linked to a few times in this issue over the last few weeks.

@studio42

This comment has been minimized.

studio42 commented May 31, 2018

Does now a core solution exist ?
I need it for acy mailing, chronoform, Joomla forms, user connect, Virtuemart.... in same website
I dont want add a plugin for each case, because i dont want that a user have to confirm in each form but only one time if the user is connected for eg.
And this is only for 1 website, i have some website using more forms.
What is the current solution/plugin/component that can handel all case ?
Regards

@mbabker

This comment has been minimized.

Member

mbabker commented May 31, 2018

It is still a work in progress as indicated by the activity on the https://github.com/joomla-projects/privacy-framework repo.

@Quy

This comment has been minimized.

Contributor

Quy commented Jun 20, 2018

See PR #20800

@brianteeman

This comment has been minimized.

Contributor

brianteeman commented Jul 5, 2018

A lot of people here wanted to see the privacy component in Joomla. We worked hard to create it - now its your turn. You volunteered to help test - without those tests it will not be released. Find out how here #20800

@brianteeman

This comment has been minimized.

Contributor

brianteeman commented Jul 30, 2018

Seriously? There were loads of people that said they wanted this feature and stood up to be testers and yet hardly any of them have bothered. Maybe we should just drop this. It's a major new feature and if its not widely tested then I would be very reluctant to add it.

@Wildfigmedia

This comment has been minimized.

Wildfigmedia commented Jul 31, 2018

Sorry guys, Been on some much needed leave. On this now

@Ruud68

This comment has been minimized.

Contributor

Ruud68 commented Sep 19, 2018

Hi, not sure if this is the correct thread to start a discussion, if not please point to the correct place :)

I am currently testing 3.9-beta2 (nightly) on my acceptance environment: Complements on everybody involved, looks great!

One thing I noticed is that all privacy related components (com_privacy, com_actionlogs) do not have ACL permissions implemented but are (hard) limited to super users only.

Now I am maintaining some Joomla sites for 'larger' organizations. There is a strict segregation of duties as to what a super user can do and a manager / administrator: this is mainly based on technical capabilities (install / update components, etc.)

Furthermore these organizations most likely have an employee in their mids who is responsible for privacy related things (e.g. a DPO: data protection officer).

Now because the current privacy implementation is based on the rule: super user only, this requires a 'mix' of a functional and technical maintenance... This is to limiting.

What I would expect is that com_privacy, com_actionlogs has an ACL permission implementation so I (as hired technical super user) can 'delegate' the functional privacy role to the person responsible for this in the organization without giving this user super user rights....

Does this make sense?

@mbabker

This comment has been minimized.

Member

mbabker commented Sep 19, 2018

It was purposefully decided to restrict these components to super users only. In the case of a privacy request, when exporting data the person processing the request inherently must have (or gains through an information leak) read access to all of the data that plugins are responding with. When removing data, the person processing the request must inherently have (or gains by bypassing ACL) what is in essence a combination of core.edit and core.delete permissions. And for action logs, someone could see a user performing actions in a component they may not even be aware is installed because ACL restricts them from even seeing it.

If we made it so non super users were able to access these components, inherently we are either dealing with some form of ACL violation in other parts of the system or the processed data must be ACL restricted to that user's role, resulting in an incomplete action log or data export.

@micker

This comment has been minimized.

micker commented Sep 19, 2018

i am not sure that only for super user ... in RGPD, DPO is a person who need to check this, it's not webmaster of site, its a security responsability... i think we need to allow this ability to a specific user or group

@mbabker

This comment has been minimized.

Member

mbabker commented Sep 19, 2018

If someone can spec something out in a way that does not come across as an ACL bypass security issue we'll review it. It is not that we necessarily want to keep these areas super user only, but when you consider what the components are designed to do, you also must keep in mind that the DPO person essentially has to have a combination of core.manage, core.edit, core.edit.state, and core.delete permissions to adequately handle requests and that through the privacy component the person processing a delete request will more than likely bypass any "regular" ACL checks for the edit and delete permissions that would be done on a normal save or delete action.

At the end of it all, we still need to be mindful of the site's ACL restrictions and that what the last comments here are proposing would in effect act as a ACL bypass through the privacy component interface; it is entirely possible through what you both are requesting that a DPO would be given read or edit access to an item they would not normally otherwise have and I'm not sure that is an acceptable risk for core.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment