Allowing fields to be made Private #256

Closed
paullenz opened this Issue Nov 21, 2013 · 13 comments

Projects

None yet

4 participants

@paullenz
Member

There are circumstances in which some fields of a public PopIt might need to be kept private. For instance, where a PopIt Instance is powering a WriteIt instance a politician might provide an email address that they are happy for WriteTo use to send message, but they don't want publicly exposing due to the risk of spam, abuse etc.

Therefore PopIt needs to be able to have some fields kept private.

https://github.com/lfalvarez - What are your thoughts on how this should work:

  1. Any field has the option to be made private via a toggle

  2. Only certain fields can be made private (if so, which ones)

  3. Something else?

@maugsbur

I would think option 1 seems the more robust solution, since basically any information can be sensible given a context. This would only exclude the identifier, for obvious reasons.

@chrismytton chrismytton was assigned Nov 25, 2013
@chrismytton
Member

The plan is to have a collection in mongo to store the hidden field data in, this will store the collection type and id for the document with hidden fields, along with the actual hidden fields themselves.

This would be represented something like:

> db.hidden.findOne()
{
  "id": "529371f56bea9cb1bc000002",
  "related_doc": {
    "namespace": "persons",
    "oid": "529371f56bea9cb1bc000001"
  },
  "fields": {
    "email": "bob@example.com"
  }
}

It seems that Mongo DBRefs would be a good way to represent this relationship from the hidden fields document to an unknown collection.

When data is POSTed to the API, the request body is checked for a hidden field, if present this represents the fields that are to be considered hidden. The hidden field is structured in the same way as normal fields would be in a regular, public request.

If an authenticated GET request is made to the API, we do a check in the hidden fields collection for a matching document or documents, then augment the document with the hidden field information. If a public unauthenticated request is made to the API then the public representation of the document is returned without the hidden fields.

@maugsbur

It seems like a good solution!

I'm wondering though, wouldn't it be easier to just manage the
hidden/public status at the access level? That meaning you store hidden
values like regular values, and just "hide" them when building the json
response. The "hidden" collection would then hold the name of the field,
but not the value.

This way, importing/exporting data is going to be a lot easier, and more
important, if I change the hidden/public status of a field, it will only
mean extending/contracting the hidden fields list, instead of
importing/exporting data from one collection to another.

Furthermore, you could eventually manage different privacy levels and
settings: the owner could access all fields, close collaborators could
access a subset of these, regular collaborators could access less fields,
etc.

Same goes eventually for editing, or just about any management setting. All
of these settings could be collection metadata, and the filtering done when
the GET, POST, PUT, DELETE or other actions are triggered.

Any thoughts?

P.D.
I'm not sure if field names are unique. If they're not, my solution doesn't
work...

2013/11/25 Chris Mytton notifications@github.com

The plan is to have a collection in mongo to store the hidden field data
in, this will store the collection type and id for the document with
hidden fields, along with the actual hidden fields themselves.

This would be represented something like:

db.hidden.findOne(){
"id": "529371f56bea9cb1bc000002",
"related_doc": {
"namespace": "persons",
"oid": "529371f56bea9cb1bc000001"
},
"fields": {
"email": "bob@example.com"
}}

It seems that Mongo DBRefshttp://docs.mongodb.org/manual/reference/database-references/#dbrefwould be a good way to represent this relationship from the hidden fields
document to an unknown collection.

When data is POSTed to the API, the request body is checked for a hiddenfield, if present this represents the fields that are to be considered
hidden. The fields property is structured in the same way as normal fields
would be in a regular, public request.

If an authenticated request is made to the API, we do a check in the
hidden fields collection for a matching document or documents, then augment
the document with the hidden field information. If a public unauthenticated
request is made to the API then the public representation of the document
is returned without the hidden fields.


Reply to this email directly or view it on GitHubhttps://github.com/mysociety/popit/issues/256#issuecomment-29220540
.

@chrismytton
Member

I'm wondering though, wouldn't it be easier to just manage the
hidden/public status at the access level? That meaning you store hidden
values like regular values, and just "hide" them when building the json
response. The "hidden" collection would then hold the name of the field,
but not the value.

I like this approach, as you say it certainly makes hiding and showing fields much simpler.

My only concern is array fields in the json, e.g. on http://popoloproject.com/specs/person.html#serialization the contact_details property on a person is an array, if you want to hide their mobile number but show their office number then it gets a bit complicated.

I suppose hiding contact_details that had the type cell could be represented something like:

{
  "contact_details": {
    "type": "cell"
  }
}

This could still get a bit tricky when there is a long list of contact details, but you could always use the above but specify the actual value that you want to hide if there is any ambiguity with the type, but still store the actual data against the person directly. So this would hide any contact_details members with the value of +1-555-555-0100

{
  "contact_details": {
    "value": "+1-555-555-0100"
  }
}

Does this sound reasonable?

@maugsbur

It sounds good to me!

Something else that came to my mind. This approach is perfect in case I
want to make a single person's data private, but it becomes a bit more
complicated if I want to make a certain field hidden for all the persons in
an instance, which is actually a more probable use case. In the
contact_details case, you'd have to copy/erase all values when making a
field globally hidden/public. There could be, besides the per person
privacy setting, a global privacy setting implemented the same way (without
the reference to the person or the value).

2013/11/26 Chris Mytton notifications@github.com

I'm wondering though, wouldn't it be easier to just manage the
hidden/public status at the access level? That meaning you store hidden
values like regular values, and just "hide" them when building the json
response. The "hidden" collection would then hold the name of the field,
but not the value.

I like this approach, as you say it certainly makes hiding and showing
fields much simpler.

My only concern is array fields in the json, e.g. on
http://popoloproject.com/specs/person.html#serialization the
contact_details property on a person is an array, if you want to hide
their mobile number but show their office number then it gets a bit
complicated.

I suppose hiding contact_details that had the type cell could be
represented something like:

{
"contact_details": {
"type": "cell"
}}

This could still get a bit tricky when there is a long list of contact
details, but you could always use the above but specify the actual valuethat you want to hide if there is any ambiguity with the
type, but still store the actual data against the person directly. So
this would hide any contact_details members with the value of
+1-555-555-0100

{
"contact_details": {
"value": "+1-555-555-0100"
}}

Does this sound reasonable?


Reply to this email directly or view it on GitHubhttps://github.com/mysociety/popit/issues/256#issuecomment-29288516
.

@chrismytton
Member

Just discussed this a bit more with @maugsbur and @lfalvarez. It makes sense to do the instance wide showing/hiding of properties first, as that's going to the the most common way it will be used initially. Then we can move to a per-document approach.

So, for example, if you wanted to hide all emails and cell numbers in the persons collection, the entry would look something like:

{
  "collection": "persons",
  "hidden_fields": {
    "email": true,
    "contact_details": {
      "type": "cell"
    }
  }
}

Other things that were discussed:

  • At some point it may be useful to make these hidden fields more granular, as per @maugsbur's comment above "you could eventually manage different privacy levels and settings: the owner could access all fields, close collaborators could access a subset of these, regular collaborators could access less fields, etc"
  • API keys seems like the simplest way to grant these access levels (though perhaps not the most secure approach).
  • It would be good to get popit.mysociety.org under SSL to make the API keys approach slightly more secure.
@chrismytton
Member

@maugsbur, @lfalvarez Hey guys!

I've got an initial implementation of the hidden fields feature implemented in mysociety/popit-api#17 which allows hiding a field collection-wide as well as per-document.

There are some initial docs available in the README on the hidden-fields branch, as they explain, currently you can hide fields by inserting a document using mongo on the command line. Is that going to be enough? Or do you think it would be better if there was a command line tool or an API call to specify hidden fields?

What are your thoughts?

@chrismytton chrismytton referenced this issue in mysociety/popit-api Dec 3, 2013
Merged

Hidden fields #17

5 of 5 tasks complete
@lfalvarez

Cool! I'm gonna check it out ASAP!

@chrismytton
Member

Ok, you can now specify fields to be hidden globally directly in the configuration, as detailed in the README. As discussed in the future we can update the API so fields can be hidden per-person when creating the document, but this provides enough to get started and test the feature.

@chrismytton
Member

Hey! I've merged the hidden fields work into master now and released v0.0.11 of popit-api to npm incorporating these changes. So you should be able to get the latest version with npm install popit-api.

@chrismytton
Member

Re-opening this ticket for now because while this is working in popit-api, it hasn't actually been implemented in the popit admin UI (this repo). To keep things simple I think it makes sense to hide fields if the user is logged out (public view), then give them full access when they're logged in.

The only slight issue is until #264 is resolved there may be some duplication getting the hidden fields to work properly in the UI, perhaps that's an argument for tackling that ticket first so there is a single interface to the models?

@chrismytton chrismytton reopened this Dec 6, 2013
@maugsbur
maugsbur commented Dec 6, 2013

Excelent work Chris! We actually only need it in the API for now, but of
course it would be great to have it in the popit admin.

2013/12/6 Chris Mytton notifications@github.com

Reopened #256 #256.


Reply to this email directly or view it on GitHubhttps://github.com/mysociety/popit/issues/256
.

@chrismytton
Member

Going to close this and open a separate ticket for hidden fields in the UI.

@chrismytton chrismytton closed this Jan 9, 2014
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment