Iqss/1724 anonymized access #7908

qqmyers · 2021-05-27T17:28:35Z

What this PR does / why we need it: This PR enables a new option when creating a Private URL - to create one that only allows 'anonymized access'. When a user accesses a dataset using this type of Private URL:

author names are withheld from the dataset and file citations
citation downloads are disabled
the version tables (dataset and file) do not show contributor names
a configurable set of metadata fields (e.g. author, datasetContact, depositor, contributor, publication, and/or grantNumber) are also 'withheld'/their values not shown
metadata downloads are disabled

Anonymized access is intended for situtations such as making a dataset available for anonymized journal review.

Which issue(s) this PR closes:

Closes #1724

Special notes for your reviewer: The code includes a mechanism to add a boolean flag to private url role assignments to track whether the privateURL is limited to anonymized access along with UI/API code to request this type of url, and the changes in .xhtml files to not render the relevant page parts. An :AnonymizedFieldTypeNames property (comma separated list) allows configuration of which fields (and potentially ones in custom blocks) that should be anonymized. As of now, there is no default list of fields to hide - we could add a default set and/or only show the anonymized access url option if the property is set.

Note that with the addition of code to limit API access (to avoid leaking metadata), previewers that try to retrieve metadata (all our standard ones) also fail. Since the don't actually display metadata when used in an iframe, they don't need to make this call in that case. I have a modification to the previewer javascript that will handle this and restore the in inframe views. (Changing to use signed URLs could avoid this as well.) In any case - if the solution here is merged, we should make the previewer change as well. Alternately, a mechanism that removes anonymized fields from the json returned by various API calls could be developed instead.

Suggestions on how to test this: Create a PrivateURL with/without anonymization and verify that access using the URL works in both cases and that, in the anonymized case, the parts listed above are not included.

Does this PR introduce a user interface change? If mockups are available, please link/include them here: changes to the dialog to create a PrivateURL, changes in the dataset/file views to indicate the withheld info listed above.

Is there a release notes update needed for this change?: The existing of support for anonymized access to draft datasets could be announced. No default changes are required.

Additional documentation: documentation of the setting and user doc on creating an anonymized access url are included.

…_access

adam3smith · 2021-06-16T19:40:11Z

From presentation comments at DV community meeting:

Hi! When I downloaded the metadata (xml) file for the dataset, it shows your name as the author. Is there a way to also anonymize the metadata file?

qqmyers · 2021-06-16T19:49:29Z

I assume this via API? (Any metadata download buttons are turned off afaik). The API may still not be checking the flag on the private URL.

adam3smith · 2021-06-16T19:52:53Z

@qqmyers Datafile citation metadata is available & not anonymized

djbrooke · 2021-06-23T19:35:57Z

Thx @qqmyers for the PR. A developer should feel free to review the code here, but I believe you and @TaniaSchlatter were also discussing a walkthrough of the functionality, so I'll assign her as well. Happy to join in - this is a neat feature.

pdurbin · 2021-06-24T19:04:21Z

@qqmyers great demo! One thing you may not be aware of is that I wrote developer-oriented docs at https://github.com/IQSS/dataverse/blob/v4.5/src/main/java/edu/harvard/iq/dataverse/privateurl/package-info.java . I don't know if you're interested in updating them for this pull request but you should definitely feel free!

qqmyers · 2021-06-24T22:00:32Z

Per demo/review, I've added logic to disable the anonymized access button if there's a published version and added a popup title that only appears when the button is disabled:

(Note - the screen capture was from QDR where 'datasets' are 'data projects' - 'datasets' is used in this PR.

Also added the same check to the command so the API will also enforce this.
Moved errors strings in the command to the bundle as well for i18n
Added notes to package-info
Updated tests to not check exact error messages, added test for anonymized and released (should fail)

sekmiller · 2021-06-25T13:46:18Z

We need a sql script to add privateurlanonymizedaccess to RoleAssignment

sekmiller · 2021-06-25T13:55:51Z

src/main/java/edu/harvard/iq/dataverse/RoleAssignment.java

@@ -71,13 +71,21 @@
    @Column(nullable = true)
    private String privateUrlToken;

+    @Column(nullable = true)
+    private Boolean privateUrlAnonymizedAccess;
+


We need a sql update script for this new field:

ALTER TABLE RoleAssignment ADD COLUMN IF NOT EXISTS privateUrlAnonymizedAccess BOOLEAN;

UPDATE RoleAssignment SET privateUrlAnonymizedAccess = false WHERE privateUrlAnonymizedAccess IS NULL;

Good catch. I don't think the second line is needed (at least it works for QDR without it).

sekmiller · 2021-06-25T14:26:52Z

src/main/java/edu/harvard/iq/dataverse/api/AbstractApiBean.java

        if (privateUrlUser != null) {
+            if (privateUrlUser.hasAnonymizedAccess()) {
+                String pathInfo = httpRequest.getPathInfo();
+                if (!(pathInfo.startsWith("/access/datafile/") && !pathInfo.substring(17).contains("/"))) {


I'm wary of using "17" here. will this fail if we change the access path in any way? is there another way to express this?

"/access/datafile/".length() ?

TaniaSchlatter · 2021-06-25T15:15:28Z

Thanks for the demo and making the changes @qqmyers. Looks good to me. I took screenshots and will add them to the issue for reference.

sekmiller · 2021-06-25T16:43:38Z

src/main/java/propertyFiles/Bundle.properties

@@ -1736,6 +1741,7 @@ file.dataFilesTab.versions=Versions
 file.dataFilesTab.versions.headers.dataset=Dataset
 file.dataFilesTab.versions.headers.summary=Summary
 file.dataFilesTab.versions.headers.contributors=Contributors
+file.dataFilesTab.versions.headers.contributors=Contributor name(s) withheld


missing .withheld here

sekmiller

Looks good. just one missing tag in the bundle and merge the latest from dev

qqmyers added 4 commits May 27, 2021 12:49

initial anonymized access private url support

ece9e1f

also disable metadata downloads.

ec52ab1

handle above the fold fields

9204c45

Merge remote-tracking branch 'IQSS/develop' into IQSS/1724-anonymized…

00a0e64

…_access

qqmyers added 7 commits June 17, 2021 17:28

don't show citation downloads in file table menu/file page

f7055a3

typo

79ea29d

block most api calls for anonymized

1b6ae48

Send unauth error

9d2a679

missing import

d47f7de

add missing citation updates

2d22e26

Add 'Author name(s) withheld' to bundle

66dd55d

djbrooke added this to Review 🦁 in IQSS/dataverse (TO BE RETIRED / DELETED in favor of project 34) via automation Jun 22, 2021

djbrooke assigned TaniaSchlatter Jun 23, 2021

sekmiller self-assigned this Jun 24, 2021

qqmyers added 5 commits June 24, 2021 15:34

missing property

e211c55

disable button when dataset has been published already

560bebb

add anonymized request and released check, i18n messages

0828366

add to notes

603c6f1

update tests, add test, drop exact message checks

2d23fe7

sekmiller reviewed Jun 25, 2021

View reviewed changes

flyway script

f2e4158

sekmiller reviewed Jun 25, 2021

View reviewed changes

TaniaSchlatter removed their assignment Jun 25, 2021

sekmiller reviewed Jun 25, 2021

View reviewed changes

fixes per review

d671456

sekmiller approved these changes Jun 25, 2021

View reviewed changes

IQSS/dataverse (TO BE RETIRED / DELETED in favor of project 34) automation moved this from Review 🦁 to QA 🔎✅ Jun 25, 2021

djbrooke unassigned sekmiller Jun 28, 2021

kcondon self-assigned this Jun 28, 2021

qqmyers added 5 commits June 29, 2021 10:02

disable if setting isn't set

e808fb9

docs, disallow anonymizedaccess param when seting not set

3ec5223

bad cut/paste

0c91a34

add label

32213aa

extra .

a6f06ca

kcondon merged commit 57ab27e into IQSS:develop Jun 29, 2021

IQSS/dataverse (TO BE RETIRED / DELETED in favor of project 34) automation moved this from QA 🔎✅ to Done 🚀 Jun 29, 2021

djbrooke added this to the 5.6 milestone Jun 29, 2021

pdurbin mentioned this pull request Jun 30, 2021

get "create dataset" working again #7986 #7987

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Iqss/1724 anonymized access #7908

Iqss/1724 anonymized access #7908

qqmyers commented May 27, 2021 •

edited

adam3smith commented Jun 16, 2021

qqmyers commented Jun 16, 2021

adam3smith commented Jun 16, 2021

djbrooke commented Jun 23, 2021

pdurbin commented Jun 24, 2021

qqmyers commented Jun 24, 2021 •

edited

sekmiller commented Jun 25, 2021

sekmiller Jun 25, 2021

qqmyers Jun 25, 2021

sekmiller Jun 25, 2021

qqmyers Jun 25, 2021

TaniaSchlatter commented Jun 25, 2021

sekmiller Jun 25, 2021

sekmiller left a comment

Iqss/1724 anonymized access #7908

Iqss/1724 anonymized access #7908

Conversation

qqmyers commented May 27, 2021 • edited

adam3smith commented Jun 16, 2021

qqmyers commented Jun 16, 2021

adam3smith commented Jun 16, 2021

djbrooke commented Jun 23, 2021

pdurbin commented Jun 24, 2021

qqmyers commented Jun 24, 2021 • edited

sekmiller commented Jun 25, 2021

sekmiller Jun 25, 2021

Choose a reason for hiding this comment

qqmyers Jun 25, 2021

Choose a reason for hiding this comment

sekmiller Jun 25, 2021

Choose a reason for hiding this comment

qqmyers Jun 25, 2021

Choose a reason for hiding this comment

TaniaSchlatter commented Jun 25, 2021

sekmiller Jun 25, 2021

Choose a reason for hiding this comment

sekmiller left a comment

Choose a reason for hiding this comment

qqmyers commented May 27, 2021 •

edited

qqmyers commented Jun 24, 2021 •

edited