File Upload w/ Rsync UI/Workflow changes

[djbrooke]

Users should be able to upload files via rsync. When rsync is available, users will not be able to upload via http methods.

The technical underpinnings for rsync are present, but we need to decide on what the UI/UX should be for rsync uploads.

[pdurbin]

During the 2016-09-08 SBGrid Sprint Planning meeting ( https://docs.google.com/document/d/1wWSdKUOGA1L7UqFsgF3aOs8_9uyjnVpsPAxk7FObOOI/edit ) this issue was given an effort level of "5".

The code that @bmckinney demo'ed lives at https://github.com/bmckinney/dataverse-canonical/tree/dcm-beta and it works (deployed to https://dv.sbgrid.org currently) but would probably benefit from @scolapasta and @mheppler taking a look so I'm going to assign this issue to them and me and Bill to figure out what happens next to this code.

[pdurbin]

@bmckinney @djbrooke and I discussed this and I believe that what should happen next is for a branch to be pushed to the main IQSS repo so that @mheppler can easily switch to it and make UI improvements.

[TaniaSchlatter]

There is a plan for design/usability in Trello: https://trello.com/c/Nbte37k1/9-sbgrid-rsync-file-uploads-4-8. In terms of timing, is it accurate to say that this is planned to be presented at June's Dataverse Community Meeting as production-ready code or at least a prototype?

[djbrooke]

That's correct!

[landreev]

Fixed the issue 2. that @kcondon reported lat night (closing the popup via the "x" box), by adding an ajax event="close" listener.
The branch should be ready for merging into the rsync download branch.

[matthew-a-dunlap]

Two restAssured errors uncovered after merge of #3561 into develop:

• Email addresses should not be directly explosed
  • Looks like a regression of a bug fixed in Configuration option for excluding email addresses in exports #3443
  • https://build.hmdc.harvard.edu:8443/job/phoenix.dataverse.org-apitest-develop/edu.harvard.iq$dataverse/134/testReport/junit/edu.harvard.iq.dataverse.api/DatasetsIT/testExcludeEmail/
  • Line 417 in DatasetsIT.java (I think)
• dataFile.contentType is null/blank
  • https://build.hmdc.harvard.edu:8443/job/phoenix.dataverse.org-apitest-develop/edu.harvard.iq$dataverse/134/testReport/junit/edu.harvard.iq.dataverse.api/FilesIT/test_006_ReplaceFileGoodTabular/
  • Line 463 in FilesIT.java (I think)

[pdurbin]

I'll take a look at the failing REST Assured tests.

[pdurbin]

In ce0e97c I re-fixed #3443 about excluding email addresses from metadata export (they were being exposed), marked a method as deprecated to hopefully prevent regressions like this in the future, and added a "Privacy Considerations" section to the Installation Guide. I gave @matthew-a-dunlap a brain dump about the fix.

[pdurbin]

In 840c7a2 @matthew-a-dunlap and I fixed a bug in native file add that appears in develop as of 9d319bd where if you upload a .dat file you get a 500 error rather than a proper JSON response. The 500 was resulting from a null user attempting to create a lock:

Caused by: java.lang.IllegalArgumentException: Cannot lock a dataset for a null user
	at edu.harvard.iq.dataverse.DatasetLock.<init>(DatasetLock.java:117)
	at edu.harvard.iq.dataverse.DatasetLock.<init>(DatasetLock.java:104)
	at edu.harvard.iq.dataverse.DatasetServiceBean.addDatasetLock(DatasetServiceBean.java:540)

I also tested adding the same file via the GUI but was unable to reproduce the bug. I can only assume this means that the GUI and the API are taking separate execution paths, which is unfortunate. Anyway, should be fixed.