8878 5.11.1 hotfix

doc/release-notes/5.11.1-release-notes.md

@@ -0,0 +1,72 @@
+# Dataverse Software 5.11.1
+
+This is a bug fix release of the Dataverse Software. The .war file for v5.11 will no longer be made available and installations should upgrade directly from v5.10.1 to v5.11.1. To do so you will need **to follow the instructions for installing release 5.11 using the v5.11.1 war file**. (Note specifically the upgrade steps 6-9 from the 5.11 release note; most importantly, the ones related to the citation block and the Solr schema). **If you had previously installed v5.11** (no longer available), follow the simplified instructions below.
+
+## Release Highlights
+
+Dataverse Software 5.11 contains two critical issues that are fixed in this release.
+
+First, if you delete a file from a published version of a dataset that has restricted files, the file will be deleted from the file system (or S3) and lose its "owner id" in the database. For details, see Issue #8867.
+
+Second, if you are a superuser, it's possible to click "Delete Draft" and delete a published dataset if it has restricted files. For details, see #8845 and #8742.
+
+## Notes for Dataverse Installation Administrators
+
+### Identifying Datasets with Deleted Files
+
+If you have been running 5.11, check if any files show "null" for the owner id. The "owner" of a file is the parent dataset:
+
+```
+select * from dvobject where dtype = 'DataFile' and owner_id is null;
+```
+
+For any of these files, change the owner id to the database id of the parent dataset. In addition, the file on disk (or in S3) is likely gone. Look at the "storageidentifier" field from the query above to determine the location of the file then restore the file from backup.
+
+### Identifying Datasets Superusers May Have Accidentally Destroyed 
+
+Check the "actionlogrecord" table for DestroyDatasetCommand. While these "destroy" entries are normal when a superuser uses the API to destroy datasets, an entry is also created if a superuser has accidentally deleted a published dataset in the web interface with the "Delete Draft" button.
+
+## Complete List of Changes
+
+For the complete list of code changes in this release, see the [5.11.1 Milestone](https://github.com/IQSS/dataverse/milestone/105?closed=1) in GitHub.
+
+For help with upgrading, installing, or general questions please post to the [Dataverse Community Google Group](https://groups.google.com/forum/#!forum/dataverse-community) or email support@dataverse.org.
+
+## Installation
+
+If this is a new installation, please see our [Installation Guide](https://guides.dataverse.org/en/5.11.1/installation/). Please also contact us to get added to the [Dataverse Project Map](https://guides.dataverse.org/en/5.11.1/installation/config.html#putting-your-dataverse-installation-on-the-map-at-dataverse-org) if you have not done so already.
+
+## Upgrade Instructions
+
+0\. These instructions assume that you've already successfully upgraded from Dataverse Software 4.x to Dataverse Software 5 following the instructions in the [Dataverse Software 5 Release Notes](https://github.com/IQSS/dataverse/releases/tag/v5.0). After upgrading from the 4.x series to 5.0, you should progress through the other 5.x releases before attempting the upgrade to 5.11.1. **To upgrade from 5.10.1, follow the instructions for installing release 5.11 using the v5.11.1 war file**. If you had previously installed v5.11 (no longer available), follow the simplified instructions below.
+
+If you are running Payara as a non-root user (and you should be!), **remember not to execute the commands below as root**. Use `sudo` to change to that user first. For example, `sudo -i -u dataverse` if `dataverse` is your dedicated application user.
+
+In the following commands we assume that Payara 5 is installed in `/usr/local/payara5`. If not, adjust as needed.
+
+`export PAYARA=/usr/local/payara5`
+
+(or `setenv PAYARA /usr/local/payara5` if you are using a `csh`-like shell)
+
+1\. Undeploy the previous version.
+
+- `$PAYARA/bin/asadmin list-applications`
+- `$PAYARA/bin/asadmin undeploy dataverse<-version>`
+
+2\. Stop Payara and remove the generated directory
+
+- `service payara stop`
+- `rm -rf $PAYARA/glassfish/domains/domain1/generated`
+
+3\. Start Payara
+
+- `service payara start`
+
+4\. Deploy this version.
+
+- `$PAYARA/bin/asadmin deploy dataverse-5.11.1.war`
+
+5\. Restart Payara
+
+- `service payara stop`
+- `service payara start`

doc/sphinx-guides/source/conf.py

@@ -66,9 +66,9 @@
 # built documents.
 #
 # The short X.Y version.
-version = '5.11'
+version = '5.11.1'
 # The full version, including alpha/beta/rc tags.
-release = '5.11'
+release = '5.11.1'
 
 # The language for content autogenerated by Sphinx. Refer to documentation
 # for a list of supported languages.

doc/sphinx-guides/source/versions.rst

@@ -6,8 +6,8 @@ Dataverse Software Documentation Versions
 
 This list provides a way to refer to the documentation for previous versions of the Dataverse Software. In order to learn more about the updates delivered from one version to another, visit the `Releases <https://github.com/IQSS/dataverse/releases>`__ page in our GitHub repo.
 
-- 5.11
-
+- 5.11.1
+- `5.11 </en/5.11/>`__
 - `5.10.1 </en/5.10.1/>`__
 - `5.10 </en/5.10/>`__
 - `5.9 </en/5.9/>`__

modules/dataverse-parent/pom.xml

@@ -129,7 +129,7 @@
 
     <properties>
         <!-- This is a special Maven property name, do not change! -->
-        <revision>5.11</revision>
+        <revision>5.11.1</revision>
 
         <target.java.version>11</target.java.version>
         <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>

src/main/java/edu/harvard/iq/dataverse/DatasetPage.java

@@ -404,7 +404,7 @@ public Boolean isHasValidTermsOfAccess() {
                hasValidTermsOfAccess = true;
                return hasValidTermsOfAccess;
             } else {
-                hasValidTermsOfAccess = TermsOfUseAndAccessValidator.isTOUAValid(dataset.getEditVersion().getTermsOfUseAndAccess(), null);
+                hasValidTermsOfAccess = TermsOfUseAndAccessValidator.isTOUAValid(dataset.getLatestVersion().getTermsOfUseAndAccess(), null);
                 return hasValidTermsOfAccess;
             }
         }    
@@ -3416,6 +3416,15 @@ private void deleteFiles(List<FileMetadata> filesToDelete) {
             refreshSelectedFiles(filesToDelete);
         }
 
+        if (dataset.getThumbnailFile() != null) {
+            for (FileMetadata fmd : filesToDelete) {
+                if (fmd.getDataFile().equals(dataset.getThumbnailFile())) {
+                    dataset.setThumbnailFile(null);
+                    break;
+                }
+            }
+        }
+
         for (FileMetadata markedForDelete : filesToDelete) {
 
             if (markedForDelete.getId() != null) {
@@ -3439,9 +3448,7 @@ private void deleteFiles(List<FileMetadata> filesToDelete) {
                 // passed to the UpdateDatasetCommand. -- L.A. Aug 2017
 
                 FileMetadataUtil.removeFileMetadataFromList(workingVersion.getFileMetadatas(), markedForDelete);
-
                 FileMetadataUtil.removeDataFileFromList(newFiles, markedForDelete.getDataFile());
-                FileUtil.deleteTempFile(markedForDelete.getDataFile(), dataset, ingestService);
 
             }
         }

src/test/java/edu/harvard/iq/dataverse/export/OpenAireExporterTest.java

@@ -97,7 +97,7 @@ public void testValidateExportDataset() throws Exception {
         }
         InputStream xmlStream = new ByteArrayInputStream(byteArrayOutputStream.toByteArray());
         DocumentBuilderFactory factory = DocumentBuilderFactory.newInstance();
-        factory.setValidating(true);
+        //factory.setValidating(true);
         factory.setNamespaceAware(true);
         factory.setAttribute("http://java.sun.com/xml/jaxp/properties/schemaLanguage",
                 "http://www.w3.org/2001/XMLSchema");