Skip to content

Commit

Permalink
#10151 add upgrade instructions
Browse files Browse the repository at this point in the history
  • Loading branch information
sekmiller committed Dec 7, 2023
1 parent d2427bd commit 3a13ac8
Show file tree
Hide file tree
Showing 2 changed files with 81 additions and 5 deletions.
81 changes: 81 additions & 0 deletions doc/release-notes/6.1-release-notes.md
Original file line number Diff line number Diff line change
Expand Up @@ -51,6 +51,13 @@ Previously, customization was possible by editing `Bundle.properties` but this i

For details, see https://dataverse-guide--10122.org.readthedocs.build/en/10122/installation/config.html#bag-info-txt

### Direct Upload setting added
A Dataverse installation can be now be configured to allow out-of-band upload by setting the `dataverse.files.<id>.upload-out-of-band` JVM option to `true`.

By default, Dataverse supports uploading files via the [add a file to a dataset](https://dataverse-guide--9003.org.readthedocs.build/en/9003/api/native-api.html#add-a-file-to-a-dataset) API. With S3 stores, a direct upload process can be enabled to allow sending the file directly to the S3 store (without any intermediate copies on the Dataverse server).

With the upload-out-of-band option enabled, it is also possible for file upload to be managed manually or via third-party tools, with the [Adding the Uploaded file to the Dataset](https://dataverse-guide--9003.org.readthedocs.build/en/9003/developers/s3-direct-upload-api.html#adding-the-uploaded-file-to-the-dataset) API call (described in the [Direct DataFile Upload/Replace API](https://dataverse-guide--9003.org.readthedocs.build/en/9003/developers/s3-direct-upload-api.html) page) used to add metadata and inform Dataverse that a new file has been added to the relevant store.


### Improvements in the dataset versions API
- optional pagination has been added to `/api/datasets/{id}/versions` that may be useful in datasets with a large number of versions
Expand Down Expand Up @@ -138,6 +145,7 @@ to generate updated versions.
See also #10060.

- Functionality has been added to help validate dataset JSON prior to dataset creation. There are two new API endpoints in this release. The first takes in a collection alias and returns a custom dataset schema based on the required fields of the collection. The second takes in a collection alias and a dataset JSON file and does an automated validation of the JSON file against the custom schema for the collection. In this release functionality is limited to JSON format validation and validating required elements. Future releases will address field types, controlled vocabulary, etc. (Issue #9464 and #9465)
- Validation has been added for the Geographic Bounding Box values in the Geospatial metadata block. This will prevent improperly defined bounding boxes from being created via the edit page or metadata imports. (issue 9547). This also fixes the issue where existing datasets with invalid geoboxes were quietly failing to get reindexed.

### Solr Improvements
- As of this release application-side support is added for the "circuit breaker" mechanism in Solr that makes it drop requests more gracefully when the search engine is experiencing load issues.
Expand Down Expand Up @@ -214,6 +222,79 @@ Upgrading requires a maintenance window and downtime. Please plan ahead, create

These instructions assume that you've already upgraded through all the 5.x releases and are now running Dataverse 6.0.

0\. These instructions assume that you are upgrading from 6.0. If you are running an earlier version, the only safe way to upgrade is to progress through the upgrades to all the releases in between before attempting the upgrade to 5.14.

If you are running Payara as a non-root user (and you should be!), **remember not to execute the commands below as root**. Use `sudo` to change to that user first. For example, `sudo -i -u dataverse` if `dataverse` is your dedicated application user.

In the following commands we assume that Payara 6 is installed in `/usr/local/payara6`. If not, adjust as needed.

`export PAYARA=/usr/local/payara6`

(or `setenv PAYARA /usr/local/payara6` if you are using a `csh`-like shell)

1\. Undeploy the previous version.

- `$PAYARA/bin/asadmin undeploy dataverse-6.0`

2\. Stop Payara and remove the generated directory

- `service payara stop`
- `rm -rf $PAYARA/glassfish/domains/domain1/generated`

3\. Start Payara

- `service payara start`

4\. Deploy this version.

- `$PAYARA/bin/asadmin deploy dataverse-6.1.war`

5\. Restart Payara

- `service payara stop`
- `service payara start`

6\. Update Geospatial Metadata Block (to improve validation of bounding box values)

- `wget https://github.com/IQSS/dataverse/releases/download/v6.1/geospatial.tsv`
- `curl http://localhost:8080/api/admin/datasetfield/load -H "Content-type: text/tab-separated-values" -X POST --upload-file @geospatial.tsv`

6a\. Update Citation Metadata Block (to make Alternative Title repeatable)

- `curl http://localhost:8080/api/admin/datasetfield/load -H "Content-type: text/tab-separated-values" -X POST --upload-file scripts/api/data/metadatablocks/citation.tsv`

7\. Upate Solr schema.xml to allow multiple Alternative Titles to be used. See specific instructions below for those installations without custom metadata blocks (7a) and those with custom metadata blocks (7b).

7a\. For installations without custom or experimental metadata blocks:

- Stop Solr instance (usually `service solr stop`, depending on Solr installation/OS, see the [Installation Guide](https://guides.dataverse.org/en/5.14/installation/prerequisites.html#solr-init-script))

- Replace schema.xml

- `cp /tmp/dvinstall/schema.xml /usr/local/solr/solr-9.3.0/server/solr/collection1/conf`

- Start Solr instance (usually `service solr start`, depending on Solr/OS)

7b\. For installations with custom or experimental metadata blocks:

- Stop Solr instance (usually `service solr stop`, depending on Solr installation/OS, see the [Installation Guide](https://guides.dataverse.org/en/5.14/installation/prerequisites.html#solr-init-script))

- There are 2 ways to regenerate the schema: Either by collecting the output of the Dataverse schema API and feeding it to the `update-fields.sh` script that we supply, as in the example below (modify the command lines as needed):
```
wget https://raw.githubusercontent.com/IQSS/dataverse/master/conf/solr/9.3.0/update-fields.sh
chmod +x update-fields.sh
curl "http://localhost:8080/api/admin/index/solr/schema" | ./update-fields.sh /usr/local/solr/solr-9.3.0/server/solr/collection1/conf/schema.xml
```
OR, alternatively, you can edit the following line in your schema.xml by hand as follows (to indicate that alternative title is now `multiValued="true"`):
```
<field name="alternativeTitle" type="text_en" multiValued="true" stored="true" indexed="true"/>
```
- Restart Solr instance (usually `service solr restart` depending on solr/OS)

8\. Run ReExportAll to update dataset metadata exports. Follow the directions in the [Admin Guide](http://guides.dataverse.org/en/5.14/admin/metadataexport.html#batch-exports-through-the-api).


## Backward Incompatibilities
- Since Alternative Title is repeatable now, old JSON APIs would not be compatible with a new version
- Several issues (#9952, #9953, #9957) where the Signposting output did not match the Signposting specification introduce backward-incompatibility,
Expand Down
5 changes: 0 additions & 5 deletions doc/release-notes/9002_allow_direct_upload_setting.md

This file was deleted.

0 comments on commit 3a13ac8

Please sign in to comment.