Skip to content

Commit

Permalink
the release note - it's coming along. #9670
Browse files Browse the repository at this point in the history
  • Loading branch information
@landreev
landreev committed Jun 29, 2023
1 parent 0ecd7c6 commit cde956a
Showing 1 changed file with 125 additions and 123 deletions.
248 changes: 125 additions & 123 deletions doc/release-notes/5.14-release-notes.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -2,9 +2,132 @@

This release brings new features, enhancements, and bug fixes to the Dataverse software. Thank you to all of the community members who contributed code, suggestions, bug reports, and other assistance across the project.

You can jump directly to the Installation and/or Upgrade sections.
Please note that, as an experiment, the sections of this release note are organized in a different order. The Upgrade and Installation sections are at the top, with the sections highlighting new features and fixes in detail further down.

## Release Highlights, New Features and Use Case Scenarios
## Installation

If this is a new installation, please see our [Installation Guide](https://guides.dataverse.org/en/5.14/installation/). Please don't be shy about [asking for help](https://guides.dataverse.org/en/5.14/installation/intro.html#getting-help) if you need it!

After your installation has gone into production, you are welcome to add it to our [map of installations](https://dataverse.org/installations) by opening an issue in the [dataverse-installations](https://github.com/IQSS/dataverse-installations) repo.

## Upgrade Instructions

0\. These instructions assume that you are upgrading from 5.13. If you are running an earlier version, the only safe way to upgrade is to progress through the upgrades to all the releases in between before attempting the upgrade to 5.14.

If you are running Payara as a non-root user (and you should be!), **remember not to execute the commands below as root**. Use `sudo` to change to that user first. For example, `sudo -i -u dataverse` if `dataverse` is your dedicated application user.

In the following commands we assume that Payara 5 is installed in `/usr/local/payara5`. If not, adjust as needed.

`export PAYARA=/usr/local/payara5`

(or `setenv PAYARA /usr/local/payara5` if you are using a `csh`-like shell)

1\. Undeploy the previous version.

- `$PAYARA/bin/asadmin list-applications`
- `$PAYARA/bin/asadmin undeploy dataverse<-version>`

2\. Stop Payara and remove the generated directory

- `service payara stop`
- `rm -rf $PAYARA/glassfish/domains/domain1/generated`

3\. Start Payara

- `service payara start`

4\. Deploy this version.

- `$PAYARA/bin/asadmin deploy dataverse-5.14.war`

5\. Restart Payara

- `service payara stop`
- `service payara start`

6\. Update the Citation metadata block: (the update makes the field Series repeatable)

- `wget https://github.com/IQSS/dataverse/releases/download/v5.14/citation.tsv`
- `curl http://localhost:8080/api/admin/datasetfield/load -X POST --data-binary @citation.tsv -H "Content-type: text/tab-separated-values"`

If you are running an English-only installation, you are finished with the citation block. Otherwise, download the updated citation.properties file and place it in the [`dataverse.lang.directory`](https://guides.dataverse.org/en/5.14/installation/config.html#configuring-the-lang-directory); `/home/dataverse/langBundles` used in the example below.

- `wget https://github.com/IQSS/dataverse/releases/download/v5.14/citation.properties`
- `cp citation.properties /home/dataverse/langBundles`

7\. Replace Solr schema.xml to allow multiple series to be used. See specific instructions below for those installations without custom metadata blocks (1a) and those with custom metadata blocks (1b).

7a\. For installations without custom or experimental metadata blocks:

- Stop Solr instance (usually service solr stop, depending on Solr installation/OS, see the [Installation Guide](https://guides.dataverse.org/en/5.14/installation/prerequisites.html#solr-init-script)

- Replace schema.xml

- `cp /tmp/dvinstall/schema.xml /usr/local/solr/solr-8.11.1/server/solr/collection1/conf`

- Start solr instance (usually service solr start, depending on Solr/OS)

7b\. For installations with custom or experimental metadata blocks:

- Stop solr instance (usually service solr stop, depending on solr installation/OS, see the [Installation Guide](https://guides.dataverse.org/en/5.14/installation/prerequisites.html#solr-init-script)

- Edit the following lines in your schema.xml (to indicate that series and its components are now `multiValued="true"`):

`<field name="series" type="string" stored="true" indexed="true" multiValued="true"/>`
`<field name="seriesInformation" type="text_en" multiValued="true" stored="true" indexed="true"/>`
`<field name="seriesName" type="text_en" multiValued="true" stored="true" indexed="true"/>`

**TODO: why are we recommending editing the schema file by hand, instead of re-running the update script?**

- Restart Solr instance (usually service solr start, depending on solr/OS)

8\. Run ReExportAll to update dataset metadata exports

Follow the directions in the [Admin Guide](http://guides.dataverse.org/en/5.14/admin/metadataexport.html#batch-exports-through-the-api)

## New JVM Options and MicroProfile Config Options

The following PID provider settings are now available. See the section "Changes to PID Provider JVM Settings" below for more information.

- `dataverse.pid.datacite.mds-api-url`
- `dataverse.pid.datacite.rest-api-url`
- `dataverse.pid.datacite.username`
- `dataverse.pid.datacite.password`
- `dataverse.pid.handlenet.key.path`
- `dataverse.pid.handlenet.key.passphrase`
- `dataverse.pid.handlenet.index`
- `dataverse.pid.permalink.base-url`
- `dataverse.pid.ezid.api-url`
- `dataverse.pid.ezid.username`
- `dataverse.pid.ezid.password`

The following MicroProfile Config options have been added as part of [Signposting](https://signposting.org/) support. See the section "Signposting for Dataverse" for details.

- `dataverse.signposting.level1-author-limit`
- `dataverse.signposting.level1-item-limit`

The following JVM options are descrbed in the "Creating datasets with incomplete metadata through API" section below.

- `dataverse.api.allow-incomplete-metadata`
- `dataverse.ui.show-validity-filter`
- `dataverse.ui.allow-review-for-incomplete`

The following JVM/MicroProfile setting is for External Exporters, see "Mechanism Added for Adding External Exporters".

- `dataverse.spi.export.directory`

The JVM/MicroProfile setting for handling of support emails ("Contact Email Improvements")

- `dataverse.mail.support-email`
- `dataverse.mail.cc-support-on-contact-emails`



## Backward Incompatibilities

**TODO: (L.A.)**

## Detailed Release Highlights, New Features and Use Case Scenarios

### For Dataverse developers, support for running Dataverse in Docker (experimental)

Expand Down Expand Up @@ -220,7 +343,6 @@ Please see the [Update Metadata For a Dataset](https://guides.dataverse.org/en/l
- A date column has been added to the restricted file access request overview, indicating when the earliest request by that user was made. An issue was fixed where where the request list was not updated when a request was approved or rejected. (PR #9257)
- A feature flag called "api-session-auth" has been added temporarily to aid in the development of the new frontend (#9063) but will be removed once bearer tokens (#9229) have been implemented. There is a security risk (CSRF) in enabling this flag! Do not use it in production! For more information, see http://preview.guides.gdcc.io/en/develop/installation/config.html#feature-flags
- Changes made in v5.13 and v5.14 in multiple PRs to improve the embedded Schema.org metadata in dataset pages will only be propagated to the Schema.Org JSON-LD metadata export if a reExportAll() is done. (PR #9102)
- TODO: The 5.14 release notes should include the standard instructions for doing a reExportAll after updating the code, on account of the item above. (L.A.)
- It is now possible to write external vocabulary scripts that target a single child field in a metadata block. Example scripts are now available at https://github.com/gdcc/dataverse-external-vocab-support that can be configured to support lookup from the Research Orgnaization Registry (ROR) for the Author Affiliation Field and for the CrossRef Funding Registry (Fundreg) in the Funding Information/Agency field, both in the standard Citation metadata block. Application if these scripts to other fields, and the development of other scripts targetting child fields are now possible (PR #9402)
- A feature flag called "api-bearer-auth" has been added. This allows OIDC useraccounts to send authenticated API requests using Bearer Tokens. Note: This feature is limited to OIDC! For more information, see http://preview.guides.gdcc.io/en/develop/installation/config.html#feature-flags (PR #9591)
- An attempt will be made to extract a geospatial bounding box (west, south, east, north) from NetCDF and HDF5 files and then insert these values into the geospatial metadata block, if enabled. (#9541)
Expand All @@ -232,130 +354,10 @@ Please see the [Update Metadata For a Dataset](https://guides.dataverse.org/en/l
- Direct upload via the Dataverse UI will now support any algorithm configured via the `:FileFixityChecksumAlgorithm` setting. External apps using the direct upload API can now query Dataverse to discover which algorithm should be used. Sites that have been using an algorithm other than MD5 and direct upload and/or dvwebloader may want to use the `/api/admin/updateHashValues` call (see https://guides.dataverse.org/en/latest/installation/config.html?highlight=updatehashvalues#filefixitychecksumalgorithm) to replace any MD5 hashes on existing files. (PR #9482)


## New JVM Options and MicroProfile Config Options

**TODO: the section below is from 5.13; still needs to be updated for 5.14 (L.A.)**

The following JVM option is now available:

- `dataverse.personOrOrg.assumeCommaInPersonName` - the default is false

The following MicroProfile Config options are now available (these can be treated as JVM options):

- `dataverse.files.uploads` - alternative storage location of generated temporary files for UI file uploads
- `dataverse.api.signing-secret` - used by signed URLs
- `dataverse.solr.host`
- `dataverse.solr.port`
- `dataverse.solr.protocol`
- `dataverse.solr.core`
- `dataverse.solr.path`
- `dataverse.rserve.host`

The following existing JVM options are now available via MicroProfile Config:

- `dataverse.siteUrl`
- `dataverse.fqdn`
- `dataverse.files.directory`
- `dataverse.rserve.host`
- `dataverse.rserve.port`
- `dataverse.rserve.user`
- `dataverse.rserve.password`
- `dataverse.rserve.tempdir`

## Notes for Developers and Integrators

See the "Backward Incompatibilities" section below.

## Backward Incompatibilities

**TODO: (L.A.)**


## Complete List of Changes

For the complete list of code changes in this release, see the [5.14 milestone](https://github.com/IQSS/dataverse/milestone/108?closed=1) on GitHub.

## Installation

If this is a new installation, please see our [Installation Guide](https://guides.dataverse.org/en/5.14/installation/). Please don't be shy about [asking for help](https://guides.dataverse.org/en/5.14/installation/intro.html#getting-help) if you need it!

After your installation has gone into production, you are welcome to add it to our [map of installations](https://dataverse.org/installations) by opening an issue in the [dataverse-installations](https://github.com/IQSS/dataverse-installations) repo.



## Upgrade Instructions

0\. These instructions assume that you've already successfully upgraded from version 4.x to 5.0 of the Dataverse software following the instructions in the [release notes for version 5.0](https://github.com/IQSS/dataverse/releases/tag/v5.0). After upgrading from the 4.x series to 5.0, you should progress through the other 5.x releases before attempting the upgrade to 5.14.

If you are running Payara as a non-root user (and you should be!), **remember not to execute the commands below as root**. Use `sudo` to change to that user first. For example, `sudo -i -u dataverse` if `dataverse` is your dedicated application user.

In the following commands we assume that Payara 5 is installed in `/usr/local/payara5`. If not, adjust as needed.

`export PAYARA=/usr/local/payara5`

(or `setenv PAYARA /usr/local/payara5` if you are using a `csh`-like shell)

1\. Undeploy the previous version.

- `$PAYARA/bin/asadmin list-applications`
- `$PAYARA/bin/asadmin undeploy dataverse<-version>`

2\. Stop Payara and remove the generated directory

- `service payara stop`
- `rm -rf $PAYARA/glassfish/domains/domain1/generated`

3\. Start Payara

- `service payara start`

4\. Deploy this version.

- `$PAYARA/bin/asadmin deploy dataverse-5.14.war`

5\. Restart Payara

- `service payara stop`
- `service payara start`

6\. Update the Citation metadata block: (the update makes the field Series repeatable)

- `wget https://github.com/IQSS/dataverse/releases/download/v5.14/citation.tsv`
- `curl http://localhost:8080/api/admin/datasetfield/load -X POST --data-binary @citation.tsv -H "Content-type: text/tab-separated-values"`

If you are running an English-only installation, you are finished with the citation block. Otherwise, download the updated citation.properties file and place in the [`dataverse.lang.directory`](https://guides.dataverse.org/en/5.14/installation/config.html#configuring-the-lang-directory).

- `wget https://github.com/IQSS/dataverse/releases/download/v5.14/citation.properties`
- `cp citation.properties /home/dataverse/langBundles`

7\. Replace Solr schema.xml to allow multiple series to be used. See specific instructions below for those installations without custom metadata blocks (1a) and those with custom metadata blocks (1b).

Note: with this release support for indexing of the experimental workflow metadata block has been removed from the standard schema.xml.
If you are using the workflow metadata block be sure to follow the instructions in step 7b) below to maintain support for indexing workflow metadata.

7a\. For installations without custom or experimental metadata blocks:

- Stop Solr instance (usually service solr stop, depending on Solr installation/OS, see the [Installation Guide](https://guides.dataverse.org/en/5.14/installation/prerequisites.html#solr-init-script)

- Replace schema.xml

- `cp /tmp/dvinstall/schema.xml /usr/local/solr/solr-8.11.1/server/solr/collection1/conf`

- Start solr instance (usually service solr start, depending on Solr/OS)

7b\. For installations with custom or experimental metadata blocks:

- Stop solr instance (usually service solr stop, depending on solr installation/OS, see the [Installation Guide](https://guides.dataverse.org/en/5.14/installation/prerequisites.html#solr-init-script)

- Edit the following lines in your schema.xml (to indicate that series and its components are now `multiValued="true"`):

`<field name="series" type="string" stored="true" indexed="true" multiValued="true"/>`
`<field name="seriesInformation" type="text_en" multiValued="true" stored="true" indexed="true"/>`
`<field name="seriesName" type="text_en" multiValued="true" stored="true" indexed="true"/>`

**TODO: why are we recommending editing the schema file by hand, instead of re-running the update script?**

- Restart Solr instance (usually service solr start, depending on solr/OS)

### TODO: any optional upgrade steps to be added here

0 comments on commit cde956a

Please sign in to comment.