Skip to content

v6.12.1

Latest
Latest

Choose a tag to compare

View all tags
@pdurbin pdurbin released this 08 Oct 23:17
v6.12.1
bddc062
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

Dataverse 6.12.1

Please note: To read these instructions in full, please go to https://github.com/IQSS/dataverse/releases/tag/v6.12.1 rather than the list of releases, which will cut them off.

This release brings new features, enhancements, and bug fixes to Dataverse. Thank you to all of the community members who contributed code, suggestions, bug reports, and other assistance across the project!

Release Highlights

Highlights for Dataverse 6.12.1 include:

  • Performance improvements
  • New and improved APIs
  • Bug fixes
  • Security fixes
  • Platform update: Payara upgraded to 7.2026.9

Performance Improvements

Clicking "Edit Metadata", Language Metadata Field

It came to our attention that once you have created a dataset and then click "Edit Metadata" to add additional fields, it was taking 10 seconds or longer for the page to load. This was due to the thousands of languages that were being loaded for the Language metadata field. (The number of languages was greatly increased to the full ISO 639-3 list back in #10762 for Dataverse 6.4.)

As of this version, the page should load much faster when you click "Edit Metadata". The UI/UX for the Language field has been changed to load languages on demand. You should also notice a similar speed up when editing templates. See #11223 and #12617.

Faster Dataset and File Pages for Datasets with Many Files

The number of database queries required to load datasets and their files has been greatly reduced for large datasets: it no longer grows with the number of files, or with the number of variables of tabular files. An additional change improves the loading time of the file facets on the dataset page. The result is significantly faster dataset and file pages, and faster responses of the dataset and file listing APIs, for datasets with many files. See #12739 and #12740.

Bug Fixes

  • The "Cancel" button on the Guestbook Response dialogs did not close the dialog. This has been fixed. See #12205 and #12220.

Other Changes

API Updates

  • Search API now returns "isLinked" in the JSON response for datasets and collections that are linked. "isLinked" will also be returned in direct dataset and collection lookups. This attribute will only be included when the value is true. See #11845 and #12485.
  • When downloading a file citation it is now possible to specify the version of the dataset. See the guides, #12512, and #12711.

Security Updates

This release contains important security updates. If you are not receiving security advisories, please sign up by following the steps in the guides (email support@dataverse.org to ask for an invite). These advisories are sent through the dataverse-security Google Group, as announced on the mailing list.

Notes for Dataverse Installation Administrators

If you attempted a new installation of Dataverse 6.12 with PostgreSQL 16, it probably failed. (Upgrades of an existing Dataverse installation and an existing PostgreSQL 16 were not affected.) The Installation Guide has been updated to indicated that PostgreSQL 17+ is required for new installations. See the guides, #12724 and #12738.

Complete List of Changes

For the complete list of code changes in this release, see the 6.12.1 milestone in GitHub.

Getting Help

For help with upgrading, installing, or general questions please see getting help in the Installation Guide.

Installation

If this is a new installation, please follow our Installation Guide. Please don't be shy about asking for help if you need it!

Once you are in production, we would be delighted to update our map of Dataverse installations around the world to include yours! Please create an issue or email us at support@dataverse.org to join the club!

You are also very welcome to join the Global Dataverse Community Consortium (GDCC).

Upgrade Instructions

Upgrading requires a maintenance window and downtime. Please plan accordingly, create backups of your database, etc.

Note: These instructions assume that you are upgrading from the immediate previous version. That is to say, you've already upgraded through all the 6.x releases and are now running Dataverse 6.12. See tags on GitHub for a list of versions. If you are running an earlier version, the only supported way to upgrade is to progress through the upgrades to all the releases in between before attempting the upgrade to this version.

If you are running Payara as a non-root user (and you should be!), remember not to execute the commands below as root. By default, Payara runs as the dataverse user. In the commands below, we use sudo to run the commands as a non-root user.

Also, we assume that Payara is installed in /usr/local/payara7. If not, adjust as needed.

Payara has been upgraded to 7.2026.9. Dataverse 6.12 is not compatible with Payara 7.2026.9. You must run Dataverse 6.12.1 instead, as described below.

The instructions below describe the upgrade procedure based on moving your existing Payara 7.2026.8 domain directory into the new Payara 7.2026.9 distribution. We recommend this method because it is the easiest way to recreate your current configuration and preserve your data.

A quick note for admins connecting to their servers from MacOS systems: Once logged in, consider configuring your terminal as follows:

export TERM_PROGRAM=Apple_Terminal

You may see some junk characters added to the output of asadmin without it. Due, apparently, to a bug in a display library used in Payara 7.2026.9. (It appears to be harmless, functionality-wise.)

  1. Undeploy Dataverse, if deployed, using the unprivileged service account ("dataverse", by default).

    The new version of Payara is not compatible with previous versions of Dataverse, so you should undeploy the running Dataverse 6.12 war file first.

    sudo -u dataverse /usr/local/payara7/bin/asadmin list-applications

sudo -u dataverse /usr/local/payara7/bin/asadmin undeploy dataverse-6.12

  2. Stop Payara.

    sudo systemctl stop payara

  3. Move the current Payara directory out of the way. Version 7.2026.8 is used in the examples below. Adjust accordingly if you are upgrading from a different version.

    sudo mv /usr/local/payara7 /usr/local/payara7-2026.8

  4. Download the new Payara version 7.2026.9, and unzip it.

    curl -L -O https://nexus.payara.fish/repository/payara-community/fish/payara/distributions/payara/7.2026.9/payara-7.2026.9.zip

sudo unzip payara-7.2026.9.zip -d /usr/local/

  5. Set ownership of the payara distribution to root, making it read-only for the service account ("dataverse" by default).

    sudo chown -R root:root /usr/local/payara7

  6. Replace the brand new payara7/glassfish/domains/domain1 with your old, preserved domain1.

    sudo mv /usr/local/payara7/glassfish/domains/domain1 /usr/local/payara7/glassfish/domains/domain1_DIST

sudo mv /usr/local/payara7-2026.8/glassfish/domains/domain1 /usr/local/payara7/glassfish/domains/

  7. Set ownership of the domain for the service account ("dataverse" by default). Ownership should have been preserved during the move above, but this step assures that the domain is writable by the service account.

    sudo chown -R dataverse:dataverse /usr/local/payara7/glassfish/domains/domain1

  8. Remove the cache directories.

    sudo rm -rf /usr/local/payara7/glassfish/domains/domain1/generated/

sudo rm -rf /usr/local/payara7/glassfish/domains/domain1/osgi-cache/

  9. Start Payara.

    sudo systemctl start payara

  10. Deploy the Dataverse 6.12.1 war file.

    wget https://github.com/IQSS/dataverse/releases/download/v6.12.1/dataverse-6.12.1.war

sudo -u dataverse /usr/local/payara7/bin/asadmin deploy dataverse-6.12.1.war

  11. Update Solr schema.

    Due to changes in the Solr schema (the addition of "isLinked" in #12485), updating the Solr schema and reindexing is required.

    First, back up your existing schema.xml file.

    sudo cp /usr/local/solr/solr-9.8.0/server/solr/collection1/conf/schema.xml /usr/local/solr/solr-9.8.0/server/solr/collection1/conf/schema.xml.orig

    (Note that Docker-based installations use this directory: solr/data/data/collection1/conf/schema.xml.)

    Install the stock, default solr schema:

    wget https://raw.githubusercontent.com/IQSS/dataverse/v6.12.1/conf/solr/schema.xml

    sudo cp schema.xml /usr/local/solr/solr-9.8.0/server/solr/collection1/conf

    If you do not have any custom metadata blocks, you can go directly to the "Reload the Solr core" step below.
    If you do have custom metadata blocks, run the update-fields.sh script that we supply. The example below shows the default path for a non-Docker installation, but adjust the path as necessary.

    wget https://raw.githubusercontent.com/IQSS/dataverse/v6.12.1/conf/solr/update-fields.sh

    chmod +x update-fields.sh

    curl "http://localhost:8080/api/admin/index/solr/schema" | sudo ./update-fields.sh /usr/local/solr/solr-9.8.0/server/solr/collection1/conf/schema.xml

    Reload the Solr core

    curl "http://localhost:8983/solr/admin/cores?action=RELOAD&core=collection1"

  12. Reindex Solr.

    The only real change in the schema is the new "isLinked" field added in #12485, to mark linked datasets and collections in the output of the search API. If a full reindex is a quick task on your instance (for ex., it should only take a few minutes on an instance with a few hundred, or even thousands of datasets), we recommend that you do that, to be safe, via

    curl http://localhost:8080/api/admin/index

    However, if yours is one of the much larger instances where this takes a significant amount of CPU time, you may want to put more thought into it. Such as, consider whether your users will actually benefit from the field in the search API; and/or whether you actually have any linked objects in the repository in the first place. And if so, consider reindexing just the linked datasets and collections selectively.

Assets 4
Loading