Merge pull request #8310 from IQSS/8309-postgres-updates

Postgres 9.x EOL messaging, release notes updates for Postgres 10

doc/release-notes/8309-postgres-required.md

@@ -0,0 +1,37 @@
+## Notes for Dataverse Installation Administrators
+
+### PostgreSQL Version 10+ Required
+
+If you are still using PostgreSQL 9.X, now is the time to upgrade. PostgreSQL is now EOL (no longer supported, as of January 2022), and the Flyway library in the next release of the Dataverse Software will no longer work with versions prior to 10. 
+
+The Dataverse Software has been tested with PostgreSQL versions up to 13. The current stable version 13.5 is recommended. If that's not an option for reasons specific to your installation (for example, if PostgreSQL 13.5 is not available for the OS distribution you are using), any 10+ version should work.
+
+See the upgrade section for more information. 
+
+
+### PostgreSQL Upgrade (for the "Upgrade Instructions" section)
+
+The tested and recommended way of upgrading an existing database is as follows:
+
+- Export your current database with ``pg_dumpall``;
+- Install the new version of PostgreSQL; (make sure it's running on the same port, so that no changes are needed in the Payara configuration)
+- Re-import the database with ``psql``, as the user ``postgres``.
+
+It is strongly recommended to use the versions of the ``pg_dumpall`` and ``psql`` from the old and new versions of PostgreSQL, respectively. For example, the commands below were used to migrate a database running under PostgreSQL 9.6 to 13.5. Adjust the versions and the path names to match your environment. 
+
+Back up/export:
+
+``/usr/pgsql-9.6/bin/pg_dumpall -U postgres > /tmp/backup.sql``
+
+Restore/import:
+
+``/usr/pgsql-13/bin/psql -U postgres -f /tmp/backup.sql``
+
+When upgrading the production database here at Harvard IQSS we were able to go from version 9.6 all the way to 13.3 without any issues.
+
+You may want to try these backup and restore steps on a test server, to get an accurate estimate of how much downtime to expect with the final production upgrade. That of course will depend on the size of your database. 
+
+Consult the PostgreSQL upgrade documentation for more information, for example <https://www.postgresql.org/docs/13/upgrading.html#UPGRADING-VIA-PGDUMPALL>.
+
+
+

doc/sphinx-guides/source/developers/dev-environment.rst

@@ -94,23 +94,23 @@ To install Payara, run the following commands:
 Install PostgreSQL
 ~~~~~~~~~~~~~~~~~~
 
-For the past few release cycles much of the development has been done under PostgreSQL 9.6. While that version is known to be very stable, it is nearing its end-of-life (in Nov. 2021). The Dataverse Software has now been tested with versions up to 13 (13.2 is the latest released version as of writing this).  
+The Dataverse Software has been tested with PostgreSQL versions up to 13. PostgreSQL version 10+ is required. 
 
-On Mac, go to https://www.postgresql.org/download/macosx/ and choose "Interactive installer by EDB" option. Note that version 9.6 is used in the command line examples below, but the process will be identical for any version up to 13. When prompted to set a password for the "database superuser (postgres)" just enter "password".
+On Mac, go to https://www.postgresql.org/download/macosx/ and choose "Interactive installer by EDB" option. Note that version 13.5 is used in the command line examples below, but the process should be similar for other versions. When prompted to set a password for the "database superuser (postgres)" just enter "password".
 
 After installation is complete, make a backup of the ``pg_hba.conf`` file like this:
 
-``sudo cp /Library/PostgreSQL/9.6/data/pg_hba.conf /Library/PostgreSQL/9.6/data/pg_hba.conf.orig``
+``sudo cp /Library/PostgreSQL/13/data/pg_hba.conf /Library/PostgreSQL/13/data/pg_hba.conf.orig``
 
 Then edit ``pg_hba.conf`` with an editor such as vi:
 
-``sudo vi /Library/PostgreSQL/9.6/data/pg_hba.conf``
+``sudo vi /Library/PostgreSQL/13/data/pg_hba.conf``
 
-In the "METHOD" column, change all instances of "md5" to "trust". This will make it so PostgreSQL doesn't require a password.
+In the "METHOD" column, change all instances of "scram-sha-256" (or whatever is in that column) to "trust". This will make it so PostgreSQL doesn't require a password.
 
-In the Finder, click "Applications" then "PostgreSQL 9.6" and launch the "Reload Configuration" app. Click "OK" after you see "server signaled".
+In the Finder, click "Applications" then "PostgreSQL 13" and launch the "Reload Configuration" app. Click "OK" after you see "server signaled".
 
-Next, to confirm the edit worked, launch the "pgAdmin" application from the same folder. Under "Browser", expand "Servers" and double click "PostgreSQL 9.6". When you are prompted for a password, leave it blank and click "OK". If you have successfully edited "pg_hba.conf", you can get in without a password.
+Next, to confirm the edit worked, launch the "pgAdmin" application from the same folder. Under "Browser", expand "Servers" and double click "PostgreSQL 13". When you are prompted for a password, leave it blank and click "OK". If you have successfully edited "pg_hba.conf", you can get in without a password.
 
 On Linux, you should just install PostgreSQL using your favorite package manager, such as ``yum``. (Consult the PostgreSQL section of :doc:`/installation/prerequisites` in the main Installation guide for more info and command line examples). Find ``pg_hba.conf`` and set the authentication method to "trust" and restart PostgreSQL.
 

doc/sphinx-guides/source/installation/installation-main.rst

@@ -22,7 +22,7 @@ You should have already downloaded the installer from https://github.com/IQSS/da
 
 Unpack the zip file - this will create the directory ``dvinstall``.
 
-**Important:** The installer will need to use the PostgreSQL command line utility ``psql`` in order to configure the database. If the executable is not in your system PATH, the installer will try to locate it on your system. However, we strongly recommend that you check and make sure it is in the PATH. This is especially important if you have multiple versions of PostgreSQL installed on your system. Make sure the psql that came with the version that you want to use with your Dataverse installation is the first on your path. For example, if the PostgreSQL distribution you are running is installed in  /Library/PostgreSQL/9.6, add /Library/PostgreSQL/9.6/bin to the beginning of your $PATH variable. If you are *running* multiple PostgreSQL servers, make sure you know the port number of the one you want to use, as the installer will need it in order to connect to the database (the first PostgreSQL distribution installed on your system is likely using the default port 5432; but the second will likely be on 5433, etc.) Does every word in this paragraph make sense? If it does, great - because you definitely need to be comfortable with basic system tasks in order to install the Dataverse Software. If not - if you don't know how to check where your PostgreSQL is installed, or what port it is running on, or what a $PATH is... it's not too late to stop. Because it will most likely not work. And if you contact us for help, these will be the questions we'll be asking you - so, again, you need to be able to answer them comfortably for it to work. 
+**Important:** The installer will need to use the PostgreSQL command line utility ``psql`` in order to configure the database. If the executable is not in your system PATH, the installer will try to locate it on your system. However, we strongly recommend that you check and make sure it is in the PATH. This is especially important if you have multiple versions of PostgreSQL installed on your system. Make sure the psql that came with the version that you want to use with your Dataverse installation is the first on your path. For example, if the PostgreSQL distribution you are running is installed in  /Library/PostgreSQL/13, add /Library/PostgreSQL/13/bin to the beginning of your $PATH variable. If you are *running* multiple PostgreSQL servers, make sure you know the port number of the one you want to use, as the installer will need it in order to connect to the database (the first PostgreSQL distribution installed on your system is likely using the default port 5432; but the second will likely be on 5433, etc.) Does every word in this paragraph make sense? If it does, great - because you definitely need to be comfortable with basic system tasks in order to install the Dataverse Software. If not - if you don't know how to check where your PostgreSQL is installed, or what port it is running on, or what a $PATH is... it's not too late to stop. Because it will most likely not work. And if you contact us for help, these will be the questions we'll be asking you - so, again, you need to be able to answer them comfortably for it to work. 
 
 **It is no longer necessary to run the installer as root!**
 

doc/sphinx-guides/source/installation/prerequisites.rst

@@ -98,7 +98,7 @@ PostgreSQL
 Installing PostgreSQL
 =======================
 
-The application has been tested with PostgreSQL versions up to 13. We recommend installing the latest version that is available for your OS distribution. *For example*, to install PostgreSQL 13 under RHEL7/derivative::
+The application has been tested with PostgreSQL versions up to 13 and version 10+ is required. We recommend installing the latest version that is available for your OS distribution. *For example*, to install PostgreSQL 13 under RHEL7/derivative::
 
 	# yum install -y https://download.postgresql.org/pub/repos/yum/reporpms/EL-7-x86_64/pgdg-redhat-repo-latest.noarch.rpm
 	# yum makecache fast

scripts/installer/README_python.txt

@@ -39,15 +39,15 @@ in your PATH. If you have multiple versions of PostgresQL installed,
 make sure the version that you will be using with Dataverse is the
 first on your PATH. For example,
 
-   PATH=/usr/pgsql-9.6/bin:$PATH; export PATH
+   PATH=/usr/pgsql-13/bin:$PATH; export PATH
 
 Certain libraries and source include files, both for PostgresQL and
 Python, are also needed to compile the module. On
 RedHat/CentOS/etc. you may need to install the -devel packages, *for
 the specific versions* of PostgreSQL and Python you will be using. For
 example:
 
-	yum install postgresql96-devel
+	yum install postgresql13-devel
 	yum install python37-devel
 	etc.