@jomtov, thanks for opening this issue. When we discuss more we may retitle as "upgrade to version..." before we pull this into a sprint.

@djbrooke FWIW dataverse-ansible quietly bumped the PG version to 10 this summer, as the version of python3-psycopg2 packaged with RHEL/CentOS8 isn't backwards-compatible with PG 9.6. I did some testing against 12 this summer; nothing major comes to memory.

• We'll try to handle this with a discussion in dv-tech or tech hours
• We should make sure to read the release notes as there may be incompatibilities

@djbrooke FWIW I just ran the Jenkins IT test suite against Postgres 13.1 using the most recent JDBC driver

1311 tests run, 0 errors, 0 failures, no SQL errors in server.log.*

I can leave the instance up if anybody wants to poke at it, or tear it down if you say.

thought: I would be glad and honored to be part of the discussion.

Everything appears to be working, with newer PostgreSQL versions, up to v. 13. Special thanks to @donsizemore for confirming that the Jenkins/RestAssured tests are passing under v.13.
Before I make a PR and put it up for review, this is what I'm planning to do, in case anyone has any objections/suggestions, etc.:

• Remove the language about v. 9.6 being "strongly recommended";
• I don't feel like it is necessary to require upgrading to the latest version. So I'm planning to make the guide say that the application has been tested with versions up to 13; and that we "recommend installing the latest version that's supported and available for your OS distribution".
• I believe there are legitimate cases where an installation may want to stay on an older version. For example, ours here. We are using Amazon RDS that's managed for us by another Harvard group. In the past when they would no longer support an older version, they would create a new instance for us and ask us to migrate the database. Presumably we'll need to do this again, but I don't feel like nagging them to make it happen ahead of their schedule.
• I'm updating the version of the jdbc driver we are bundling with the war file. (it's minor - we already have 42.2.18 in pom.xml, the latest available is 42.2.19).
• Another small thing - the Prerequisites section of the Installation Guide still says that our installer comes packaged with a jdbc driver that may need to be updated if they want to use a newer version of PostgreSQL - which is no longer true, now that the driver is bundled with the war file. Will remove.
• It's going to say that if anyone needs to upgrade an existing installation, the recommended and tested way is to export (via pg_dump) the old database, then re-import under newer version of PostgreSQL.

Any feedback is welcome.
@poikilotherm: it doesn't sound like any of this is going to affect your use cases... but I may be missing something.

re: removing the old info about how to update the driver: Is it worth still having a note about how to update the driver now that it is in the war? Presumably we can keep it nearly up-to-date in the new releases, but for older versions to run on newer postgres (or if the releases don't update every time postgress' driver does) - is there a place some guidance should go?

@qqmyers
I agree, it should be an "edit", not a "remove".
I still hope that few people would ever have a reason to actually do that... But we should mention it.

re: removing the old info about how to update the driver: Is it worth still having a note about how to update the driver now that it is in the war?

I'm fine with removing language to strong recommend a specific version but our guides are currently very specific about versions when it comes to actually installing postgres. It looks like yum install -y postgresql96-server for example. I assume this is staying the same. It's like... if you don't know which version to choose, here's a specific version mentioned in the guides.

I think we should update the version in Vagrant.

I think we should update the version in docker-aio too.

Overall, the plan sounds great.

I was going to update the yum command lines with something newer; and also to spell out clearly that they are just an example.
Not sure about touching vagrant and docker in the same pr.
(will commit my changes shortly)

@qqmyers I honestly got stuck a little yesterday trying to rewrite that paragraph, about the jdbc driver. It's easy to tell them to drop another version of the jar in the exploded directory under domain1/applications... but I had trouble thinking of how to start that sentence. Why would they ever want to do that. I mean, it's reasonable to believe that the version bundled with the 5.4 war file will be guaranteed to work with any PostgreSQL 13, even if more minor versions of the former are released during the lifecycle of 5.4. And we don't want them to go past 13 (even if they are stuck on 5.4 long enough that PostgreSQL 14 is readily available...).

And if there is some major security hole found in the current jdbdc driver - then we will have to contact all the installations, so we will supply the upgrade instructions as part of that...

Anyway, I ended up just dropping that paragraph. But if you can/are willing to put together a replacement, please do.

Thank you @landreev, this is exactly my argument why it's perfectly fine to include the driver in the WAR.

If someone really wants to change it for whatever reason, it's fairly easy to rebuild the WAR with a different version. We could provide a small guide for that, though.

(It's actually a Maven one liner, overriding the version from cmdline)

@poikilotherm

If someone really wants to change it for whatever reason, it's fairly easy to rebuild the WAR with a different version. We could provide a small guide for that, though.
...

I was thinking of a (very hypothetical) situation where a security issue is discovered in the current jdbdc driver, and we need to contact all the installations (who don't necessarily know how to build their own war files)... But if that ever happens, we will simply build a new war file ourselves, and ask everybody else to deploy it.
So no, I don't think there ever will be a situation where some installation admin needs to drop a replacement jdbc jar into the deployed application directory...