Psql bump #39

jburel · 2024-02-18T15:25:27Z

This PR ads support for psql 15+
version 15 and 16 have been added to the testing matrix
~~I had to introduce a new parameter to postgresql_users~~
~~The changes should be backward compatible~~
The changes are NOT backward compatible
i cherry-picked the commit from #33 and fix warning introduced

i will fix the other warnings in a follow-up PR to simplify the review of the required changes to support psql 15+

cc @technics3 mlukasik-dev

see https://www.postgresql.org/docs/release/15.0/ for background

…ersion 15 https://www.postgresql.org/docs/current/release-15.html

sbesson

Generally, adding support for PSQL 15/ 16 makes complete sense.

The conditional omission of stats_temp_directory from the templates matches the breaking removal of this property from PSQL 15 - https://www.postgresql.org/docs/15/release-15.html

I am confused on the contract of the new privileges item. Part of this confusion comes from the complexity of the existing logic in

ansible-role-postgresql/tasks/databases.yml

Lines 28 to 112 in e16285b

    
           # Setting privileges is complicated: 
        
           # - https://stackoverflow.com/a/39029296 
        
           # From https://www.postgresql.org/docs/9.6/static/sql-grant.html: 
        
           # 
        
           # "The key word PUBLIC indicates that the privileges are to be granted to 
        
           # all roles, including those that might be created later. PUBLIC can be 
        
           # thought of as an implicitly defined group that always includes all roles. 
        
           # Any particular role will have the sum of privileges granted directly to 
        
           # it, privileges granted to any role it is presently a member of, and 
        
           # privileges granted to PUBLIC." 
        
           # 
        
           # "There is no need to grant privileges to the owner of an object (usually 
        
           # the user that created it), as the owner has all privileges by default. 
        
           # (The owner could, however, choose to revoke some of their own privileges 
        
           # for safety.)" 
        
           # 
        
           # "PostgreSQL grants default privileges on some types of objects to PUBLIC. 
        
           # No privileges are granted to PUBLIC by default on tables, columns, schemas 
        
           # or tablespaces. For other types, the default privileges granted to PUBLIC 
        
           # are as follows: CONNECT and CREATE TEMP TABLE for databases; EXECUTE 
        
           # privilege for functions; and USAGE privilege for languages." 
        
           - name: postgres | revoke default permissions 
        
             postgresql_privs: 
        
               database: "{{ item.name }}" 
        
               privs: ALL 
        
               roles: PUBLIC 
        
               state: absent 
        
               type: database 
        
             when: "item.restrict | default(False)" 
        
             with_items: 
        
               - "{{ postgresql_databases }}" 
        
             changed_when: false 
        
           # Revoke the default permissions on the public schema 
        
           - name: postgres | revoke default schema permissions 
        
             postgresql_privs: 
        
               database: "{{ item.name }}" 
        
               obj: public 
        
               privs: ALL 
        
               roles: PUBLIC 
        
               state: absent 
        
               type: schema 
        
             when: "item.restrict | default(False)" 
        
             with_items: 
        
               - "{{ postgresql_databases }}" 
        
             changed_when: false 
        
           # The default public schema is owned by postgres, and since the PUBLIC 
        
           # privileges are revoked we must grant them back to the owner 
        
           - name: postgres | grant database owner public schema privileges 
        
             postgresql_privs: 
        
               database: "{{ item.name }}" 
        
               obj: public 
        
               privs: ALL 
        
               roles: "{{ item.owner }}" 
        
               state: present 
        
               type: schema 
        
             when: item.owner is defined 
        
             with_items: 
        
               - "{{ postgresql_databases }}" 
        
           - name: postgres | grant connect privileges 
        
             postgresql_privs: 
        
               database: "{{ item.1 }}" 
        
               privs: CONNECT 
        
               roles: "{{ item.0.user }}" 
        
               state: present 
        
               type: database 
        
             with_subelements: 
        
               - "{{ postgresql_users }}" 
        
               - databases 
        
           - name: postgres | grant usage privileges on default public schema 
        
             postgresql_privs: 
        
               database: "{{ item.1 }}" 
        
               objs: public 
        
               privs: USAGE 
        
               roles: "{{ item.0.user }}" 
        
               state: present 
        
               type: schema 
        
             with_subelements: 
        
               - "{{ postgresql_users }}" 
        
               - databases

. I assume the proposed changes are related to the public schema changes in PSQL 15 - see https://www.postgresql.org/docs/15/release-15.html. A few questions:

did Molecule tests fail without this change on PSQL 15+? if so, are these genuine failures?
, it would be useful to work through scenarios of how postgresql_users/postgresql_databases maps into PSQL privileges
do the new public schema restrictions in PSQL 15+ overlap with the contract of restrict: True? In that case, should we redefine the scope of this variable ?

jburel · 2024-02-18T21:52:54Z

The changes are related to the public schema.
The failure without the suggested changes

TASK [create test tables publicdb] *********************************************
  failed: [postgresql-15-u2204] (item=create table if not exists regular (text text primary key);) => {"ansible_loop_var": "item", "changed": true, "cmd": ["env", "PGPASSWORD=alice123", "psql", "-h", "localhost", "-U", "alice", "publicdb", "-c", "create table if not exists regular (text text primary key);"], "delta": "0:00:00.058269", "end": "2024-02-15 15:48:06.778143", "item": "create table if not exists regular (text text primary key);", "msg": "non-zero return code", "rc": 1, "start": "2024-02-15 15:48:06.719874", "stderr": "ERROR:  permission denied for schema public\nLINE 1: create table if not exists regular (text text primary key);\n                                   ^", "stderr_lines": ["ERROR:  permission denied for schema public", "LINE 1: create table if not exists regular (text text primary key);", "                                   ^"], "stdout": "", "stdout_lines": []}

Basically
With Postgresql 15, the default permissions for a public schema have been modified. Therefore, the correct permissions will have to be granted to alice for Postgresql 15.

Since the alice user is the one to create the database tables we need to have

GRANT ALL ON SCHEMA public TO alice;

and not USAGE as it is currently in the role but at the same time we do not have to grant to other users

The problem occurs when creating the public db.

jburel · 2024-02-18T22:23:13Z

Setting restrict to true for publicdb creation leads to the same error ERROR: permission denied for schema public

sbesson · 2024-02-19T08:59:38Z

With Postgresql 15, the default permissions for a public schema have been modified. Therefore, the correct permissions will have to be granted to alice for Postgresql 15.

I think my concern is about the validity of the scenario tested by Molecule. In all the use cases I am aware of, tables should be created by the database owner e.g . alice/secretdb test. Is there a use case where we want a non-owner to be granted creation privileges on the public schema?

Setting restrict to true for publicdb creation leads to the same error ERROR: permission denied for schema public

Exactly. So the way I see it, the second part of the restrict variable has no effect on PSQL 15+. I see two paths:

we update this role to maintain the current behavior. With the current proposal, I think we need additional explanation on the privileges variable. Additionally, we should link to CVE-2018-1058 since this reverts a CVE mitigation path.
we update this role to comply with the mitigation CVE-2018-1058 and match the PSQL 15+ changesi.e. we restrict privileges on the public schema across the board except for non database owners, non-superusers

jburel · 2024-02-19T15:02:45Z

Last set of commits:

removes the new parameter privilege
introduces a superuser for molecules tests
restricts by default the creation of table/schema in publicdb. This is a break change. This should match the changes introduced in psql 15+.

sbesson · 2024-02-19T22:27:19Z

Discussed with @jburel earlier today, the new proposed behavior makes sense to me and is inline with the breaking privileges changes made in PSQL 15+ but it would be useful to have at least another review from the OME team

The latest changes update the default to restrict: True but the Molecule tests are no longer testing restrict: False use case. Given the behavior would be PSQL version dependent, would we consider dropping restrict fully and make all databases restricted? Or keep restrict: False for backwards-compatibility?
The README should be updated accordingly

jburel · 2024-02-20T04:36:45Z

Considering that it is a security breach fixed in newer version of psql, I think it will make more sense to restrict all databases and drop the restrict flag

jburel · 2024-02-20T05:33:30Z

@pwalczysko @khaledk2 What do you think?

pwalczysko · 2024-02-20T10:48:25Z

README.md

@@ -8,13 +8,15 @@ Install upstream PostgreSQL server.

 Optionally creates users and databases.
 If you wish to use your distribution's packages then do not use this role.
+This role revokes default `PUBLIC` privileges from database and `public` schema for all supported versions of PostgreSQL.
+This is to be inline with the breaking privileges made in PostgreSQL 15.


Suggested change

This is to be inline with the breaking privileges made in PostgreSQL 15.

This is to be inline with the breaking privileges made in PostgreSQL 15.

breaking privileges ? Maybe breaking changes ?

pwalczysko · 2024-02-20T10:55:44Z

@pwalczysko @khaledk2 What do you think?

What will this change do with an already deployed psql database if I run the playbook with the postgresql role including this change on such already deployed psql DB ?
It says in the header of this PR The changes are NOT backward compatible.

pwalczysko · 2024-02-20T14:00:27Z

Test 1:

on omero-ci-upgrade testing server RHEL 9, with the postgres-14 installed and running by ansible with the postgresql role from this branch, run:

the playbook from https://github.com/openmicroscopy/management_tools/pull/1710 with the postgresql role installed from this branch and the postgres version set as 15. Result: the postgres-15 service does not start, cannot find anything in the journalctl or systemd status.
delete everything you can (/opt/omero, stop and disable the postgres and omero-server services, rerun the playbook as per Ad 1. above. Result: fails at the stage install omero

TASK [ome.omero_server : omero server | install omero] ********************************************
[WARNING]: Module remote_tmp /opt/omero/server/.ansible/tmp did not exist and was created with a
mode of 0700, this may cause issues when running as another user. To avoid this, create the
remote_tmp dir with the correct permissions manually
fatal: [134.36.4.3]: FAILED! => {"changed": true, "cmd": ["/opt/omero/server/venv3/bin/omego", "install", "--release", "5.6.10", "--sym", "OMERO.server", "--ice", "3.6", "--no-start", "--no-web", "--ignoreconfig", "--omerocli", "/opt/omero/server/venv3/bin/omero", "-qq", "--dbhost", "localhost", "--dbuser", "omero", "--dbname", "omero", "--dbpass", "omero", "--managedb", "--rootpass", "omero"], "delta": "0:00:26.107814", "end": "2024-02-20 13:55:41.233835", "msg": "non-zero return code", "rc": 1, "start": "2024-02-20 13:55:15.126021", "stderr": "OMERO.py version:\nOMERO.server version:\n5.6.10-ice36\n2024-02-20 13:55:37,583 [omego.extern] ERROR Failed [0.037 s]\n2024-02-20 13:55:37,584 [    omego.db] ERROR Non-zero return code\ncommand: psql -v ON_ERROR_STOP=on -d omero -h localhost -U omero -w -A -t -c SELECT currentversion, currentpatch FROM dbpatch ORDER BY id DESC LIMIT 1\nreturn code: 1\nstdout: b''\nstderr: b'ERROR:  relation \"dbpatch\" does not exist\\nLINE 1: SELECT currentversion, currentpatch FROM dbpatch ORDER BY id...\\n                                                 ^\\n'\n2024-02-20 13:55:41,195 [omego.extern] ERROR Failed [0.034 s]\nTraceback (most recent call last):\n  File \"/opt/omero/server/venv3/bin/omego\", line 33, in <module>\n    sys.exit(load_entry_point('omego==0.7.0', 'console_scripts', 'omego')())\n  File \"/opt/omero/server/venv3/lib64/python3.9/site-packages/omego/main.py\", line 48, in entry_point\n    main(\"omego\", items=[\n  File \"/opt/omero/server/venv3/lib64/python3.9/site-packages/yaclifw/framework.py\", line 188, in main\n    ns.func(ns)\n  File \"/opt/omero/server/venv3/lib64/python3.9/site-packages/omego/upgrade.py\", line 534, in __call__\n    UnixInstall(self.NAME, args)\n  File \"/opt/omero/server/venv3/lib64/python3.9/site-packages/omego/upgrade.py\", line 78, in __init__\n    self.handle_database()\n  File \"/opt/omero/server/venv3/lib64/python3.9/site-packages/omego/upgrade.py\", line 290, in handle_database\n    db.init()\n  File \"/opt/omero/server/venv3/lib64/python3.9/site-packages/omego/db.py\", line 121, in init\n    self.psql('-f', omerosql)\n  File \"/opt/omero/server/venv3/lib64/python3.9/site-packages/omego/db.py\", line 276, in psql\n    stdout, stderr = external.run('psql', args, capturestd=True, env=env)\n  File \"/opt/omero/server/venv3/lib64/python3.9/site-packages/omego/external.py\", line 82, in run\n    raise RunException(\nomego.external.RunException: Non-zero return code\ncommand: psql -v ON_ERROR_STOP=on -d omero -h localhost -U omero -w -A -t -f omero-20240220-135537-584658.sql\nreturn code: 3\nstdout: b'BEGIN\\n'\nstderr: b'psql:omero-20240220-135537-584658.sql:27: ERROR:  permission denied for schema public\\n'", "stderr_lines": ["OMERO.py version:", "OMERO.server version:", "5.6.10-ice36", "2024-02-20 13:55:37,583 [omego.extern] ERROR Failed [0.037 s]", "2024-02-20 13:55:37,584 [    omego.db] ERROR Non-zero return code", "command: psql -v ON_ERROR_STOP=on -d omero -h localhost -U omero -w -A -t -c SELECT currentversion, currentpatch FROM dbpatch ORDER BY id DESC LIMIT 1", "return code: 1", "stdout: b''", "stderr: b'ERROR:  relation \"dbpatch\" does not exist\\nLINE 1: SELECT currentversion, currentpatch FROM dbpatch ORDER BY id...\\n                                                 ^\\n'", "2024-02-20 13:55:41,195 [omego.extern] ERROR Failed [0.034 s]", "Traceback (most recent call last):", "  File \"/opt/omero/server/venv3/bin/omego\", line 33, in <module>", "    sys.exit(load_entry_point('omego==0.7.0', 'console_scripts', 'omego')())", "  File \"/opt/omero/server/venv3/lib64/python3.9/site-packages/omego/main.py\", line 48, in entry_point", "    main(\"omego\", items=[", "  File \"/opt/omero/server/venv3/lib64/python3.9/site-packages/yaclifw/framework.py\", line 188, in main", "    ns.func(ns)", "  File \"/opt/omero/server/venv3/lib64/python3.9/site-packages/omego/upgrade.py\", line 534, in __call__", "    UnixInstall(self.NAME, args)", "  File \"/opt/omero/server/venv3/lib64/python3.9/site-packages/omego/upgrade.py\", line 78, in __init__", "    self.handle_database()", "  File \"/opt/omero/server/venv3/lib64/python3.9/site-packages/omego/upgrade.py\", line 290, in handle_database", "    db.init()", "  File \"/opt/omero/server/venv3/lib64/python3.9/site-packages/omego/db.py\", line 121, in init", "    self.psql('-f', omerosql)", "  File \"/opt/omero/server/venv3/lib64/python3.9/site-packages/omego/db.py\", line 276, in psql", "    stdout, stderr = external.run('psql', args, capturestd=True, env=env)", "  File \"/opt/omero/server/venv3/lib64/python3.9/site-packages/omego/external.py\", line 82, in run", "    raise RunException(", "omego.external.RunException: Non-zero return code", "command: psql -v ON_ERROR_STOP=on -d omero -h localhost -U omero -w -A -t -f omero-20240220-135537-584658.sql", "return code: 3", "stdout: b'BEGIN\\n'", "stderr: b'psql:omero-20240220-135537-584658.sql:27: ERROR:  permission denied for schema public\\n'"], "stdout": "5.18.0", "stdout_lines": ["5.18.0"]

pwalczysko · 2024-02-20T14:05:55Z

I think the main point from the above is that the following line from the output strongly suggests that this change will break the install of OMERO via ansible playbooks even on a clean system where the postgres-14,15 and OMERO.server is not installed. I would think that the OMERO.server role is trying to execute psql cmds during install where the schema used is public - see below from the output of the ansible cmd.

psql -v ON_ERROR_STOP=on -d omero -h localhost -U omero -w -A -t -f omero-20240220-135537-584658.sql\nreturn code: 3\nstdout: b'BEGIN\\n'\nstderr: b'psql:omero-20240220-135537-584658.sql:27: ERROR: permission denied for schema public\\n'

pwalczysko

This is not working for me atm - it seems that it is crashing on install of omero step. It seems that the omero-server role is using the public schema.

pwalczysko · 2024-02-20T14:10:35Z

I can see that the tests here are not trying to install OMERO server which would be using this role - it seems that the problem here is indeed #39 (comment)

jburel · 2024-02-20T14:44:23Z

The error means that the user omero does not have the correct permissions. I think that should be fixed at the level of the playbook and not the role

pwalczysko · 2024-02-20T15:32:29Z

The error means that the user omero does not have the correct permissions. I think that should be fixed at the level of the playbook and not the role

I do not see any permissions setting possibilities inside the playbook.
Also note that the DBs created by the playbook are owned by user postgres.

    - role: ome.postgresql
      postgresql_databases:
      - name: omero
      postgresql_users:
      - user: omero
        password: xxx
        databases: [omero]

bash-5.1$ psql -l
                                  List of databases
   Name    |  Owner   | Encoding |   Collate   |    Ctype    |   Access privileges   
-----------+----------+----------+-------------+-------------+-----------------------
 omero     | postgres | UTF8     | en_US.UTF-8 | en_US.UTF-8 | =Tc/postgres         +
           |          |          |             |             | postgres=CTc/postgres+
           |          |          |             |             | omero=c/postgres
 postgres  | postgres | UTF8     | en_US.UTF-8 | en_US.UTF-8 | 
 template0 | postgres | UTF8     | en_US.UTF-8 | en_US.UTF-8 | =c/postgres          +
           |          |          |             |             | postgres=CTc/postgres
 template1 | postgres | UTF8     | en_US.UTF-8 | en_US.UTF-8 | =c/postgres          +
           |          |          |             |             | postgres=CTc/postgres
(4 rows)

jburel · 2024-02-20T15:49:23Z

    - role: ome.postgresql
      postgresql_databases:
      - name: omero
        owner: omero
      postgresql_users:
      - user: omero
        password: xxx

i.e. makes omero the owner of the omero db

pwalczysko · 2024-02-20T15:51:14Z

    - role: ome.postgresql
      postgresql_databases:
      - name: omero
        owner: omero
      postgresql_users:
      - user: omero
        password: xxx
        databases: [omero]

i.e. makes omero the owner of the omero db

But see below: the psql -l output is saying that the owner is still postgres.

bash-5.1$ psql -l
                                  List of databases
   Name    |  Owner   | Encoding |   Collate   |    Ctype    |   Access privileges   
-----------+----------+----------+-------------+-------------+-----------------------
 omero     | postgres | UTF8     | en_US.UTF-8 | en_US.UTF-8 | =Tc/postgres         +
           |          |          |             |             | postgres=CTc/postgres+
           |          |          |             |             | omero=c/postgres
 postgres  | postgres | UTF8     | en_US.UTF-8 | en_US.UTF-8 | 
 template0 | postgres | UTF8     | en_US.UTF-8 | en_US.UTF-8 | =c/postgres          +
           |          |          |             |             | postgres=CTc/postgres
 template1 | postgres | UTF8     | en_US.UTF-8 | en_US.UTF-8 | =c/postgres          +
           |          |          |             |             | postgres=CTc/postgres
(4 rows)

jburel · 2024-02-20T15:53:48Z

I made an adjustment to the previous command

pwalczysko · 2024-02-20T16:17:02Z

I made an adjustment to the previous command

This failed with

TASK [postgresql : postgres | grant connect privileges] *******************************************
fatal: [134.36.4.3]: FAILED! => {"msg": "could not find 'databases' key in iterated item '{'user': 'omero', 'password': 'omero'}'"}

@sbesson is here and suggesting a test where I have a postgres 14 installed and then I just bump to 15 (ignoring this PR). Doing just that to verify.

pwalczysko · 2024-02-21T09:44:11Z

I can install postgres 15 and server successfully.

But, in case there was a pre-existing running server with postgres 14, and I do following steps (stop and remove the postgres 14 service, remove the 14 folder of /var/lib/pgsql/, uninstall postgres, the OMERO.server will not restart, see below for Blitz log error:

[root@ome-ci-upgrade pgsql]# systemctl stop omero-server
[root@ome-ci-upgrade pgsql]# sudo -u postgres -s
bash-5.1$
bash-5.1$
bash-5.1$ psql -l
                              	List of databases
   Name	|  Owner   | Encoding |   Collate   |	Ctype	|   Access privileges   
-----------+----------+----------+-------------+-------------+-----------------------
 omero 	| postgres | UTF8 	| en_US.UTF-8 | en_US.UTF-8 | =Tc/postgres     	+
       	|      	|      	|         	|         	| postgres=CTc/postgres+
       	|      	|      	|         	|         	| omero=c/postgres
 postgres  | postgres | UTF8 	| en_US.UTF-8 | en_US.UTF-8 |
 template0 | postgres | UTF8 	| en_US.UTF-8 | en_US.UTF-8 | =c/postgres      	+
       	|      	|      	|         	|         	| postgres=CTc/postgres
 template1 | postgres | UTF8 	| en_US.UTF-8 | en_US.UTF-8 | =c/postgres      	+
       	|      	|      	|         	|         	| postgres=CTc/postgres
(4 rows)


bash-5.1$ pg_dump -Fc -f ome-learning-20240221-test.db.dump omero
bash-5.1$ pwd
/var/lib/pgsql
bash-5.1$ ls
14  ome-learning-20240221-test.db.dump
bash-5.1$ ls -lah
total 960K
drwx------.  4 postgres postgres  153 Feb 21 09:05 .
drwxr-xr-x. 56 root 	root 	4.0K Dec  4 18:22 ..
drwx------.  4 postgres postgres   51 Feb 21 08:41 14
drwxr-xr-x.  3 postgres postgres   17 Oct 19 14:12 .ansible
-rw-------.  1 postgres postgres 8.1K Feb 20 17:01 .bash_history
-rwx------.  1 postgres postgres  266 Feb 21 08:40 .bash_profile
-rw-r--r--.  1 postgres postgres 929K Feb 21 09:05 ome-learning-20240221-test.db.dump
-rw-------.  1 postgres postgres 2.1K Feb 20 17:00 .psql_history
-rw-------.  1 postgres postgres  781 Jan  8 10:03 .viminfo
bash-5.1$ mv ome-learning-20240221-test.db.dump
mv: missing destination file operand after 'ome-learning-20240221-test.db.dump'
Try 'mv --help' for more information.
bash-5.1$ mv ome-learning-20240221-test.db.dump /tmp/
[root@ome-ci-upgrade pgsql]# systemctl stop postgresql-14.service
[root@ome-ci-upgrade pgsql]#
[root@ome-ci-upgrade pgsql]#
[root@ome-ci-upgrade pgsql]# systemctl disable postgresql-14.service
Removed "/etc/systemd/system/multi-user.target.wants/postgresql-14.service".

Caused by: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'currentDetails' defined in class path resource [ome/services/sec-primitives.xml]: Cannot resolve reference to bean 'roles' while setting constructor argument; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'dbPatchCheck' defined in class path resource [ome/services/startup.xml]: Invocation of init method failed; nested exception is ome.conditions.InternalException: 
***************************************************************************************
Error connecting to database table dbpatch. You may need to bootstrap.
See https://docs.openmicroscopy.org/latest/omero/sysadmins/server-upgrade.html
***************************************************************************************
    at org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveReference(BeanDefinitionValueResolver.java:359)
    at org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveValueIfNecessary(BeanDefinitionValueResolver.java:108)
    at org.springframework.beans.factory.support.ConstructorResolver.resolveConstructorArguments(ConstructorResolver.java:648)
    at org.springframework.beans.factory.support.ConstructorResolver.autowireConstructor(ConstructorResolver.java:145)
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.autowireConstructor(AbstractAutowireCapableBeanFactory.java:1198)
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBeanInstance(AbstractAutowireCapableBeanFactory.java:1100)
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:511)
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:481)
    at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:312)
    at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:230)
    at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:308)
    at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:197)
    at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:761)
    at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:867)
    at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:543)
    at org.springframework.context.support.ClassPathXmlApplicationContext.<init>(ClassPathXmlApplicationContext.java:139)
    at org.springframework.context.support.ClassPathXmlApplicationContext.<init>(ClassPathXmlApplicationContext.java:105)
    at ome.system.OmeroContext.<init>(OmeroContext.java:97)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
    at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
    at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
    at org.springframework.beans.BeanUtils.instantiateClass(BeanUtils.java:142)
    ... 27 common frames omitted
Caused by: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'dbPatchCheck' defined in class path resource [ome/services/startup.xml]: Invocation of init method failed; nested exception is ome.conditions.InternalException: 
***************************************************************************************
Error connecting to database table dbpatch. You may need to bootstrap.
See https://docs.openmicroscopy.org/latest/omero/sysadmins/server-upgrade.html
***************************************************************************************
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1631)
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:553)
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:481)
    at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:312)
    at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:230)
    at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:308)
    at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:197)
    at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:297)
    at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:197)
    at org.springframework.beans.factory.support.ConstructorResolver.instantiateUsingFactoryMethod(ConstructorResolver.java:372)
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.instantiateUsingFactoryMethod(AbstractAutowireCapableBeanFactory.java:1178)
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBeanInstance(AbstractAutowireCapableBeanFactory.java:1072)
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:511)
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:481)
    at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:312)
    at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:230)
    at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:308)
    at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:197)
    at org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveReference(BeanDefinitionValueResolver.java:351)
    ... 49 common frames omitted
Caused by: ome.conditions.InternalException: 
...

pwalczysko · 2024-02-21T13:25:16Z

I can deploy with this role on
postgres 15 and 16 as upgrade from postgres 14 on the RHEL 9 testing server. The problem mentioned above #39 (comment) is caused by the playbook ignoring the necessity to initialize the DB before the sever start is attempted (as the server is already installed).

jburel · 2024-02-21T17:10:34Z

Merging and tagging as 5.4.0
Thanks

mlukasik-dev and others added 10 commits February 17, 2024 22:02

Remove obsolete server variable stats_temp_directory for PostgreSQL v…

170bce6

…ersion 15 https://www.postgresql.org/docs/current/release-15.html

use quote to avoid warning

72c7120

grant privileges on publicdb

52fe542

specify privilege

c52b7d3

adjust test for psql 15

887506f

add psql 15 to the matrix

602c3fa

add supported value and parameter

93e7ccd

remove whitespace

5ffd84d

add missing variable

28bca66

add support for psql 16

8a762e2

jburel requested review from sbesson and khaledk2 February 18, 2024 15:25

jburel mentioned this pull request Feb 18, 2024

Remove obsolete server variable stats_temp_directory for PostgreSQL version 15 #33

Closed

add psql 16 to README

de5d70d

sbesson reviewed Feb 18, 2024

View reviewed changes

jburel added 10 commits February 19, 2024 11:17

remove privilege

5996c62

add new tester user with superuser role

0da4f59

adjust the tests

64ab02d

check tester

22ead45

drop databases

6ca6988

remove privilege

dcb5e62

adjust tests

d3bbcac

remove version check

e300bbf

remove comment no longer valid

3cca161

check superuser

b0cf325

clean up

ff8fc23

jburel added 3 commits February 20, 2024 04:39

no need to set since it is the default

cbb0e48

remove restrict parameter

be79283

Update information about the role

07ec729

jburel requested a review from pwalczysko February 20, 2024 05:08

sbesson approved these changes Feb 20, 2024

View reviewed changes

pwalczysko reviewed Feb 20, 2024

View reviewed changes

adjust wording

5188ec7

pwalczysko requested changes Feb 20, 2024

View reviewed changes

pwalczysko approved these changes Feb 21, 2024

View reviewed changes

jburel merged commit a546449 into ome:master Feb 21, 2024
4 checks passed

sbesson mentioned this pull request Mar 1, 2024

Fix Molecule ome/ansible-role-omero-server#76

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Psql bump #39

Psql bump #39

jburel commented Feb 18, 2024 •

edited

Loading

sbesson left a comment

jburel commented Feb 18, 2024 •

edited

Loading

jburel commented Feb 18, 2024

sbesson commented Feb 19, 2024

jburel commented Feb 19, 2024 •

edited

Loading

sbesson commented Feb 19, 2024

jburel commented Feb 20, 2024

jburel commented Feb 20, 2024

pwalczysko Feb 20, 2024

jburel Feb 20, 2024

pwalczysko commented Feb 20, 2024

pwalczysko commented Feb 20, 2024

pwalczysko commented Feb 20, 2024

pwalczysko left a comment •

edited

Loading

pwalczysko commented Feb 20, 2024

jburel commented Feb 20, 2024 •

edited

Loading

pwalczysko commented Feb 20, 2024 •

edited

Loading

jburel commented Feb 20, 2024 •

edited

Loading

pwalczysko commented Feb 20, 2024

jburel commented Feb 20, 2024

pwalczysko commented Feb 20, 2024

pwalczysko commented Feb 21, 2024

pwalczysko commented Feb 21, 2024 •

edited

Loading

jburel commented Feb 21, 2024

	# Setting privileges is complicated:
	# - https://stackoverflow.com/a/39029296

	# From https://www.postgresql.org/docs/9.6/static/sql-grant.html:
	#
	# "The key word PUBLIC indicates that the privileges are to be granted to
	# all roles, including those that might be created later. PUBLIC can be
	# thought of as an implicitly defined group that always includes all roles.
	# Any particular role will have the sum of privileges granted directly to
	# it, privileges granted to any role it is presently a member of, and
	# privileges granted to PUBLIC."
	#
	# "There is no need to grant privileges to the owner of an object (usually
	# the user that created it), as the owner has all privileges by default.
	# (The owner could, however, choose to revoke some of their own privileges
	# for safety.)"
	#
	# "PostgreSQL grants default privileges on some types of objects to PUBLIC.
	# No privileges are granted to PUBLIC by default on tables, columns, schemas
	# or tablespaces. For other types, the default privileges granted to PUBLIC
	# are as follows: CONNECT and CREATE TEMP TABLE for databases; EXECUTE
	# privilege for functions; and USAGE privilege for languages."

	- name: postgres \| revoke default permissions
	postgresql_privs:
	database: "{{ item.name }}"
	privs: ALL
	roles: PUBLIC
	state: absent
	type: database
	when: "item.restrict \| default(False)"
	with_items:
	- "{{ postgresql_databases }}"
	changed_when: false

	# Revoke the default permissions on the public schema
	- name: postgres \| revoke default schema permissions
	postgresql_privs:
	database: "{{ item.name }}"
	obj: public
	privs: ALL
	roles: PUBLIC
	state: absent
	type: schema
	when: "item.restrict \| default(False)"
	with_items:
	- "{{ postgresql_databases }}"
	changed_when: false

	# The default public schema is owned by postgres, and since the PUBLIC
	# privileges are revoked we must grant them back to the owner
	- name: postgres \| grant database owner public schema privileges
	postgresql_privs:
	database: "{{ item.name }}"
	obj: public
	privs: ALL
	roles: "{{ item.owner }}"
	state: present
	type: schema
	when: item.owner is defined
	with_items:
	- "{{ postgresql_databases }}"

	- name: postgres \| grant connect privileges
	postgresql_privs:
	database: "{{ item.1 }}"
	privs: CONNECT
	roles: "{{ item.0.user }}"
	state: present
	type: database
	with_subelements:
	- "{{ postgresql_users }}"
	- databases

	- name: postgres \| grant usage privileges on default public schema
	postgresql_privs:
	database: "{{ item.1 }}"
	objs: public
	privs: USAGE
	roles: "{{ item.0.user }}"
	state: present
	type: schema
	with_subelements:
	- "{{ postgresql_users }}"
	- databases

	This is to be inline with the breaking privileges made in PostgreSQL 15.
	This is to be inline with the breaking privileges made in PostgreSQL 15.

Psql bump #39

Psql bump #39

Conversation

jburel commented Feb 18, 2024 • edited Loading

sbesson left a comment

Choose a reason for hiding this comment

jburel commented Feb 18, 2024 • edited Loading

jburel commented Feb 18, 2024

sbesson commented Feb 19, 2024

jburel commented Feb 19, 2024 • edited Loading

sbesson commented Feb 19, 2024

jburel commented Feb 20, 2024

jburel commented Feb 20, 2024

pwalczysko Feb 20, 2024

Choose a reason for hiding this comment

jburel Feb 20, 2024

Choose a reason for hiding this comment

pwalczysko commented Feb 20, 2024

pwalczysko commented Feb 20, 2024

pwalczysko commented Feb 20, 2024

pwalczysko left a comment • edited Loading

Choose a reason for hiding this comment

pwalczysko commented Feb 20, 2024

jburel commented Feb 20, 2024 • edited Loading

pwalczysko commented Feb 20, 2024 • edited Loading

jburel commented Feb 20, 2024 • edited Loading

pwalczysko commented Feb 20, 2024

jburel commented Feb 20, 2024

pwalczysko commented Feb 20, 2024

pwalczysko commented Feb 21, 2024

pwalczysko commented Feb 21, 2024 • edited Loading

jburel commented Feb 21, 2024

jburel commented Feb 18, 2024 •

edited

Loading

jburel commented Feb 18, 2024 •

edited

Loading

jburel commented Feb 19, 2024 •

edited

Loading

pwalczysko left a comment •

edited

Loading

jburel commented Feb 20, 2024 •

edited

Loading

pwalczysko commented Feb 20, 2024 •

edited

Loading

jburel commented Feb 20, 2024 •

edited

Loading

pwalczysko commented Feb 21, 2024 •

edited

Loading