Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Server Instance Feature #1450

Merged
merged 8 commits into from Oct 10, 2023
Merged

Conversation

SimonHoenscheid
Copy link
Collaborator

@SimonHoenscheid SimonHoenscheid commented Jun 21, 2023

Summary

This PR adds server instances to the module.

Additional Context

  • Currently the server_instances define relies on the server class providing a baseline
  • instances with different versions are not possible, except for upgrade scenarios (e.g 13 was managed before, 14 is the new managed version)

Related Issues (if any)

@SimonHoenscheid SimonHoenscheid force-pushed the instances_define branch 4 times, most recently from c8e64a1 to 8b6bb15 Compare June 28, 2023 14:57
@SimonHoenscheid SimonHoenscheid force-pushed the instances_define branch 20 times, most recently from 0a95726 to b3cbf1e Compare July 6, 2023 12:32
@SimonHoenscheid SimonHoenscheid force-pushed the instances_define branch 4 times, most recently from 7191732 to d053e38 Compare August 2, 2023 07:00
@SimonHoenscheid SimonHoenscheid force-pushed the instances_define branch 8 times, most recently from ce41078 to 70170bc Compare September 25, 2023 15:50
@SimonHoenscheid
Copy link
Collaborator Author

I think this looks good at the moment. Can you maybe add an acceptance test to spin up multiple instances?

acceptance test added

instance directories is now a hash, update example
* postgresql::server::config_entry in postgresql::server::instance::config
* easier path management in postgresql::server::config_entry
* add taget to $pg_hba_conf_defaults
Copy link
Contributor

@Ramesh7 Ramesh7 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The code looks good but need more review from postgres experts.

Copy link
Collaborator

@smortex smortex left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks globally good. I have some concerns about the interface which feels very complex and repetitive but I don't think there is much we can do unfortunately.

Comment on lines +125 to +127
# stop default main instance
postgresql::server::service_ensure: "stopped"
postgresql::server::service_enable: false
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should go in the above profile rather than in Hiera.

Comment on lines +132 to +252
config_settings:
pg_hba_conf_path: "/opt/pgsql/data/13/test1/pg_hba.conf"
postgresql_conf_path: "/opt/pgsql/data/13/test1/postgresql.conf"
pg_ident_conf_path: "/opt/pgsql/data/13/test1/pg_ident.conf"
datadir: "/opt/pgsql/data/13/test1"
service_name: "postgresql@13-test1"
port: 5433
pg_hba_conf_defaults: false
service_settings:
service_name: "postgresql@13-test1"
service_status: "systemctl status postgresql@13-test1.service"
service_ensure: "running"
service_enable: true
initdb_settings:
auth_local: "peer"
auth_host: "md5"
needs_initdb: true
datadir: "/opt/pgsql/data/13/test1"
encoding: "UTF-8"
lc_messages: "en_US.UTF8"
locale: "en_US.UTF8"
data_checksums: false
group: "postgres"
user: "postgres"
username: "ins_test1"
config_entries:
authentication_timeout:
value: "1min"
comment: "a test"
log_statement_stats:
value: "off"
autovacuum_vacuum_scale_factor:
value: 0.3
databases:
testdb1:
encoding: "UTF8"
locale: "en_US.UTF8"
owner: "dba_test1"
testdb2:
encoding: "UTF8"
locale: "en_US.UTF8"
owner: "dba_test1"
roles:
"ins_test1":
superuser: true
login: true
"dba_test1":
createdb: true
login: true
"app_test1":
login: true
"rep_test1":
replication: true
login: true
"rou_test1":
login: true
pg_hba_rules:
"local all INSTANCE user":
type: "local"
database: "all"
user: "ins_test1"
auth_method: "peer"
order: 1
"local all DB user":
type: "local"
database: "all"
user: "dba_test1"
auth_method: "peer"
order: 2
"local all APP user":
type: "local"
database: "all"
user: "app_test1"
auth_method: "peer"
order: 3
"local all READONLY user":
type: "local"
database: "all"
user: "rou_test1"
auth_method: "peer"
order: 4
"remote all INSTANCE user PGADMIN server":
type: "host"
database: "all"
user: "ins_test1"
address: "192.168.22.131/32"
auth_method: "md5"
order: 5
"local replication INSTANCE user":
type: "local"
database: "replication"
user: "ins_test1"
auth_method: "peer"
order: 6
"local replication REPLICATION user":
type: "local"
database: "replication"
user: "rep_test1"
auth_method: "peer"
order: 7
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These 120 lines of boilerplate are scary 😱

I guess that if it was trivial it would have been done, but just to be sure, we have no way to skip some parameters that feel redundant (e.g. pass a datadir and infer the path to config files, generate the service name based on the instance name, etc)? That would require to change a lot of parameter default values and use the value of previous parameters instead of taking a value directly from params.pp?

"/opt/pgsql/log/13":
ensure: directory
"/opt/pgsql/log/13/test1":
ensure: directory
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we place the directory names in an array? This would reduce the number of lines.

Another option would be to make use of the dirtree module and the dirtree type/provider inside the instance defined type.
Using dirtree option should be done in a different PR as this will raise different discussion.

"/opt/pgsql/log/13/test1":
ensure: directory
config_settings:
pg_hba_conf_path: "/opt/pgsql/data/13/test1/pg_hba.conf"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here we have to replicate the version in many occasions.
Maybe version should become a hiera data, so it can be reused where needed.
Hardcoding version might make sense, if one is able to install multiple postgresql versions in parallel.

@Ramesh7 Ramesh7 merged commit ef2e9bc into puppetlabs:main Oct 10, 2023
38 checks passed

Puppet::Type.type(:postgresql_conf).provide(:ruby) do
desc 'Set keys, values and comments in a postgresql config file.'
confine kernel: 'Linux'
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(re-adding this comment because I was not reporting it where I though)

This seems to break the module on all non-linux OS:

root@agrajag /usr/home/romain # facter os
{
  architecture => "amd64",
  family => "FreeBSD",
  hardware => "amd64",
  name => "FreeBSD",
  release => {
    branch => "RC4",
    full => "14.0-RC4",
    major => "14",
    minor => "0"
  }
}
root@agrajag /usr/home/romain # puppet agent -t -E romain --tags Postgresql
[...]
Notice: /Stage[main]/Postgresql::Server::Service/Postgresql::Server::Instance::Service[main]/Anchor[postgresql::server::service::begin::main]: Dependency Postgresql_conf[port_for_instance_main] has failures: true
Notice: /Stage[main]/Postgresql::Server::Service/Postgresql::Server::Instance::Service[main]/Anchor[postgresql::server::service::begin::main]: Dependency Postgresql_conf[password_encryption_for_instance_main] has failures: true
Notice: /Stage[main]/Postgresql::Server::Service/Postgresql::Server::Instance::Service[main]/Anchor[postgresql::server::service::begin::main]: Dependency Postgresql_conf[data_directory_for_instance_main] has failures: true
[...]
Error: Could not find a suitable provider for postgresql_conf

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet