Skip to content

Commit

Permalink
Refactor zabbix proxy (#1196)
Browse files Browse the repository at this point in the history
* Enable skipping zabbix_valid_server_versions check

This table is incomplete, and a hinderance for deployments to valid
combinations. Allow the user to bypass this by supplying

  `-e enable_version_check=false`

* Put zabbix_repo_deb_url in defaults

Instead of having a partial url in vars/Debian.yml, and appending to
it with additional info via set_fact, if zabbix_repo_deb_url is not
defined. Just supply it as a default which can be overriden by user.

Notes on Raspbian
There are very few ways do differentiate Raspbian from Debian with
ansible_facts. The only candidate seems to be ansible_facts.lsb.id.

The lsb section does not get filled in unless some packages are
installed. But luckily those packages come installed on Raspbian
systems. And we just default it back to ansible_facts.distribution
if lsb.id is not present. So we're gonna simplify and drop some
tasks.

Tested with ansible-core 2.13.13 on;

  2024-03-12-raspios-bullseye-armhf-lite
  2024-03-15-raspios-bookworm-armhf-lite

If this for some unforseen reason wouldn't work on older or newer
versions of raspbian, there's always the option of just overriding
the zabbix_repo_deb_url.

* Single common task to install zabbix-proxy

Install the zabbix-proxy-{{ zabbix_proxy_database }} from a single
common task.

RedHat can pin the minor version, and has a toggle to disable repo.
Debian can't pin minor version. The other debian options
cache_valid_time, default_release and force seem irrelevant.

We use the common package module, which is just a wrapper around
apt/yum, and use this construction;

  user_supplied_var | default(_calculated_var | default(omit))

to send additional parameters to the respective modules.

* Single common task to install zabbix-sql-scripts

We don't need additional tasks for installing zabbix-sql-scripts,
we can just tack them on to zabbix-proxy-{{ database }}, with the
when condition found only on the Debian side of things.

* Install MySQL dependencies from mysql task

It's cleaner, less conditional checking, and a single task for all
supported systems.

Remove PyMySQL installation via pip task and switch from mysqldb to
pymysql for all debian based systems. The only reason we're
installing any python/mysql dependencies at all is to use the
ansible community.mysql collection, which has a preference for
pymysql, and mysqldb as a backup [1].

Upgrade system-packages of pymysql with pip for a couple of
distributions. Upgrading system-packages with pip is generally a
bad idea, older versions of pymysql has issues with newer version
of mysql (>=8).

pymysql>=0.9.0,<0.10.0 fixes the issue with passwords missing
during login, but still has a problem setting
log_bin_trust_function_creators.

pymysql>=0.10.0,<0.11.0, fixes the remaining issue. We don't want
to stray to far off from the system defaults

Also, drop zabbix_python_prefix. It's all python3.

[1] https://github.com/ansible-collections/community.mysql/blob/main/plugins/module_utils/mysql.py#L21-L36

* Install PostgreSQL dependencies from postresql task

It's cleaner, less conditional checking, and a single task for all
supported systems. The list of packages also remains the same for
all supported systems within the os_family, so we can reduce the
lookup.

We're also trimming out the last of zabbix_python_prefix, and
always go for python3

The Debian systems had split installing the dependencies in two
tasks, one for the python dependency and one that gets triggered if
zabbix_proxy_install_database_client is true.

We're gonna reuse this variable, and bring it to RedHat systems,
and for mysql aswell in this commit.

And to bring it all back to one task, we use this construction;

- package:
    name: "{{ _dependencies | select | list }}"
  vars:
    _dependencies:
      - "{{ install_client | ternary('client-package', '') }}"
      - some-python-dependency

This will create a list of two items, the python-dependency, and
possibly an empty string. We use `| select` to filter out the empty
strings, and `| list` while strictly not necessary, was
historically appended in case the preceeding result ended up being
a generator.

* Install SQLite3 dependencies from sqlite3 task

Less conditional checking, and should feel familiar as both mysql
and postgresql do this now.

* Simplify logic for dbhost_run_install (mysql)

It was difficult to see what this code was supposed to achieve. The
most important variable zabbix_proxy_dbhost_run_install was hidden
behind two conditions of delegated_dbhost. But here we try to bring
it back to view.

if zabbix_proxy_dbhost_run_install is true, we want the code to be
delegated_to the zabbix_proxy_dbhost, otherwise we want to run it
from the zabbix_proxy.

However, we have a funny situation where zabbix_proxy_dbhost is
by default localhost, which means we'd still be on the
zabbix_proxy. We store this calculation in delegated_dbhost.

  if (dbhost == localhost) then zabbix_proxy else dbhost

The mysql tasks have an additional variable
zabbix_proxy_real_dbhost, which we try applying first and if not,
we default to whatever zabbix_proxy_dbhost_run_install wants.

* pgsql: Consolidate delegate and remote tasks

With a clear understanding of zabbix_proxy_dbhost_run_install, we
can now use it to determine whether or not we need to become the
postgres user, and rely upon the default(omit) construction for any
potentially provided or missing arguments to login_{user,pass,host}

* Refactor MySQL schema import

Similar to pgsql, we check for existing dbversion in a block, if
the query fails, we create the database in a rescue section. There
are a few things to note however.

The check and schema creation is not delegated. It happens from
zabbix_proxy. This is to verify that our dbuser can access the
dbname from zabbix_proxy and has all the permissions needed (after
creation) to alter tables between zabbix version upgrades.

For the schema creation we actually need some extra privileges, so
we read what they currently are, set them to what we need, import
our schema, and revert the extra privileges back to their original
state. The reverting happens in the `always:` section, so if any
other task should fail in schema creation (`rescue:` section) we
always revert the privileges before failing.

We also have to deal with varying paths to zabbix-proxy schema on
RedHat based systems.

  e.g. /usr/share/doc/zabbix-proxy-pgsql-X.Y.Z

And you don't necessarily know which version you're installing when
you just want the latest. The code to retain support for older
versions is already in place, by leveraging the ls_output_schema
for the legacy path.

* Refactor PostgreSQL schema import

We take advantage of the community.postgres modules, and do a query
for 'mandatory' in 'dbversion'. If the query fails, we're going to
assume the database has not been populated yet, and rescue the task
by populating it. Thereby alleviating the need for .done files.

While zabbix-server has predictable paths to the schemas, the
zabbix-proxy schema paths are versioned on RedHat based systems.

  e.g. /usr/share/doc/zabbix-proxy-pgsql-X.Y.Z

And you don't necessarily know which version you're installing when
you just want the latest. The code to retain support for older
versions is already in place, by leveraging the ls_output_schema
for the legacy path.

* Refactor SQLite3 schema import

Mostly reusing the code-style seen in the other database imports.
Simplifying the commands used for creation, while keeping support
for older style schemas (zabbix < 6). And the addition of setting a
secure mode (0600) to the database file.

* Simplify legacy tasks

* Prefix database task-files with initialize

It reflects more accurately what we're trying to achieve. Also
strip out zabbix_proxy_db_long, it serves no purpose anymore.

* Group similar directory tasks

* Drop zabbix 5.0 compatibility

This simplifies the code by removing zabbix-5.0 quirks/workarounds.

---------
  • Loading branch information
eb4x committed May 18, 2024
1 parent ea1edfb commit c263a6a
Show file tree
Hide file tree
Showing 19 changed files with 363 additions and 649 deletions.
16 changes: 7 additions & 9 deletions .github/workflows/proxy.yml
Original file line number Diff line number Diff line change
Expand Up @@ -40,12 +40,11 @@ jobs:
- v64
- v62
- v60
include:
- interpreter: python3
- interpreter: python
container: centos7
exclude:
- container: debian12
- container:
name: debian12
base_image: debian:12
os_family: debian
version: v62
- container: ubuntu2404
version: v62
Expand Down Expand Up @@ -81,10 +80,9 @@ jobs:

- name: Run role tests
run: >-
MY_MOLECULE_CONTAINER=${{ matrix.container }}
MY_MOLECULE_IMAGE=${{ matrix.container }}
MY_MOLECULE_CONTAINER=${{ matrix.container.name }}
MY_MOLECULE_IMAGE=${{ matrix.container.base_image }}
MY_MOLECULE_OS_FAMILY=${{ matrix.container.os_family }}
MY_MOLECULE_VERSION=${{ matrix.version }}
MY_MOLECULE_DATABASE=${{ matrix.database }}
MY_MOLECULE_INTERPRETER=${{ matrix.interpreter }}
MY_MOLECULE_DOCKER_COMMAND=${{ matrix.command }}
molecule test -s ${{ matrix.collection_role }}
3 changes: 3 additions & 0 deletions molecule/zabbix_proxy/Dockerfile.debian.j2
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
FROM {{ item.base_image }}
RUN apt-get update && DEBIAN_FRONTEND=noninteractive apt-get install -y python3-pip systemd-sysv
CMD ["/sbin/init"]
3 changes: 3 additions & 0 deletions molecule/zabbix_proxy/Dockerfile.redhat.j2
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
FROM {{ item.base_image }}
RUN yum install -y python3-pip
CMD ["/sbin/init"]
11 changes: 5 additions & 6 deletions molecule/zabbix_proxy/molecule.yml
Original file line number Diff line number Diff line change
Expand Up @@ -3,10 +3,11 @@ driver:
name: docker
platforms:
- name: zabbix-proxy-${MY_MOLECULE_VERSION:-v64}-${MY_MOLECULE_DATABASE:-mysql}-${MY_MOLECULE_CONTAINER:-rockylinux8}
image: geerlingguy/docker-${MY_MOLECULE_IMAGE:-rockylinux8}-ansible:latest
dockerfile: Dockerfile.${MY_MOLECULE_OS_FAMILY:-redhat}.j2
base_image: ${MY_MOLECULE_IMAGE:-rockylinux/rockylinux:8-ubi-init}
image: ${MY_MOLECULE_CONTAINER:-rockylinux8}
privileged: true
pre_build_image: true
command: ${MOLECULE_DOCKER_COMMAND:-""}
command: /sbin/init
networks:
- name: zabbix
volumes:
Expand All @@ -22,10 +23,8 @@ provisioner:
ANSIBLE_ROLES_PATH: $HOME/.ansible/collections/ansible_collections/community/zabbix/roles
inventory:
group_vars:
python3:
all:
ansible_python_interpreter: /usr/bin/python3
python:
ansible_python_interpreter: /usr/bin/python
v64:
zabbix_proxy_version: 6.4
v62:
Expand Down
16 changes: 0 additions & 16 deletions molecule/zabbix_proxy/prepare.yml
Original file line number Diff line number Diff line change
Expand Up @@ -95,19 +95,3 @@
dest: /etc/sudoers
line: "Defaults !requiretty"
state: present

- name: "Make sure the docs can be installed. (RedHat)"
ansible.builtin.lineinfile:
dest: /etc/yum.conf
line: "tsflags=nodocs"
state: absent
when:
- ansible_os_family == 'RedHat'

- name: "Make sure the docs can be installed. (Debian)"
ansible.builtin.lineinfile:
path: /etc/dpkg/dpkg.cfg.d/excludes
state: absent
regexp: "path-exclude=/usr/share/doc/*"
when:
- ansible_os_family != 'RedHat'
2 changes: 0 additions & 2 deletions roles/zabbix_proxy/defaults/main.yml
Original file line number Diff line number Diff line change
Expand Up @@ -43,8 +43,6 @@ zabbix_repo_yum_schema: https
zabbix_repo_yum_gpgcheck: 0
zabbix_repo_deb_url: "http://repo.zabbix.com/zabbix/{{ zabbix_proxy_version }}/{{ ansible_facts.lsb.id | default(ansible_facts['distribution']) | lower }}{% if ansible_facts['architecture'] == 'aarch64' and ansible_facts.lsb.id | default(ansible_facts['distribution']) in ['Debian', 'Ubuntu'] %}-arm64{% endif %}"
zabbix_repo_deb_component: main
zabbix_proxy_disable_repo:
- epel
zabbix_repo_yum:
- name: zabbix
description: Zabbix Official Repository - $basearch
Expand Down
106 changes: 0 additions & 106 deletions roles/zabbix_proxy/tasks/Debian.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,6 @@
- name: "Debian | Set short version name"
ansible.builtin.set_fact:
zabbix_short_version: "{{ zabbix_proxy_version | regex_replace('\\.', '') }}"
zabbix_python_prefix: "python{% if ansible_python_version is version('3', '>=') %}3{% endif %}"
zabbix_underscore_version: "{{ zabbix_proxy_version | regex_replace('\\.', '_') }}"
tags:
- always
Expand Down Expand Up @@ -102,108 +101,3 @@
become: true
tags:
- install

- name: "Debian | Installing zabbix-proxy-{{ zabbix_proxy_database }}"
ansible.builtin.apt:
pkg: "zabbix-proxy-{{ zabbix_proxy_database }}"
update_cache: true
cache_valid_time: 0
force: true
state: "{{ zabbix_proxy_package_state }}"
default_release: "{{ ansible_distribution_release }}"
environment:
http_proxy: "{{ zabbix_http_proxy | default(None) | default(omit) }}"
https_proxy: "{{ zabbix_https_proxy | default(None) | default(omit) }}"
register: is_zabbix_proxy_package_installed
until: is_zabbix_proxy_package_installed is succeeded
become: true
tags:
- install

- name: "Debian | Installing zabbix-sql-scripts"
ansible.builtin.apt:
pkg: zabbix-sql-scripts
state: "{{ zabbix_proxy_package_state }}"
update_cache: true
cache_valid_time: 0
default_release: "{{ ansible_distribution_release }}"
environment:
http_proxy: "{{ zabbix_http_proxy | default(None) | default(omit) }}"
https_proxy: "{{ zabbix_https_proxy | default(None) | default(omit) }}"
register: zabbix_proxy_package_sql_installed
until: zabbix_proxy_package_sql_installed is succeeded
when:
- zabbix_proxy_version is version('6.0', '>=')
become: true
tags:
- install

- name: "Debian | Install Ansible module dependencies"
ansible.builtin.apt:
name: "{{ zabbix_python_prefix }}-psycopg2"
state: present
environment:
http_proxy: "{{ zabbix_http_proxy | default(None) | default(omit) }}"
https_proxy: "{{ zabbix_https_proxy | default(None) | default(omit) }}"
register: zabbix_proxy_dependencies_installed
until: zabbix_proxy_dependencies_installed is succeeded
become: true
when:
- zabbix_proxy_database_creation
tags:
- install
- dependencies

- name: "Debian | Install Mysql Client package"
ansible.builtin.apt:
name: "{{ mysql_client_pkgs[ansible_distribution_major_version] }}"
state: present
environment:
http_proxy: "{{ zabbix_http_proxy | default(None) | default(omit) }}"
https_proxy: "{{ zabbix_https_proxy | default(None) | default(omit) }}"
register: zabbix_proxy_dependencies_installed
until: zabbix_proxy_dependencies_installed is succeeded
become: true
when:
- zabbix_proxy_database == 'mysql'
- zabbix_proxy_install_database_client
tags:
- install
- dependencies
- database

- name: "Debian | Install PostgreSQL Client package"
ansible.builtin.apt:
name: postgresql-client
state: present
environment:
http_proxy: "{{ zabbix_http_proxy | default(None) | default(omit) }}"
https_proxy: "{{ zabbix_https_proxy | default(None) | default(omit) }}"
register: are_zabbix_proxy_dependency_packages_installed
until: are_zabbix_proxy_dependency_packages_installed is succeeded
become: true
when:
- zabbix_proxy_database_creation or zabbix_proxy_database_sqlload
- zabbix_proxy_database == 'pgsql'
- zabbix_proxy_install_database_client
tags:
- install
- dependencies
- database

- name: "Debian | Install sqlite3"
ansible.builtin.apt:
name: sqlite3
state: present
environment:
http_proxy: "{{ zabbix_http_proxy | default(None) | default(omit) }}"
https_proxy: "{{ zabbix_https_proxy | default(None) | default(omit) }}"
register: are_zabbix_proxy_dependency_packages_installed
until: are_zabbix_proxy_dependency_packages_installed is succeeded
become: true
when:
- zabbix_proxy_database == 'sqlite3'
tags:
- install
- dependencies
- database
108 changes: 3 additions & 105 deletions roles/zabbix_proxy/tasks/RedHat.yml
Original file line number Diff line number Diff line change
Expand Up @@ -7,19 +7,6 @@
tags:
- always

- name: "RedHat | Define package without version"
ansible.builtin.set_fact:
zabbix_proxy_package: "zabbix-proxy-{{ zabbix_proxy_database }}"
cacheable: true
tags:
- always

- name: "RedHat | Set facts for Zabbix"
ansible.builtin.set_fact:
datafiles_path: "/usr/share/doc/zabbix-sql-scripts/{{ zabbix_proxy_db_long }}"
tags:
- always

- name: "RedHat | Make sure old file is absent"
ansible.builtin.file:
path: /etc/yum.repos.d/zabbix-supported.repo
Expand Down Expand Up @@ -47,99 +34,10 @@
tags:
- install

- name: Install packages for Zabbix Repository
block:
- name: "RedHat | Installing zabbix-proxy-{{ zabbix_proxy_database }}"
ansible.builtin.yum:
pkg: "{{ zabbix_proxy_package }}-{{ zabbix_proxy_version }}.{{ zabbix_proxy_version_minor }}"
state: "{{ zabbix_proxy_package_state }}"
disablerepo: "{{ zabbix_proxy_disable_repo | default(omit) }}"
environment:
http_proxy: "{{ zabbix_http_proxy | default(None) | default(omit) }}"
https_proxy: "{{ zabbix_https_proxy | default(None) | default(omit) }}"
become: true
register: is_zabbix_proxy_package_installed
until: is_zabbix_proxy_package_installed is succeeded

- name: "RedHat | Installing zabbix-sql-scripts"
ansible.builtin.yum:
pkg: "zabbix-sql-scripts-{{ zabbix_proxy_version }}.{{ zabbix_proxy_version_minor }}"
state: "{{ zabbix_proxy_package_state }}"
disablerepo: "{{ zabbix_proxy_disable_repo | default(omit) }}"
environment:
http_proxy: "{{ zabbix_http_proxy | default(None) | default(omit) }}"
https_proxy: "{{ zabbix_https_proxy | default(None) | default(omit) }}"
register: zabbix_proxy_sql_package_installed
until: zabbix_proxy_sql_package_installed is succeeded
become: true
tags:
- install

- name: "RedHat | Install Ansible PostgreSQL Client package"
- name: "RedHat | Add EPEL Repo (Centos 7 Only)"
when: ansible_facts['distribution_major_version'] == '7'
ansible.builtin.yum:
name: "{{ pgsql_depenencies[ansible_distribution_major_version] }}"
state: present
environment:
http_proxy: "{{ zabbix_http_proxy | default(None) | default(omit) }}"
https_proxy: "{{ zabbix_https_proxy | default(None) | default(omit) }}"
become: true
register: are_zabbix_proxy_pgsql_packages_installed
until: are_zabbix_proxy_pgsql_packages_installed is succeeded
when:
- zabbix_proxy_database_creation or zabbix_proxy_database_sqlload
- zabbix_proxy_database == 'pgsql'
tags:
- install
- database
- dependencies

- name: "RedHat | Install Mysql Client Package"
block:
- name: "RedHat | Add Mysql Repo (Centos 7 Only)"
ansible.builtin.yum_repository:
name: mariadb
description: MariaDB 10.8 CentOS repository list
file: mariadb
baseurl: "https://mirror.rackspace.com/mariadb/yum/10.11/centos{{ ansible_distribution_major_version }}-amd64"
gpgcheck: no
when: ansible_distribution_major_version == '7'

- name: "RedHat | Install Mysql Client package"
ansible.builtin.yum:
name: "{{ mysql_client_pkgs[ansible_distribution_major_version] }}"
state: installed
environment:
http_proxy: "{{ zabbix_http_proxy | default(None) | default(omit) }}"
https_proxy: "{{ zabbix_https_proxy | default(None) | default(omit) }}"
become: true
register: are_zabbix_proxy_mysql_packages_installed
until: are_zabbix_proxy_mysql_packages_installed is succeeded
when:
- zabbix_proxy_database_creation or zabbix_proxy_database_sqlload
- zabbix_proxy_install_database_client
- zabbix_proxy_database == 'mysql'
tags:
- install
- database
- dependencies

- name: "RedHat | Install sqlite3"
ansible.builtin.yum:
name:
- sqlite
state: present
environment:
http_proxy: "{{ zabbix_http_proxy | default(None) | default(omit) }}"
https_proxy: "{{ zabbix_https_proxy | default(None) | default(omit) }}"
register: zabbix_proxy_sqlite_packages_installed
until: zabbix_proxy_sqlite_packages_installed is succeeded
become: true
when:
- zabbix_proxy_database == 'sqlite3'
tags:
- install
- database
- dependencies
name: epel-release

- name: "Configure SELinux when enabled"
ansible.builtin.include_tasks: selinux.yml
Expand Down

0 comments on commit c263a6a

Please sign in to comment.