Bug Report: database.php hardcoded 'encoding' => 'utf8' fails MISP 2.5.38+ diagnostics check

[hlijan]

Bug Report: database.php hardcoded 'encoding' => 'utf8' fails MISP 2.5.38+ diagnostics check

Summary

Starting with MISP v2.5.38, the Server Settings & Maintenance diagnostics page enforces that app/Config/database.php contains 'encoding' => 'utf8mb4 COLLATE utf8mb4_unicode_ci' as the database connection encoding.

The NUKIB image hardcodes 'encoding' => 'utf8' in its Jinja2 config template with no
mechanism to override it via environment variables, causing a permanent red diagnostic error
on NUKIB-based deployments running MISP ≥ 2.5.38.

---

Environment

Field	Value
MISP version	2.5.40
Deployment method	Docker Compose via Ansible
MariaDB	11.1

---

Steps to Reproduce

1. Deploy NUKIB MISP stack at MISP version 2.5.40
2. Navigate to Administration → Server Settings & Maintenance → Diagnostics
3. Observe the following red error:

"Incorrect database encoding setting: Your database connection is currently NOT set to
utf8mb4 COLLATE utf8mb4_unicode_ci. Please make sure the
'encoding' => 'utf8mb4 COLLATE utf8mb4_unicode_ci' is set in
/var/www/MISP/app/Config/database.php"

---

Root Cause Analysis

1. The NUKIB template hardcodes the wrong encoding

The Jinja2 template at Config/database.php (line 13) in the NUKIB image currently renders:

'encoding' => 'utf8',

2. MYSQL_SETTINGS seems cannot override this

MYSQL_SETTINGS populates the settings sub-array inside database.php. This sub-array is used exclusively for per-connection SET key=value SQL commands (e.g., SET time_zone="+00:00").

The PHP code that processes it is in CakePHP's from line 194 app/Lib/cakephp/lib/Cake/Model/Datasource/Database/Mysql.php::connect():

if (!empty($config['settings'])) {
    foreach ($config['settings'] as $key => $value) {
        $this->_execute("SET $key=$value");
    }
}

The top-level 'encoding' key (line 170) is entirely separate from settings and cannot be influenced by MYSQL_SETTINGS.

3. What MISP's diagnostic check actually verifies

From app/Model/Server.php::databaseEncodingDiagnostics() (line 4369) in MISP 2.5:

public function databaseEncodingDiagnostics(&$diagnostic_errors)
{
    if (!isset($this->getDataSource()->config['encoding']) ||
        strtolower($this->getDataSource()->config['encoding']) != 'utf8mb4 collate utf8mb4_unicode_ci') {
        $diagnostic_errors++;
        return false;
    }
    return true;
}

This reads config['encoding'] directly from the parsed database.php array and requires an exact case-insensitive match to utf8mb4 collate utf8mb4_unicode_ci. No other key or server variable satisfies this check.

4. Why the full string utf8mb4 COLLATE utf8mb4_unicode_ci could be safe to use

MISP's CakePHP fork at app/Lib/cakephp/lib/Cake/Model/Datasource/Database/Mysql.php
translates 'encoding' into a PDO init command verbatim:

if (!empty($config['encoding'])) {
    $flags[PDO::MYSQL_ATTR_INIT_COMMAND] = 'SET NAMES ' . $config['encoding'];
}

This produces:

SET NAMES utf8mb4 COLLATE utf8mb4_unicode_ci

This is valid MySQL/MariaDB syntax (tested) — the COLLATE clause is an accepted optional parameter of SET NAMES.

---

Proposed Fix Options

Option A — One-line fix

Change line 13 of Config/database.php from:

'encoding' => 'utf8',

to:

'encoding' => 'utf8mb4 COLLATE utf8mb4_unicode_ci',

Option B — Expose environment variable (Recommended, more flexible)

Add a new MYSQL_ENCODING variable to bin/misp_create_configs.py:

"MYSQL_ENCODING": Option(required=False, default="utf8mb4 COLLATE utf8mb4_unicode_ci"),

And update the Jinja2 template Config/database.php line 13 to:

'encoding' => '{{ MYSQL_ENCODING }}',

This follows the existing NUKIB pattern of exposing all configuration as environment variables, allows operators to override if needed, and defaults to the correct value for MISP 2.5.38+ with no changes required for existing deployments.

---

References

• MISP diagnostic check (source): app/Model/Server.php::databaseEncodingDiagnostics()
• CakePHP encoding handling (source): app/Lib/cakephp/lib/Cake/Model/Datasource/Database/Mysql.php line ~170
• Upstream MISP issue introducing the check: Bug: Incorrect database encoding configuration in MISP 2.5.38 MISP/MISP#10786
• MISP 2.5.38 release notes: https://www.misp-project.org/2026/05/26/misp.2.5.38.released.html
• MySQL SET NAMES ... COLLATE syntax: https://dev.mysql.com/doc/refman/8.4/en/charset-applications.html

[hlijan]

A possible solution was tested and verified in our environment — the GUI no longer reports the error. We replaced the database.php file with a modified version and changed the encoding line to: 'encoding' => 'utf8mb4 COLLATE utf8mb4_unicode_ci',.

# Verification of the connection from within the mysql container:
MariaDB [misp]> SET GLOBAL general_log_file='/tmp/query.log';
MariaDB [misp]> SET GLOBAL general_log='ON';
# restart misp container and wait a bit
MariaDB [misp]> \! grep -i "SET NAMES" /tmp/query.log
                   112 Query    SET NAMES utf8mb4 COLLATE utf8mb4_unicode_ci
                   118 Query    SET NAMES utf8mb4 COLLATE utf8mb4_unicode_ci
                   ... ... ## Every single MISP connection to MariaDB sends exactly SET NAMES utf8mb4 COLLATE utf8mb4_unicode_ci
MariaDB [misp]> SET GLOBAL general_log='OFF';