Skip to content

Add PostgreSQL support#54

Merged
datacharmer merged 17 commits intomasterfrom
add-postgresql-support
Apr 10, 2026
Merged

Add PostgreSQL support#54
datacharmer merged 17 commits intomasterfrom
add-postgresql-support

Conversation

@renecannao
Copy link
Copy Markdown
Collaborator

Summary

Port the employees database to PostgreSQL 16+ with identical schema, data, and integrity checks.

Files added

File Purpose
postgresql/employees.sql Schema DDL adapted for PG (ENUM→CHECK, stripped MySQLisms)
postgresql/load_employees_db.sh Shell wrapper: pipes dump files through sed 's/\//g'` into psql
postgresql/objects.sql Stored functions/procedures in PL/pgSQL
postgresql/test_employees_md5.sql MD5 integrity test with PL/pgSQL incremental hash
postgresql/test_employees_sha.sql SHA1 integrity test using pgcrypto
.github/workflows/ci-postgresql.yml CI on PostgreSQL 16 and 17 using dbdeployer

Key design decisions

  • No data duplication: the ~167MB dump files are shared. A sed pipe strips MySQL backticks on the fly.
  • Checksum compatibility: the PL/pgSQL helper compute_table_md5() replicates MySQL's @crc := MD5(CONCAT_WS('#', @crc, ...)) pattern via a FOR loop, aiming for identical checksums.
  • dbdeployer CI: uses dbdeployer deploy postgresql with binaries extracted from apt packages, same pattern as dbdeployer's own integration tests.

Test plan

  • CI PostgreSQL workflow passes on PG 16 and 17
  • Data loads correctly (row counts match MySQL)
  • MD5 integrity test passes (8 OK)
  • SHA1 integrity test passes (8 OK)
  • Stored procedures return correct results
  • Checksums match MySQL values (if not, debug type coercion differences)

Closes #53

Port the employees database to PostgreSQL with identical schema,
data, and integrity checks.

- postgresql/employees.sql: schema DDL adapted for PG
- postgresql/load_employees_db.sh: pipes dump files (strips
  MySQL backticks) into psql
- postgresql/objects.sql: stored functions/procedures in PL/pgSQL
- postgresql/test_employees_md5.sql: MD5 integrity test using
  PL/pgSQL incremental hash helper (replicates MySQL's @Crc pattern)
- postgresql/test_employees_sha.sql: SHA1 test using pgcrypto
- .github/workflows/ci-postgresql.yml: CI on PG 16 and 17
  using dbdeployer
- README: add PostgreSQL installation/testing section + badge

Closes #53
Each row tuple in INSERT INTO found_values was missing a closing
parenthesis, causing "syntax error at or near ;" on the last row.
The dbdeployer install script extracts the binary to the current
directory but does not add it to $PATH. Explicitly move it to
/usr/local/bin after extraction.

Closes #51
- Expand MySQL CI matrix: 8.0.42, 8.4.8, 9.0.1, 9.2.0, 9.5.0, 9.6.0
- Handle glibc2.28 tarballs (MySQL 9.6+ uses glibc2.28 instead of 2.17)
- Fix Percona version check: don't fail if VERSION() lacks 'percona'
- Update cache key to v2 to invalidate old glibc2.17-only caches

Closes #51
1. --commands flag: only exists in MySQL 9.5+, not in 9.0/9.2.
   The broad `9.*` glob incorrectly applied it to all 9.x.
   Now uses numeric version comparison (major >= 9 && minor >= 5).

2. MD5() removed in MySQL 9.6: skip the MD5 integrity test for
   MySQL >= 9.6 and rely on SHA integrity check instead.

3. Added a "Detect MySQL features" step that computes feature
   flags (needs_commands, has_md5) for use in conditional steps.

Closes #51
MySQL 9.6 removed both md5() and sha() functions. The SHA test was
running unconditionally, causing failures on 9.6. Now:
- Add has_sha feature detection alongside has_md5
- Make SHA test conditional on has_sha
- Add row count verification as fallback for 9.6+
- Bump cache key to v3 to fix corrupt cached tarballs
psql does not support MySQL-style -B (batch) or -N (skip column names)
flags. Use psql's -t (tuples only) instead.
PostgreSQL requires all non-aggregated columns in GROUP BY clause,
unlike MySQL which allows functional dependency. Add dept_name and
manager to GROUP BY.
PL/pgSQL procedures cannot return result sets directly with bare SELECT.
Convert to a function returning TABLE, which is the idiomatic PostgreSQL
approach. Update CI and README to use SELECT * FROM show_departments().
The RETURNS TABLE parameters create PL/pgSQL variables that conflict
with table column names. Qualify all column references with table
aliases (de.emp_no, de.dept_no, etc.).
SHA2() is available on all MySQL versions. This test uses SHA-256
instead of SHA-1, making it the primary integrity test for MySQL 9.6+
where md5() and sha() were removed. Expected values are placeholders
pending first CI run to compute them.
SHA-256 checksums captured from MySQL 8.0 CI run, verified consistent
across 9.0.1. Added PostgreSQL SHA-256 test using pgcrypto's
encode(digest(..., 'sha256'), 'hex'). Both use the same expected values
as MySQL, ensuring cross-database checksum compatibility.
…L details

- Add supported versions table covering MySQL, Percona, MariaDB, PostgreSQL
- Document MySQL 9.5 --commands flag and 9.6 md5/sha removal
- Explain SHA-256 as the recommended cross-database integrity test
- Expand PostgreSQL section: differences from MySQL, data integrity
  guarantees, SHA-256 compatibility between MySQL and PostgreSQL
Handle older MySQL tarballs (.tar.gz with glibc2.12) in addition to
newer ones (.tar.xz with glibc2.17/2.28). Rename "Supported Versions"
to "Tested Versions" since we don't guarantee support beyond what CI
covers.
@datacharmer datacharmer merged commit 616ea96 into master Apr 10, 2026
62 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add PostgreSQL support

2 participants