Skip to content

Add @fedify/mysql package with MysqlKvStore#597

Merged
dahlia merged 17 commits intofedify-dev:mainfrom
dahlia:mysql
Mar 4, 2026
Merged

Add @fedify/mysql package with MysqlKvStore#597
dahlia merged 17 commits intofedify-dev:mainfrom
dahlia:mysql

Conversation

@dahlia
Copy link
Member

@dahlia dahlia commented Mar 4, 2026

Summary

Closes #585

This pull request adds the @fedify/mysql package, which provides a MysqlKvStore class implementing the KvStore interface backed by MySQL or MariaDB via the mysql2 driver.

New package: @fedify/mysql

API

import { createFederation } from "@fedify/fedify";
import { MysqlKvStore } from "@fedify/mysql";
import mysql from "mysql2/promise";

const pool = mysql.createPool("mysql://user:pass@localhost/db");
const federation = createFederation<void>({
  kv: new MysqlKvStore(pool),
  // ...
});

Implementation notes

  • Keys (KvKey) are serialized via JSON.stringify() into a VARCHAR(768) column with utf8mb4_bin collation for case-sensitive matching.
  • Values are stored as a native JSON column; mysql2 parses them back to JS objects automatically on read.
  • TTL is stored as an absolute DATETIME(6) computed at write time (DATE_ADD(NOW(6), INTERVAL ? SECOND)), and expired rows are filtered out on read.
  • list() uses a LIKE prefix scan (with proper escaping of %, _, and \ characters).
  • cas() (compare-and-swap) uses SELECT ... FOR UPDATE inside a transaction for atomicity. Relies on InnoDB REPEATABLE READ (the default) for gap locking on missing rows.
  • The table is created lazily via initialize() and can be dropped with drop().
  • Requires MySQL 8.0+ or MariaDB 10.6+.

Other changes

  • CI: Added a MySQL 8 service container to all three test jobs (test, test-node, test-bun). Integration tests run when the MYSQL_URL environment variable is set.
  • Docs: Added MysqlKvStore section to docs/manual/kv.md, and updated federation.md, basics.md, and relay.md to mention it alongside the other production-ready KvStore implementations.
  • fedify init: Added MySQL/MariaDB as a KvStore option.
  • Monorepo: Updated pnpm-workspace.yaml, root deno.json, and packages/fedify/README.md to register the new package.

Testing

19 integration tests cover all operations: initialize, get, set, delete, drop, list (including expired-row filtering, single-element keys, LIKE special characters, and empty-prefix listing), cas, cas with TTL, and various constructor validations. All pass against a local MySQL 8 instance.

Note

This PR implements only the KvStore half of #587. MysqlMessageQueue (#586) will follow in a separate PR.

Add a new @fedify/mysql package that provides a MysqlKvStore class
implementing the KvStore interface backed by MySQL/MariaDB.  Uses
the mysql2 driver and supports all KvStore operations: get, set
(with TTL), delete, list (with prefix matching), and compare-and-swap
(cas).

The package follows the same structure as @fedify/postgres:

 - Table auto-creation via initialize() and cleanup via drop()
 - Key serialization via JSON.stringify into VARCHAR(512)
 - Values stored as native JSON columns
 - TTL via absolute DATETIME(6) expiration timestamps
 - CAS using SELECT ... FOR UPDATE within transactions
 - Prefix listing via LIKE with proper escaping

Also updates:

 - CI workflow to run MySQL 8 service containers for all test jobs
 - Documentation (kv.md, federation.md, basics.md, relay.md)
 - fedify init to offer MySQL/MariaDB as a KvStore option
 - CHANGES.md, AGENTS.md, and package README files

fedify-dev#585
fedify-dev#587

Co-Authored-By: Claude <noreply@anthropic.com>
@issues-auto-labeler issues-auto-labeler bot added component/build Build system and packaging component/ci CI/CD workflows and GitHub Actions component/integration Web framework integration component/kv Key–value store related labels Mar 4, 2026
@gemini-code-assist
Copy link
Contributor

Summary of Changes

Hello, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request significantly enhances Fedify's data persistence capabilities by introducing a dedicated MySQL/MariaDB key-value store. This new integration provides a robust and scalable option for managing key-value data in production environments, complete with advanced features like time-to-live and atomic compare-and-swap operations. The changes also include comprehensive updates to documentation, build configurations, and the project initialization tool to fully support this new database adapter.

Highlights

  • New @fedify/mysql Package: Introduced a new package, @fedify/mysql, providing a MysqlKvStore class for key-value storage backed by MySQL or MariaDB. This implementation supports TTL, prefix listing, and compare-and-swap (cas()) operations, leveraging the mysql2 driver.
  • Enhanced Persistence Options: Expanded Fedify's production-ready KvStore implementations to include MySQL/MariaDB, offering more flexibility for users already utilizing these database systems.
  • Documentation and Tooling Integration: Updated documentation, fedify init CLI tool, and monorepo configurations to reflect the availability and usage of the new MysqlKvStore, ensuring seamless integration and discoverability.
  • Comprehensive Testing: Added 11 integration tests covering all MysqlKvStore operations, including initialize, get, set, delete, drop, list (with expired-row filtering and various prefixes), and cas (with and without TTL), ensuring robustness and reliability.
Changelog
  • AGENTS.md
    • Updated documentation to include @fedify/mysql in the list of specific database adapters.
    • Updated monorepo build process description to include @fedify/mysql.
  • CHANGES.md
    • Reordered existing changelog entries for @fedify/astro and @fedify/init.
    • Added a new changelog entry for the @fedify/mysql package, detailing its MysqlKvStore implementation and features.
  • deno.json
    • Added @fedify/mysql to the workspace paths.
    • Added mysql2 and mysql2/promise as npm dependencies.
  • deno.lock
    • Updated lock file to include mysql2 and its associated dependencies like aws-ssl-profiles, generate-function, is-property, long, lru.min, named-placeholders, and sql-escaper.
  • docs/.vitepress/config.mts
    • Added @fedify/mysql to the list of JSR documentation references.
  • docs/manual/federation.md
    • Updated the KvStore section to mention MysqlKvStore as a production-ready option.
  • docs/manual/kv.md
    • Added a new section detailing the MysqlKvStore, including installation instructions, usage examples, and a discussion of its pros and cons.
  • docs/manual/relay.md
    • Updated the warning about MemoryKvStore to include MysqlKvStore as a recommended persistent store for production.
  • docs/package.json
    • Added @fedify/mysql as a workspace dependency.
  • docs/tutorial/basics.md
    • Updated the tutorial to recommend MysqlKvStore as a persistent key-value store for production use.
  • packages/fedify/README.md
    • Added @fedify/mysql to the list of packages in the monorepo README.
  • packages/init/src/json/kv.json
    • Added mysql as a new KvStore option for the fedify init command, including its dependencies and environment variables.
  • packages/mysql/README.md
    • Added a new README file for the @fedify/mysql package, describing its purpose and providing installation and usage examples.
  • packages/mysql/deno.json
    • Added a new deno.json configuration file for the @fedify/mysql package, defining its name, version, exports, and tasks.
  • packages/mysql/package.json
    • Added a new package.json file for the @fedify/mysql package, including metadata, dependencies, and build scripts.
  • packages/mysql/src/kv.test.ts
    • Added a comprehensive suite of integration tests for MysqlKvStore, covering initialize, get, set, delete, drop, list (including expired entries and various prefixes), and cas operations.
  • packages/mysql/src/kv.ts
    • Added the MysqlKvStore class implementation, which provides a MySQL/MariaDB-backed KvStore with support for TTL, compare-and-swap, and prefix listing.
  • packages/mysql/src/mod.ts
    • Added the main export file for the @fedify/mysql package.
  • packages/mysql/tsdown.config.ts
    • Added a new tsdown.config.ts file for the @fedify/mysql package, configuring its build process for ESM and CJS formats.
  • packages/relay/README.md
    • Updated the KvStore recommendations to include MysqlKvStore for production use.
  • pnpm-lock.yaml
    • Updated lock file to include new dependencies for @fedify/mysql and mysql2.
  • pnpm-workspace.yaml
    • Added packages/mysql to the monorepo workspace.
    • Added mysql2 to the catalog of shared dependencies.
Ignored Files
  • Ignored by pattern: .github/workflows/** (1)
    • .github/workflows/main.yaml
Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature Command Description
Code Review /gemini review Performs a code review for the current pull request in its current state.
Pull Request Summary /gemini summary Provides a summary of the current pull request in its current state.
Comment @gemini-code-assist Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help /gemini help Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

  1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution.

@dahlia dahlia requested a review from Copilot March 4, 2026 07:33
@dahlia dahlia added the driver/mysql MySQL/MariaDB driver (@fedify/mysql) label Mar 4, 2026
@dahlia dahlia added this to the Fedify 2.1 milestone Mar 4, 2026
Copy link
Contributor

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces the @fedify/mysql package, providing a MysqlKvStore for using MySQL or MariaDB as a persistent key-value store with Fedify. The implementation is robust, correctly handling key serialization, TTLs with DATETIME(6), and atomic compare-and-swap operations using transactions. The changes are well-integrated into the project, with thorough tests, documentation updates across the manual and tutorials, and inclusion in the fedify init tool. My review found only minor inconsistencies in the naming within documentation files. Overall, this is an excellent and valuable addition to the Fedify ecosystem.

Note: Security Review did not run due to the size of the PR.

@dahlia
Copy link
Member Author

dahlia commented Mar 4, 2026

/gemini review

Copy link
Contributor

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces a new @fedify/mysql package, providing a KvStore implementation for MySQL and MariaDB. The implementation is well-structured and includes comprehensive tests and documentation updates. I've identified a few areas for improvement in the MysqlKvStore implementation concerning potential key length limitations, performance of the key expiration mechanism, and the robustness of prefix searching. Additionally, there's an opportunity to simplify the TTL duration calculation.

Note: Security Review did not run due to the size of the PR.

Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR introduces a new @fedify/mysql package that provides a MySQL/MariaDB-backed KvStore implementation (MysqlKvStore) using the mysql2 driver, and wires it into the monorepo, documentation, CI, and fedify init.

Changes:

  • Added @fedify/mysql package with MysqlKvStore, build config, and integration tests.
  • Updated workspace/dependency configuration (pnpm + Deno) to include mysql2 and the new package.
  • Updated docs, changelog, CI, and fedify init templates to reference/use MySQL/MariaDB KV storage.

Reviewed changes

Copilot reviewed 22 out of 24 changed files in this pull request and generated 6 comments.

Show a summary per file
File Description
pnpm-workspace.yaml Adds packages/mysql to the workspace and introduces mysql2 to the catalog.
pnpm-lock.yaml Locks new workspace package and mysql2 dependency graph.
packages/relay/README.md Mentions MysqlKvStore as a production KV option for relay usage.
packages/mysql/tsdown.config.ts Adds build configuration, including Temporal polyfill intro injection.
packages/mysql/src/mod.ts Exports MysqlKvStore public API surface.
packages/mysql/src/kv.ts Implements MysqlKvStore (get/set/delete/list/cas + initialize/drop).
packages/mysql/src/kv.test.ts Adds MySQL integration tests for the KV store operations.
packages/mysql/package.json Defines the new package metadata, exports, and deps/peers.
packages/mysql/deno.json Adds Deno/JSR metadata and tasks for the new package.
packages/mysql/README.md Adds package README with installation and usage snippet.
packages/init/src/json/kv.json Adds MySQL/MariaDB as a fedify init KV backend option.
packages/fedify/README.md Registers @fedify/mysql in the monorepo package list.
docs/tutorial/basics.md Mentions MysqlKvStore as a production KV option in the tutorial.
docs/package.json Adds @fedify/mysql to docs dependencies for examples/build.
docs/manual/relay.md Updates relay manual to include MySQL/MariaDB KV store option.
docs/manual/kv.md Adds a new MysqlKvStore section with install + example code.
docs/manual/federation.md Mentions @fedify/mysql as an additional production KV backend.
docs/.vitepress/config.mts Adds @fedify/mysql to references list.
deno.lock Locks npm:mysql2 and transitive deps for Deno usage.
deno.json Registers packages/mysql workspace path and adds mysql2 imports.
CHANGES.md Adds changelog entry for the new @fedify/mysql package.
AGENTS.md Updates contributor/agent instructions to include packages/mysql.
.github/workflows/main.yaml Adds a MySQL 8 service and MYSQL_URL to CI test jobs.
Files not reviewed (1)
  • pnpm-lock.yaml: Language not supported

@codecov
Copy link

codecov bot commented Mar 4, 2026

Codecov Report

❌ Patch coverage is 94.92386% with 10 lines in your changes missing coverage. Please review.
✅ All tests successful. No failed tests found.

Files with missing lines Patch % Lines
packages/mysql/src/kv.ts 94.89% 10 Missing ⚠️
Files with missing lines Coverage Δ
packages/mysql/src/mod.ts 100.00% <100.00%> (ø)
packages/mysql/src/kv.ts 94.89% <94.89%> (ø)

... and 1 file with indirect coverage changes

🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

dahlia and others added 8 commits March 4, 2026 16:54
The tableName option was interpolated directly into SQL statements
without any sanitization, which could allow SQL injection if the
caller passes a malicious table name.  Added a regex check in the
constructor that restricts table names to identifiers starting with
a letter or underscore, followed by letters, digits, or underscores.
A RangeError is thrown for invalid names.

fedify-dev#597 (comment)

Co-Authored-By: Claude <noreply@anthropic.com>
The 512-character limit was too tight for keys containing long URLs
or other lengthy strings.  VARCHAR(768) aligns with the InnoDB index
prefix limit for utf8mb4 (768 characters × 4 bytes = 3072 bytes),
making full use of the available index space without risking index
prefix truncation errors.

fedify-dev#597 (comment)

Co-Authored-By: Claude <noreply@anthropic.com>
Without an index on the expires column, the DELETE query in #expire()
had to perform a full table scan on every write, degrading performance
as the table grows.  initialize() now creates an index named
idx_<tableName>_expires on the expires column, making the periodic
cleanup query an efficient index range scan.

fedify-dev#597 (comment)

Co-Authored-By: Claude <noreply@anthropic.com>
When a MySQL server runs with NO_BACKSLASH_ESCAPES, the implicit
backslash escape behaviour used in the LIKE pattern is disabled,
causing prefix lookups for keys containing %, _, or \ to return
incorrect results.  Added an explicit ESCAPE '\\' clause to make the
escape character unambiguous regardless of the server's SQL mode.

fedify-dev#597 (comment)

Co-Authored-By: Claude <noreply@anthropic.com>
When value is undefined, setting a key should be treated as a no-op
rather than attempting to store JSON.stringify(undefined) (which
produces undefined, not a valid JSON value), which would fail with
a NOT NULL constraint violation on the value column.

Addresses: fedify-dev#597 (comment)

Co-Authored-By: Claude <noreply@anthropic.com>
When newValue is undefined in a compare-and-swap operation, the
intended semantics are to delete the key (if the expected value
matches) rather than inserting NULL into the NOT NULL value column,
which would cause a constraint violation.

Addresses: fedify-dev#597 (comment)

Co-Authored-By: Claude <noreply@anthropic.com>
The set() and delete() methods both call #expire() to clean up stale
entries after each mutation, but cas() was missing this call.  Without
it, expired entries accumulate until the next set() or delete().

Addresses: fedify-dev#597 (comment)

Co-Authored-By: Claude <noreply@anthropic.com>
Adds an expireCleanupRate option (0–1, default 1) that controls the
probability of running expiry cleanup on each mutation.  Setting it to
0 disables automatic cleanup entirely (useful when a separate cleanup
job handles expiry), while values between 0 and 1 perform cleanup
probabilistically to reduce per-write overhead on high-traffic tables.

Addresses: fedify-dev#597 (comment)

Co-Authored-By: Claude <noreply@anthropic.com>
@dahlia dahlia requested a review from Copilot March 4, 2026 08:14
@dahlia
Copy link
Member Author

dahlia commented Mar 4, 2026

/gemini review

Copy link
Contributor

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces a new package @fedify/mysql which provides a MysqlKvStore for using MySQL or MariaDB as a key-value store with Fedify. The implementation is robust, well-tested, and covers all the necessary features of the KvStore interface, including atomic operations and TTL handling. The changes also include updates to documentation, CI, and the project initializer to integrate the new package.

I've identified a small area for improvement in the durationToSeconds helper function within packages/mysql/src/kv.ts, where using the Temporal.Duration.prototype.total() method can simplify the code. This comment aligns with general best practices and does not contradict any specific rules.

Note: Security Review did not run due to the size of the PR.

Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 22 out of 24 changed files in this pull request and generated 9 comments.

Files not reviewed (1)
  • pnpm-lock.yaml: Language not supported

dahlia and others added 6 commits March 4, 2026 18:16
Use total({ unit: "second", relativeTo: ... }) instead of round() followed
by manual arithmetic, which is simpler and less error-prone.  The relativeTo
option is required when the duration contains calendar units (years, months,
weeks, or days) and ensures correct conversion regardless of DST or leap
seconds.

Reviewed-at: fedify-dev#597 (comment)

Co-Authored-By: Claude <noreply@anthropic.com>
After drop() the table no longer exists, so the next call to initialize()
must recreate it.  Without resetting #initialized the flag stays true and
initialize() is skipped, causing all subsequent operations to fail with a
"table not found" error.

Reviewed-at: fedify-dev#597 (comment)

Co-Authored-By: Claude <noreply@anthropic.com>
MySQL identifier names are limited to 64 characters.  The derived index
name is "idx_<tableName>_expires" which uses 12 additional characters, so
the table name itself must be at most 50 characters long.  Now a RangeError
is thrown at construction time if the name exceeds this limit, preventing a
cryptic ER_TOO_LONG_IDENT error from MySQL at initialization time.

Reviewed-at: fedify-dev#597 (comment)

Co-Authored-By: Claude <noreply@anthropic.com>
The --health-cmd previously used "mysqladmin ping -h 127.0.0.1" without
credentials.  While ping itself doesn't require auth, supplying -uroot
-pmysql makes the intent explicit and avoids potential failures on MySQL
configurations that do require auth for any connection.

Reviewed-at:
  fedify-dev#597 (comment)
  fedify-dev#597 (comment)
  fedify-dev#597 (comment)

Co-Authored-By: Claude <noreply@anthropic.com>
npm, pnpm, Yarn, and Bun users must install mysql2 alongside @fedify/mysql
because mysql2 is a peer dependency not automatically pulled in by those
package managers.  Deno users do not need a separate step since
mysql2/promise is available through the import map bundled with the package.

Updated:
  - docs/manual/kv.md (MysqlKvStore section)
  - packages/mysql/README.md (Installation section)

Reviewed-at:
  fedify-dev#597 (comment)
  fedify-dev#597 (comment)

Co-Authored-By: Claude <noreply@anthropic.com>
The SELECT ... FOR UPDATE in cas() acquires a gap lock on missing rows under
InnoDB REPEATABLE READ (the default isolation level), which prevents
concurrent insertions for the same key.  Under READ COMMITTED this gap lock
is not applied, so concurrent CAS operations on absent keys can race.
Added a comment explaining this assumption so operators who change the pool's
isolation level are warned.

Reviewed-at: fedify-dev#597 (comment)

Co-Authored-By: Claude <noreply@anthropic.com>
@dahlia
Copy link
Member Author

dahlia commented Mar 4, 2026

/gemini review

Copy link
Contributor

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces the @fedify/mysql package, providing a MysqlKvStore implementation. The changes include the new package with its source code, tests, and documentation, as well as updates to the monorepo configuration and documentation to integrate the new package. The implementation of MysqlKvStore uses MySQL-specific features like INSERT ... ON DUPLICATE KEY UPDATE and transactional locking with SELECT ... FOR UPDATE to ensure correctness and efficiency. The test suite is comprehensive and covers various use cases and edge cases. After a thorough review, I have not identified any issues of medium or higher severity.

Note: Security Review did not run due to the size of the PR.

Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 22 out of 24 changed files in this pull request and generated 1 comment.

Files not reviewed (1)
  • pnpm-lock.yaml: Language not supported

When SELECT ... FOR UPDATE included the expiry predicate
(AND expires > NOW(6)), a physically-present but logically-expired
row was not locked.  Two concurrent cas(key, undefined, ...) calls
could therefore both see "no locked row" and both succeed, violating
the create-if-absent atomicity guarantee.

Fix: remove the expiry condition from the WHERE clause of
SELECT ... FOR UPDATE so the lock is always acquired on a physically
present row.  Introduce an is_expired computed column and evaluate
expiry in application code after acquiring the lock.

Regression tests added:
- test 19: cas() treats physically-present but expired rows as undefined
- test 20: concurrent create-if-absent on expired key is atomic

Reviewed-at: fedify-dev#597 (comment)

Co-Authored-By: Claude <noreply@anthropic.com>
@dahlia dahlia merged commit de6972c into fedify-dev:main Mar 4, 2026
44 of 46 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

component/build Build system and packaging component/ci CI/CD workflows and GitHub Actions component/integration Web framework integration component/kv Key–value store related driver/mysql MySQL/MariaDB driver (@fedify/mysql)

Development

Successfully merging this pull request may close these issues.

Implement KvStore for MySQL/MariaDB (@fedify/mysql)

2 participants