Skip to content

feat(importers): MySQL/MariaDB importer wire & verify (v1.8.0)#4288

Merged
makr-code merged 3 commits intodevelopfrom
copilot/mysql-mariadb-importer-wire-verify
Mar 16, 2026
Merged

feat(importers): MySQL/MariaDB importer wire & verify (v1.8.0)#4288
makr-code merged 3 commits intodevelopfrom
copilot/mysql-mariadb-importer-wire-verify

Conversation

Copy link
Contributor

Copilot AI commented Mar 16, 2026

Wires mysql_importer.cpp into the IImporterPluginRegistry via static-init self-registration and exposes it through the admin import API as POST /api/v1/import/mysql.

Core Changes

Registry Registration (mysql_importer.h / mysql_importer.cpp)

  • Added MySQLImporterSchemePlugin : public IImporterPlugin handling mysql:// and mariadb:// URI schemes
  • Registered at static-init time via REGISTER_IMPORTER_PLUGIN — no manual wiring required:
// src/importers/mysql_importer.cpp (bottom)
namespace themis {
namespace importers {
REGISTER_IMPORTER_PLUGIN(MySQLImporterSchemePlugin)  // wraps in namespace to keep ## token-paste valid
} // namespace importers
} // namespace themis

Bug fixed: The initial attempt used REGISTER_IMPORTER_PLUGIN(themis::importers::MySQLImporterSchemePlugin) at global scope, which causes a compile error — the ## token-paste in the macro produces invalid identifiers containing ::.

Admin Import API (import_api_handler.h / .cpp)

  • Added POST /api/v1/import/mysql route → handleStartMySQLImport()
  • Handler resolves MySQLImporterSchemePlugin from IImporterPluginRegistry::instance() at request time; returns 501 if the plugin is absent, 400 for missing source_path
  • importer_interfaces.h added to import_api_handler.h includes for IImporterPluginRegistry / ImportConfig visibility

Already Satisfied ACs (no changes needed)

  • Prometheus counters importers_mysql_rows_imported_total / importers_mysql_errors_total already emitted in importData() (lines 278, 281, 874, 888)
  • Docker MySQL 8.0 integration test already present as mysql-live-integration job in importer-tests.yml

Testing

  • Unit tests added/updated — tests/test_mysql_importer_registry.cpp: 12 tests covering mysql:///mariadb:// scheme resolution, createImporter() instance isolation, plugin-id list membership, and admin route guard logic
  • Integration tests added/updated — mysql-live-integration CI job (pre-existing): seeds MySQL 8.0 Docker container, dumps with mysqldump, exercises the full importer pipeline
  • Manual testing performed

New Test Infrastructure

  • tests/mysql_importer_registry_logger_stub.cpp — no-op Logger stub so the registry test binary can link mysql_importer.cpp without the full spdlog chain (mirrors kafka_importer_logger_stub.cpp)
  • tests/CMakeLists.txt — standalone test_mysql_importer_registry target + exclude pattern to prevent duplicate extern "C" plugin symbols in the main themis_tests binary

📚 Research & Knowledge (wenn applicable)

  • Diese PR basiert auf wissenschaftlichen Paper(s) oder Best Practices?
    • Falls JA: Research-Dateien in /docs/research/ angelegt?
    • Falls JA: Im Modul-README unter "Wissenschaftliche Grundlagen" verlinkt?
    • Falls JA: In /docs/research/implementation_influence/ eingetragen?

Relevante Quellen:

  • Paper:
  • Best Practice:
  • Architecture Decision:

Checklist

  • Code follows project style guidelines
  • Self-review completed
  • Documentation updated (if needed)
  • No new warnings introduced
Original prompt

This section details on the original issue you should resolve

<issue_title>MySQL / MariaDB Importer (wire & verify)</issue_title>
<issue_description>### Context

This issue implements the roadmap item 'MySQL / MariaDB Importer (wire & verify)' for the importers domain. It is sourced from the consolidated roadmap under 🟠 High Priority — Near-term (v1.5.0 – v1.8.0) and targets milestone v1.8.0.

Primary detail section: MySQL / MariaDB Importer

Goal

Deliver the scoped changes for MySQL / MariaDB Importer (wire & verify) in src/importers/ and complete the linked detail section in a release-ready state for v1.8.0.

Detailed Scope

MySQL / MariaDB Importer

Priority: High
Target Version: v1.8.0
Status: src/importers/mysql_importer.cpp exists but needs verification that it is fully wired.

Implementation Notes:

  • [ ] Verify mysql_importer.cpp registers with ImporterRegistry and is reachable from the admin import API.
  • [ ] Add Prometheus counters importers_mysql_rows_imported_total and importers_mysql_errors_total consistent with other importer naming.
  • [ ] Add integration test using a Docker MySQL 8.0 container.

Acceptance Criteria

  • Verify mysql_importer.cpp registers with ImporterRegistry and is reachable from the admin import API.
  • Add Prometheus counters importers_mysql_rows_imported_total and importers_mysql_errors_total consistent with other importer naming.
  • Add integration test using a Docker MySQL 8.0 container.

Relationships

References

  • src/ROADMAP.md
  • src/importers/FUTURE_ENHANCEMENTS.md#mysql--mariadb-importer
  • Source key: roadmap:68:importers:v1.8.0:mysql-mariadb-importer

Generated from the consolidated source roadmap. Keep the roadmap and issue in sync when scope changes.

</issue_description>

Comments on the Issue (you are @copilot in this section)


✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.

Co-authored-by: makr-code <150588092+makr-code@users.noreply.github.com>
Co-authored-by: makr-code <150588092+makr-code@users.noreply.github.com>
Copilot AI changed the title [WIP] [IMPORT-123] Implement MySQL / MariaDB Importer (wire & verify) feat(importers): MySQL/MariaDB importer wire & verify (v1.8.0) Mar 16, 2026
@makr-code makr-code marked this pull request as ready for review March 16, 2026 12:45
@makr-code makr-code merged commit 4af6aa3 into develop Mar 16, 2026
11 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

MySQL / MariaDB Importer (wire & verify)

2 participants