Skip to content

NIFI-12219 Add Xodus Flow History Storage with Migration#7885

Merged
markap14 merged 1 commit into
apache:support/nifi-1.xfrom
exceptionfactory:NIFI-12219
Oct 18, 2023
Merged

NIFI-12219 Add Xodus Flow History Storage with Migration#7885
markap14 merged 1 commit into
apache:support/nifi-1.xfrom
exceptionfactory:NIFI-12219

Conversation

@exceptionfactory
Copy link
Copy Markdown
Contributor

Summary

NIFI-12219 Backports Flow Configuration History persistent storage using JetBrains Xodus as implemented for NIFI-12206 and adds migration capabilities to the support branch for subsequent NiFi version 1 releases.

The EntityStoreAuditService class contains the same implementation as merged for NIFI-12219, and the new FlowConfigurationHistoryMigrator supports automated migration from H2 to Xodus on startup. The migrator checks for the presence of the H2 database file named nifi-flow-audit.mv.db and proceeds with migration when found. The migration process includes checking for older H2 binary formats, supporting all previous versions of H2 from 1.4 through 2.2. The migration process then uses available paging capabilities from the H2-based AuditService to transfer records. Following successful migration, the FlowConfigurationHistoryMigrator renames nifi-flow-audit.mv.db with a backup timestamp extension.

Tracking

Please complete the following tracking steps prior to pull request creation.

Issue Tracking

Pull Request Tracking

  • Pull Request title starts with Apache NiFi Jira issue number, such as NIFI-00000
  • Pull Request commit message starts with Apache NiFi Jira issue number, as such NIFI-00000

Pull Request Formatting

  • Pull Request based on current revision of the main branch
  • Pull Request refers to a feature branch with one commit containing changes

Verification

Please indicate the verification steps performed prior to pull request creation.

Build

  • Build completed using mvn clean install -P contrib-check
    • JDK 21

Licensing

  • New dependencies are compatible with the Apache License 2.0 according to the License Policy
  • New dependencies are documented in applicable LICENSE and NOTICE files

Documentation

  • Documentation formatting appears as expected in rendered files

- Added EntityStoreAuditService implementation using JetBrains Xodus libraries
- Added FlowConfigurationHistoryMigrator for checking and migrating H2 databases containing Flow Configuration History to Xodus persistent store implementation
@markap14
Copy link
Copy Markdown
Contributor

Thanks for putting this together, @exceptionfactory! Code looks good. Did a quick test and everything appears to have been migrated properly. Able to see all of the events that I expected, including before/after values, timestamps, action taken, user... +1 will merge to support/nifi-1.x!

@markap14 markap14 merged commit 1a2d769 into apache:support/nifi-1.x Oct 18, 2023
@exceptionfactory exceptionfactory added the hacktoberfest-accepted Hacktoberfest Accepted label Oct 18, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

hacktoberfest-accepted Hacktoberfest Accepted

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants