[feature](log) Add label and txn_id columns to audit_log for transactional SQL statements#63865
Open
heguanhui wants to merge 1 commit into
Open
[feature](log) Add label and txn_id columns to audit_log for transactional SQL statements#63865heguanhui wants to merge 1 commit into
heguanhui wants to merge 1 commit into
Conversation
…ional SQL statements
### What problem does this PR solve?
Issue Number: close #xxx
Related PR: #xxx
Problem Summary: The audit_log internal table did not record transaction information (label and txn_id) for transactional SQL statements (INSERT/UPDATE/DELETE/MERGE_INTO). Users and DBAs could not trace which transaction a particular DML operation belonged to, making it difficult to correlate audit log entries with transaction metadata. Additionally, the old-style DELETE path (via DeleteHandler) did not call setOrUpdateInsertResult(), causing label/txn_id to be missing even if they were recorded.
### Release note
Add `label` (varchar(128)) and `txn_id` (bigint) columns to the `audit_log` internal table. These columns record the transaction label and transaction ID for INSERT, UPDATE, DELETE, and MERGE INTO statements, enabling better transaction tracing and audit analysis.
### Check List (For Author)
- Test: Regression test / Unit Test
- Regression test: test_audit_log_label_txn_id.groovy (INSERT with/without label, UPDATE, DELETE, SELECT)
- Unit Test: AuditLogHelperTest (4 tests), AuditLogBuilderTest (8 tests), AuditEventProcessorTest, InternalSchemaInitializerTest, InternalSchemaAlterTest, AuditLogWorkloadGroupTest (20 tests total)
- Behavior changed: Yes - audit_log table now has two additional columns; old-style DELETE now populates insertResult
- Does this need documentation: Yes (will add separately)
Contributor
|
Thank you for your contribution to Apache Doris. Please clearly describe your PR:
|
Contributor
Author
|
run buildall |
Contributor
TPC-H: Total hot run time: 31029 ms |
Contributor
TPC-DS: Total hot run time: 171053 ms |
Contributor
FE Regression Coverage ReportIncrement line coverage |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What problem does this PR solve?
Issue Number: close #xxx
Related PR: #xxx
Problem Summary: The
audit_loginternal table did not record transaction information (labelandtxn_id) for transactional SQL statements (INSERT/UPDATE/DELETE/MERGE_INTO). Users and DBAs could not trace which transaction a particular DML operation belonged to, making it difficult to correlate audit log entries with transaction metadata. Additionally, the old-style DELETE path (viaDeleteHandler) did not callsetOrUpdateInsertResult(), causinglabel/txn_idto be missing even if they were recorded.What is changed and how it works?
labelandtxn_idfields toAuditEvent— Two new@AuditFieldannotated public fields with corresponding builder setters.AuditLogHelper.logAuditLogImpl()— Only for transactional statement types (INSERT/UPDATE/DELETE/MERGE_INTO), read fromConnectContext.insertResult. Non-transactional statements (e.g., SELECT) get empty label and -1 txn_id.labelandtxn_idcolumns toInternalSchema.AUDIT_SCHEMA—labelasvarchar(128),txn_idasbigint. Existing audit_log tables will be auto-altered byInternalSchemaInitializer.alterAuditSchemaIfNeeded().AuditLoader.fillLogBuffer()— Append the two new fields to the CSV buffer for stream load into audit_log.labelfield fromLoadAuditEventandStreamLoadAuditEvent— These subclasses now inheritlabelfrom the baseAuditEvent, avoidingClass.getFields()reflection returning duplicate fields.DeleteHandler.process()— Callctx.setOrUpdateInsertResult()after successful commit so that label/txn_id are available for audit logging.Release note
Add
label(varchar(128)) andtxn_id(bigint) columns to theaudit_loginternal table. These columns record the transaction label and transaction ID for INSERT, UPDATE, DELETE, and MERGE INTO statements, enabling better transaction tracing and audit analysis.Check List (For Author)
test_audit_log_label_txn_id.groovy(INSERT with/without label, UPDATE, DELETE, SELECT)AuditLogHelperTest(4 tests),AuditLogBuilderTest(8 tests),AuditEventProcessorTest,InternalSchemaInitializerTest,InternalSchemaAlterTest,AuditLogWorkloadGroupTest(20 tests total, all passed)Check List (For Reviewer)