diff --git a/docs/binlogging-replication-improvements.md b/docs/binlogging-replication-improvements.md index 0f76153c38a..658de5c60eb 100644 --- a/docs/binlogging-replication-improvements.md +++ b/docs/binlogging-replication-improvements.md @@ -165,139 +165,212 @@ mysql> SET binlog_ddl_skip_rewrite = ON; ## Point-in-Time Recovery with `binlog_utils_udf` -Use the binlog_utils_udf component to assist with Point-in-Time Recovery (PiTR). -The component installs user-defined functions (UDFs) that help you map GTIDs to -binary log files and inspect the contents and timestamps of binlog files. +Point-in-Time Recovery (PiTR) allows you to restore a database to any specific moment in time using binary logs. The `binlog_utils_udf` component provides user-defined functions (UDFs) that simplify PiTR operations by helping you: -### Functions +* Map Global Transaction Identifiers (GTIDs) to specific binary log files +* Inspect binary log contents and timestamps +* Locate the exact binary log files needed for recovery operations -| Function | Returns | Description | -|-----------------------------------------|----------------------|-------------------------------------------------------------------------| -| get_binlog_by_gtid(gtid) | STRING (binlog name) | Returns the binlog file that contains the specified GTID. | -| get_last_gtid_from_binlog(binlog) | STRING (GTID) | Returns the last GTID found in the specified binlog. | -| get_gtid_set_by_binlog(binlog) | STRING (GTID set) | Returns all GTIDs found in the specified binlog. | -| get_binlog_by_gtid_set(gtid_set) | STRING (binlog name) | Returns the first binlog file that contains at least one GTID from the specified set. | -| get_first_record_timestamp_by_binlog(binlog) | INTEGER (timestamp) | Returns the timestamp of the first event in the specified binlog. | -| get_last_record_timestamp_by_binlog(binlog) | INTEGER (timestamp) | Returns the timestamp of the last event in the specified binlog. | +These functions are particularly useful when you need to determine which binary log files contain specific transactions or events during recovery planning. -### Notes +### Prerequisites -* Timestamp-returning functions provide values with microsecond precision in -UNIX time. Each value represents the number of microseconds since -1970-01-01 00:00:00 UTC. +Before using the `binlog_utils_udf` component, ensure the following requirements are met: -* Functions that accept a binlog name require a short file name only. Do not -include a path. If the input contains a path separator (/), the server -returns an error. +* Percona Server for MySQL: The component is only available in Percona Server for MySQL, not in standard MySQL -* The server reads binlogs from the current binlog directory defined by the -@@log_bin_basename system variable. +* Binary logging enabled: The server must have binary logging enabled (`log_bin` system variable set to `ON`) -* Functions that return a binlog file name return the short name (no path). +* GTID enabled: For GTID-related functions, GTID must be enabled (`gtid_mode` set to `ON`) -### Install the component +* MySQL privileges: You need `SYSTEM_VARIABLES_ADMIN` privilege to install components. For binary log operations, `BINLOG_ADMIN` privilege may also be required. The `SUPER` privilege is deprecated in MySQL 8.0+ and should be replaced with specific dynamic privileges -Install the component once on each server where you want to use these UDFs. +#### Install the component + +Install the component on each server where you plan to use these functions: ```{.bash data-prompt="mysql>"} mysql> INSTALL COMPONENT 'file://component_binlog_utils_udf'; ``` -You can confirm installation by checking the list of registered functions: +#### Verify installation + +Confirm the component is loaded successfully: + +```{.bash data-prompt="mysql>"} +mysql> SELECT * FROM mysql.component WHERE component_urn = 'file://component_binlog_utils_udf'; +``` + +The query should return one row if the component is installed. You can also verify by checking for the available functions: ```{.bash data-prompt="mysql>"} -mysql> SELECT ROUTINE_NAME FROM INFORMATION_SCHEMA.ROUTINES \G +mysql> SELECT ROUTINE_NAME FROM INFORMATION_SCHEMA.ROUTINES + -> WHERE ROUTINE_NAME LIKE 'get_%' AND ROUTINE_TYPE = 'FUNCTION'; ``` +### Available functions + +The `binlog_utils_udf` component provides six functions for binary log analysis and GTID mapping: + +| Function | Returns | Description | Use Case | +|-----------------------------------------|----------------------|-------------------------------------------------------------------------|----------| +| [`get_binlog_by_gtid(gtid)`](#find-binary-log-by-gtid) | STRING (binlog name) | Returns the binary log file that contains the specified GTID. | Find which binary log contains a specific transaction | +| [`get_last_gtid_from_binlog(binlog)`](#get-last-gtid-from-binary-log) | STRING (GTID) | Returns the last GTID found in the specified binary log. | Identify the final transaction in a binary log file | +| [`get_gtid_set_by_binlog(binlog)`](#get-all-gtids-from-binary-log) | STRING (GTID set) | Returns all GTIDs found in the specified binary log. | Get complete list of transactions in a binary log | +| [`get_binlog_by_gtid_set(gtid_set)`](#find-binary-log-by-gtid-set) | STRING (binlog name) | Returns the first binary log file that contains at least one GTID from the specified set. | Find binary log containing any transaction from a GTID set | +| [`get_first_record_timestamp_by_binlog(binlog)`](#get-first-event-timestamp) | INTEGER (timestamp) | Returns the timestamp of the first event in the specified binary log. | Determine when a binary log file started | +| [`get_last_record_timestamp_by_binlog(binlog)`](#get-last-event-timestamp) | INTEGER (timestamp) | Returns the timestamp of the last event in the specified binary log. | Determine when a binary log file ended | + +### Important notes + +* CAST requirement: When using these user-defined functions, you must use CAST to return a result. String functions require `CAST(...AS CHAR)` and timestamp functions require `CAST(...AS UNSIGNED)`. + +* Timestamp precision: Timestamp-returning functions provide values with microsecond precision in UNIX time format. Each value represents the number of microseconds since 1970-01-01 00:00:00 UTC. + +* Binary log file names: Functions that accept a binary log name require only the short file name (for example, `binlog.000001`). Do not include the full path. If the input contains a path separator (`/`), the server returns an error. + +* Binary log directory: The server reads binary logs from the directory defined by the `@@log_bin_basename` system variable. + +* Return values: Functions that return binary log file names return only the short name without the path. + +* Performance considerations: These functions read binary log files directly from disk. For large binary log files, the functions may take several seconds to complete. + + ### Usage examples -Replace the sample arguments with values from your environment. The examples -show the typical way to call each function. For clarity, results are aliased. +The following examples demonstrate how to use each function. Replace the sample arguments with values from your environment. All examples include the required CAST statements for proper function execution. -#### get_binlog_by_gtid() +#### Find binary log by GTID -Locate the binlog that contains a GTID: +Use `get_binlog_by_gtid()` to locate which binary log file contains a specific transaction: ```{.bash data-prompt="mysql>"} -mysql> SELECT get_binlog_by_gtid('UUID-GROUP:1') AS binlog; +mysql> SELECT CAST(get_binlog_by_gtid('550e8400-e29b-41d4-a716-446655440000:123') AS CHAR) AS binlog; ``` -#### get_last_gtid_from_binlog() +Use case: When you know a specific GTID and need to find which binary log file contains that transaction for recovery purposes. + +#### Get last GTID from binary log -Return the last GTID in a binlog +Use `get_last_gtid_from_binlog()` to find the final transaction in a specific binary log file: ```{.bash data-prompt="mysql>"} -mysql> SELECT get_last_gtid_from_binlog('binlog.000001') AS last_gtid; +mysql> SELECT CAST(get_last_gtid_from_binlog('binlog.000001') AS CHAR) AS last_gtid; ``` -#### get_gtid_set_by_binlog() +Use case: Determine the last transaction processed in a binary log file before rotating to the next file. -Return all GTIDs in a binlog +#### Get all GTIDs from binary log + +Use `get_gtid_set_by_binlog()` to retrieve all GTIDs contained in a specific binary log file: ```{.bash data-prompt="mysql>"} -mysql> SELECT get_gtid_set_by_binlog('binlog.000001') AS gtid_set; +mysql> SELECT CAST(get_gtid_set_by_binlog('binlog.000001') AS CHAR) AS gtid_set; ``` -#### get_binlog_by_gtid_set() +Use case: Get a complete list of all transactions in a binary log file for analysis or replication setup. + +#### Find binary log by GTID set -Find a binlog that contains any GTID in a set +Use `get_binlog_by_gtid_set()` to find the first binary log file that contains any GTID from a specified set: ```{.bash data-prompt="mysql>"} -mysql> SELECT get_binlog_by_gtid_set('UUID1:7,UUID1:8') AS binlog; +mysql> SELECT CAST(get_binlog_by_gtid_set('550e8400-e29b-41d4-a716-446655440000:7,550e8400-e29b-41d4-a716-446655440000:8') AS CHAR) AS binlog; ``` -#### get_first_record_timestamp_by_binlog() and get_last_record_timestamp_by_binlog(binlog) +Use case: When you have a set of GTIDs and need to find which binary log file contains at least one of those transactions. + +#### Get binary log timestamps -Get the first event timestamp from a binlog. The function returns microseconds since the UNIX epoch. Use the tabs below to -see the raw numeric value or a human-readable timestamp. +Use timestamp functions to determine when events occurred in binary log files. These functions return microsecond-precision timestamps in UNIX time format. -=== "Raw Timestamp" +##### Get first event timestamp + +Find when the first event was written to a binary log file: + +=== "Raw Timestamp (Microseconds)" ```{.bash data-prompt="mysql>"} - mysql> SELECT get_first_record_timestamp_by_binlog('binlog.000001') AS raw_ts; + mysql> SELECT CAST(get_first_record_timestamp_by_binlog('binlog.000001') AS UNSIGNED) AS raw_ts; ``` -=== "Human-Readable" +=== "Human-Readable Format" ```{.bash data-prompt="mysql>"} mysql> SELECT FROM_UNIXTIME( - get_first_record_timestamp_by_binlog('binlog.000001') DIV 1000000 + CAST(get_first_record_timestamp_by_binlog('binlog.000001') AS UNSIGNED) DIV 1000000 ) AS first_event_ts; ``` -Get the last event timestamp from a binlog +Use case: Determine when a binary log file started receiving events, useful for recovery planning. + +##### Get last event timestamp -=== "Raw Timestamp" +Find when the last event was written to a binary log file: + +=== "Raw Timestamp (Microseconds)" ```{.bash data-prompt="mysql>"} - mysql> SELECT get_last_record_timestamp_by_binlog('binlog.000001') AS raw_ts; + mysql> SELECT CAST(get_last_record_timestamp_by_binlog('binlog.000001') AS UNSIGNED) AS raw_ts; ``` -=== "Human-Readable" +=== "Human-Readable Format" ```{.bash data-prompt="mysql>"} mysql> SELECT FROM_UNIXTIME( - get_last_record_timestamp_by_binlog('binlog.000001') DIV 1000000 + CAST(get_last_record_timestamp_by_binlog('binlog.000001') AS UNSIGNED) DIV 1000000 ) AS last_event_ts; ``` +Use case: Determine when a binary log file stopped receiving events, useful for understanding binary log rotation timing. + + +### Troubleshooting + +#### Common issues + +Function returns NULL: This usually indicates that the specified GTID or binary log file does not exist. Verify that: + +* The GTID format is correct (UUID:transaction_id) - ??? example "Expected output" +* The binary log file exists in the binary log directory - ```{.text .no-copy} - +---------------+ - | binlog | - +---------------+ - | binlog.000001 | - +---------------+ - ``` +* GTID is enabled on the server -Actual values depend on your server state and binlog contents. +Error: "Unknown function": The component is not installed. Install the component using the `INSTALL COMPONENT` command. + +Error: "Access denied": You need `SYSTEM_VARIABLES_ADMIN` privilege to install the component and `BINLOG_ADMIN` privilege for binary log operations. The `SUPER` privilege is deprecated in MySQL 8.0+. + +Performance issues: These functions read binary log files directly from disk. For large binary log files, expect execution times of several seconds. + +#### Verify binary log files + +Check which binary log files are available: + +```{.bash data-prompt="mysql>"} +mysql> SHOW BINARY LOGS; +``` + +#### Check GTID status + +Verify GTID is enabled: + +```{.bash data-prompt="mysql>"} +mysql> SHOW VARIABLES LIKE 'gtid_mode'; +``` ### Uninstall the component -Remove the component and all associated UDFs: +Remove the component and all associated functions: ```{.bash data-prompt="mysql>"} mysql> UNINSTALL COMPONENT 'file://component_binlog_utils_udf'; ``` + +Verify removal: + +```{.bash data-prompt="mysql>"} +mysql> SELECT * FROM mysql.component WHERE component_urn = 'file://component_binlog_utils_udf'; +``` + +The query should return no rows if the component is successfully uninstalled. ## Limitations