Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add after nibble hook to pt-online-schema-change #645

Merged
merged 2 commits into from Jul 31, 2023

Conversation

marcelohpf
Copy link
Contributor

  • The contributed code is licensed under GPL v2.0
  • Contributor Licence Agreement (CLA) is signed
  • util/update-modules has been ran
  • Documention updated
  • Test suite update

Problem

The current progress resource only outputs information as text and it is rather limited in the information it gives. It is only possible to choose between percentage, time or iterations. Besides that, the Progress class is also shared between many workflows to output the current status of the OSC (and other module). Running the module in debug mode generates extra output with meaningful information, although it is not as handy as a metric.

I want to export metrics of the internals of the running OSC so I have better visibility over the dynamic configurations, speed and progress. It helps on troubleshoots for long-running OSCs (more than a day).
Some useful metrics:

  • current copy rate;
  • each # rows per nibble
  • each run time per nibble
  • chunk size (when using dynamic chunks)
  • current status

Proposed solution

This change adds a new hook to the pt-online-schema-change script. The hook is between the before_copy_rows and after_copy_rows. I named it on_copy_rows_after_nibble.

That new hook allows users to write a custom code to get information from table row_cnt, nibble_time, progress and rate. Metrics can be submitted from that hook without further changes to the shared Progress class.

The current position of on_copy_rows_after_nibble is at the end so all other checks has finished (catch up of replicas, load, flow control).

This change don't solve the problem for publishing current state of the script is not available (running, replica catch up, paused, ...). I'm open to suggestions for further changes.

@it-percona-cla
Copy link

it-percona-cla commented Jun 28, 2023

CLA assistant check
All committers have signed the CLA.

Copy link
Contributor

@EvgeniyPatlan EvgeniyPatlan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM 👍

@svetasmirnova svetasmirnova merged commit 8ece30f into percona:3.x Jul 31, 2023
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
4 participants