Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

import into: redact sensitive info & check active job before create #44674

Merged
merged 9 commits into from Jun 15, 2023

Conversation

D3Hunter
Copy link
Contributor

What problem does this PR solve?

Issue Number: ref #42930

Problem Summary:

What is changed and how it works?

  • add a SOFT check that only 1 import job can be run at the same time
  • redact import into in show processlist
  • use a separate session to precheck to avoid show processlist returns incorrect sql

Check List

Tests

  • Unit test
  • Integration test
  • Manual test (add detailed scripts or steps below)
  • No code

Side effects

  • Performance regression: Consumes more CPU
  • Performance regression: Consumes more Memory
  • Breaking backward compatibility

Documentation

  • Affects user behaviors
  • Contains syntax changes
  • Contains variable changes
  • Contains experimental features
  • Changes MySQL compatibility

Release note

Please refer to Release Notes Language Style Guide to write a quality release note.

None

@D3Hunter D3Hunter requested a review from a team as a code owner June 14, 2023 09:09
@ti-chi-bot ti-chi-bot bot added release-note-none size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. labels Jun 14, 2023
@D3Hunter D3Hunter changed the title import into: redact sensitive info import into: redact sensitive info & check active job before create Jun 14, 2023
@D3Hunter
Copy link
Contributor Author

/retest

2 similar comments
@D3Hunter
Copy link
Contributor Author

/retest

@D3Hunter
Copy link
Contributor Author

/retest

Copy link
Contributor

@GMHDBJD GMHDBJD left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@D3Hunter
Copy link
Contributor Author

/hold

@ti-chi-bot ti-chi-bot bot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Jun 15, 2023
// todo: we don't need to do it here, remove it.
if err2 := e.controller.InitDataFiles(ctx); err2 != nil {
return err2
}

sqlExec := e.userSctx.(sqlexec.SQLExecutor)
if err2 := e.controller.CheckRequirements(ctx, sqlExec); err2 != nil {
// must use a new session to pre-check, else the stmt in show processlist will be changed.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe ExecuteInternal will not change show processlist

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we're using it already

	sql := fmt.Sprintf("SELECT 1 FROM %s USE INDEX() LIMIT 1", common.UniqueTable(e.DBName, e.Table.Meta().Name.L))
	rs, err := conn.ExecuteInternal(ctx, sql)

@ti-chi-bot
Copy link

ti-chi-bot bot commented Jun 15, 2023

[LGTM Timeline notifier]

Timeline:

  • 2023-06-15 05:35:25.970599174 +0000 UTC m=+236722.385203253: ☑️ agreed by GMHDBJD.
  • 2023-06-15 06:11:01.85423679 +0000 UTC m=+238858.268840863: ☑️ agreed by lance6716.

@D3Hunter
Copy link
Contributor Author

/unhold

@ti-chi-bot ti-chi-bot bot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Jun 15, 2023
@D3Hunter
Copy link
Contributor Author

/hold

@ti-chi-bot ti-chi-bot bot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Jun 15, 2023
Copy link
Member

@bb7133 bb7133 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@ti-chi-bot
Copy link

ti-chi-bot bot commented Jun 15, 2023

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: bb7133, GMHDBJD, lance6716

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:
  • OWNERS [GMHDBJD,bb7133,lance6716]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@D3Hunter
Copy link
Contributor Author

/unhold

@ti-chi-bot ti-chi-bot bot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Jun 15, 2023
@D3Hunter
Copy link
Contributor Author

/retest

1 similar comment
@D3Hunter
Copy link
Contributor Author

/retest

@ti-chi-bot ti-chi-bot bot merged commit beccd05 into master Jun 15, 2023
26 checks passed
@ti-chi-bot ti-chi-bot bot deleted the import-into-redact branch June 15, 2023 11:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved lgtm release-note-none size/XL Denotes a PR that changes 500-999 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants