Skip to content

[BugFix](Load) Add a secure path for MySql Load to load local file from fe node#16653

Merged
yiguolei merged 2 commits intoapache:masterfrom
SaintBacchus:MySqlServerLoad
Feb 13, 2023
Merged

[BugFix](Load) Add a secure path for MySql Load to load local file from fe node#16653
yiguolei merged 2 commits intoapache:masterfrom
SaintBacchus:MySqlServerLoad

Conversation

@SaintBacchus
Copy link
Contributor

@SaintBacchus SaintBacchus commented Feb 13, 2023

Proposed changes

Issue Number: close #xxx

Problem summary

MySql load can load fe server node, but it will cause secure issue that user use it to detect the fe node local file.

For this reason, add a configuration named mysql_load_server_secure_path to set a secure path to load data.

By default, load fe local file feature is disabled by this configuration.

Checklist(Required)

  • Does it affect the original behavior
  • Has unit tests been added
  • Has document been added or modified
  • Does it need to update dependencies
  • Is this PR support rollback (If NO, please explain WHY)

Further comments

If this is a relatively large or complex change, kick off the discussion at dev@doris.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc...

@github-actions github-actions bot added the area/planner Issues or PRs related to the query planner label Feb 13, 2023
@hello-stephen
Copy link
Contributor

hello-stephen commented Feb 13, 2023

TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 34.93 seconds
stream load tsv: 474 seconds loaded 74807831229 Bytes, about 150 MB/s
stream load json: 40 seconds loaded 2358488459 Bytes, about 56 MB/s
stream load orc: 68 seconds loaded 1101869774 Bytes, about 15 MB/s
stream load parquet: 28 seconds loaded 861443392 Bytes, about 29 MB/s
https://doris-community-test-1308700295.cos.ap-hongkong.myqcloud.com/tmp/20230213050648_clickbench_pr_94999.html

Copy link
Contributor

@yiguolei yiguolei left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@yiguolei yiguolei merged commit 77be0d1 into apache:master Feb 13, 2023
YangShaw pushed a commit to YangShaw/doris that referenced this pull request Feb 17, 2023
…om fe node (apache#16653)

MySql load can load fe server node, but it will cause secure issue that user use it to detect the fe node local file.

For this reason, add a configuration named mysql_load_server_secure_path to set a secure path to load data.

By default, load fe local file feature is disabled by this configuration.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/planner Issues or PRs related to the query planner

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants