Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support auth for QueryRunner #11897

Merged
merged 1 commit into from Nov 1, 2023
Merged

Conversation

xiangfu0
Copy link
Contributor

@xiangfu0 xiangfu0 commented Oct 27, 2023

  • Support auth for QueryRunner
  • Some refactor

Sample usage:
Run org.apache.pinot.tools.AuthQuickstart,
Make a queryFile.txt file with content: select count(*) from airlineStats limit 10

Then run the queryRunner with command:

bin/query-runner.sh -mode=singleThread -queryFile=queryFile.txt -user admin -password verysecret -brokerPort 8000 

and sample output:

2023/10/30 13:49:18.908 INFO [QueryRunner] [main] --------------------------------------------------------------------------------
2023/10/30 13:49:18.909 INFO [QueryRunner] [main] FINAL REPORT:
2023/10/30 13:49:18.910 INFO [QueryRunner] [main] Time Passed: 787ms
Queries Executed: 59
Exceptions: 0
Average QPS: 74.96823379923761
Average Broker Time: 2.7457627118644066ms
Average Client Time: 7.627118644067797ms
2023/10/30 13:49:18.911 INFO [QueryRunner] [main] --------------------------------------------------------------------------------
2023/10/30 13:49:18.911 INFO [QueryRunner] [main] CLIENT TIME STATISTICS:
2023/10/30 13:49:18.911 INFO [QueryRunner] [main] DescriptiveStatistics:
n: 59
min: 3.0
max: 83.0
mean: 7.627118644067796
std dev: 11.113673319748722
median: 5.0
skewness: 5.844161475062559
kurtosis: 38.03544930725383

2023/10/30 13:49:18.912 INFO [QueryRunner] [main] 10th percentile: 3.0
2023/10/30 13:49:18.912 INFO [QueryRunner] [main] 25th percentile: 4.0
2023/10/30 13:49:18.913 INFO [QueryRunner] [main] 50th percentile: 5.0
2023/10/30 13:49:18.913 INFO [QueryRunner] [main] 90th percentile: 9.0
2023/10/30 13:49:18.913 INFO [QueryRunner] [main] 95th percentile: 19.0
2023/10/30 13:49:18.913 INFO [QueryRunner] [main] 99th percentile: 83.0
2023/10/30 13:49:18.913 INFO [QueryRunner] [main] 99.9th percentile: 83.0
2023/10/30 13:49:18.913 INFO [QueryRunner] [main] --------------------------------------------------------------------------------

@codecov-commenter
Copy link

codecov-commenter commented Oct 27, 2023

Codecov Report

Merging #11897 (3d199f0) into master (2e37f2c) will increase coverage by 0.01%.
The diff coverage is 0.00%.

@@             Coverage Diff              @@
##             master   #11897      +/-   ##
============================================
+ Coverage     61.43%   61.44%   +0.01%     
- Complexity     1146     1147       +1     
============================================
  Files          2376     2376              
  Lines        128773   128787      +14     
  Branches      19906    19909       +3     
============================================
+ Hits          79111    79138      +27     
+ Misses        43951    43929      -22     
- Partials       5711     5720       +9     
Flag Coverage Δ
custom-integration1 <0.01% <0.00%> (ø)
integration <0.01% <0.00%> (ø)
integration1 <0.01% <0.00%> (ø)
integration2 0.00% <0.00%> (ø)
java-11 61.41% <0.00%> (+0.03%) ⬆️
java-21 61.28% <0.00%> (-0.02%) ⬇️
skip-bytebuffers-false 61.43% <0.00%> (+0.01%) ⬆️
skip-bytebuffers-true 61.26% <0.00%> (-0.02%) ⬇️
temurin 61.44% <0.00%> (+0.01%) ⬆️
unittests 61.44% <0.00%> (+0.01%) ⬆️
unittests1 46.61% <0.00%> (-0.05%) ⬇️
unittests2 27.68% <0.00%> (+0.04%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files Coverage Δ
...rg/apache/pinot/common/auth/AuthProviderUtils.java 14.00% <0.00%> (-5.45%) ⬇️

... and 14 files with indirect coverage changes

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

@KKcorps
Copy link
Contributor

KKcorps commented Oct 30, 2023

Should this AuthUtils class be part of pinot-tools? or should it be in pinot-common ?

Reason being, we may want to avoid running into issues where any changes in Auth handling aren't replicated in the pinot-tools class and thus breaking the commands.

@xiangfu0
Copy link
Contributor Author

Should this AuthUtils class be part of pinot-tools? or should it be in pinot-common ?

Reason being, we may want to avoid running into issues where any changes in Auth handling aren't replicated in the pinot-tools class and thus breaking the commands.

moved

@xiangfu0 xiangfu0 merged commit 403790b into apache:master Nov 1, 2023
19 checks passed
@xiangfu0 xiangfu0 deleted the query-runner-auth branch November 1, 2023 21:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants