Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add a simple PinotFS benchmark driver #5160

Merged
merged 2 commits into from Mar 26, 2020
Merged

Add a simple PinotFS benchmark driver #5160

merged 2 commits into from Mar 26, 2020

Conversation

snleee
Copy link
Contributor

@snleee snleee commented Mar 17, 2020

This benchmark tests the performance of pinotFS operations.

Add a simple PinotFS benchmark driver
2262a00 
This benchmark tests the performance of pinotFS operations.
@snleee
Copy link
Contributor Author

snleee commented Mar 17, 2020 

2020/03/16 22:38:35.399 INFO [PinotFSBenchmarkDriver] [main] PinotFS has been initialized sucessfully. (mode = ALL, pinotFSClass = LocalPinotFS, configFile = /Users/snlee/workspace/pinot/pinot-tools/target/pinot-tools-pkg/config/sample2.conf, baseDirectoryUri = file:///Users/snlee/benchmark/base, localTempDir = /Users/snlee/benchmark/temp, numSegmentsForListFilesTest = 10, dataSizeInMBsForCopyTest = 100)
2020/03/16 22:38:35.409 INFO [PinotFSBenchmarkDriver] [main] ========= List Files in Multiple Directories ==========
2020/03/16 22:38:35.409 INFO [PinotFSBenchmarkDriver] [main] Created file:/Users/snlee/benchmark/base/listTestMultipleFile for list test...
2020/03/16 22:38:35.418 INFO [PinotFSBenchmarkDriver] [main] Took 9 ms to create 1 segments for directory_0
2020/03/16 22:38:35.438 INFO [PinotFSBenchmarkDriver] [main] Took 29 ms to create 10 segments for directory_1
2020/03/16 22:38:35.540 INFO [PinotFSBenchmarkDriver] [main] Took 131 ms to create 100 segments for directory_2
2020/03/16 22:38:36.552 INFO [PinotFSBenchmarkDriver] [main] Took 1143 ms to create 1000 segments for directory_3
2020/03/16 22:38:41.756 INFO [PinotFSBenchmarkDriver] [main] Took 6347 ms to create 10000 segments for directory_4
2020/03/16 22:38:41.822 INFO [PinotFSBenchmarkDriver] [main] 0: took 66 ms to listFiles. directory_0 (1 segments)
2020/03/16 22:38:41.822 INFO [PinotFSBenchmarkDriver] [main] 1: took 0 ms to listFiles. directory_0 (1 segments)
2020/03/16 22:38:41.823 INFO [PinotFSBenchmarkDriver] [main] 2: took 0 ms to listFiles. directory_0 (1 segments)
2020/03/16 22:38:41.823 INFO [PinotFSBenchmarkDriver] [main] 3: took 0 ms to listFiles. directory_0 (1 segments)
2020/03/16 22:38:41.823 INFO [PinotFSBenchmarkDriver] [main] 4: took 0 ms to listFiles. directory_0 (1 segments)
2020/03/16 22:38:41.823 INFO [PinotFSBenchmarkDriver] [main] 
2020/03/16 22:38:41.824 INFO [PinotFSBenchmarkDriver] [main] 0: took 1 ms to listFiles. directory_1 (10 segments)
2020/03/16 22:38:41.825 INFO [PinotFSBenchmarkDriver] [main] 1: took 1 ms to listFiles. directory_1 (10 segments)
2020/03/16 22:38:41.826 INFO [PinotFSBenchmarkDriver] [main] 2: took 1 ms to listFiles. directory_1 (10 segments)
2020/03/16 22:38:41.826 INFO [PinotFSBenchmarkDriver] [main] 3: took 0 ms to listFiles. directory_1 (10 segments)
2020/03/16 22:38:41.827 INFO [PinotFSBenchmarkDriver] [main] 4: took 0 ms to listFiles. directory_1 (10 segments)
2020/03/16 22:38:41.827 INFO [PinotFSBenchmarkDriver] [main] 
2020/03/16 22:38:41.833 INFO [PinotFSBenchmarkDriver] [main] 0: took 6 ms to listFiles. directory_2 (100 segments)
2020/03/16 22:38:41.837 INFO [PinotFSBenchmarkDriver] [main] 1: took 4 ms to listFiles. directory_2 (100 segments)
2020/03/16 22:38:41.843 INFO [PinotFSBenchmarkDriver] [main] 2: took 6 ms to listFiles. directory_2 (100 segments)
2020/03/16 22:38:41.847 INFO [PinotFSBenchmarkDriver] [main] 3: took 4 ms to listFiles. directory_2 (100 segments)
2020/03/16 22:38:41.851 INFO [PinotFSBenchmarkDriver] [main] 4: took 4 ms to listFiles. directory_2 (100 segments)
2020/03/16 22:38:41.851 INFO [PinotFSBenchmarkDriver] [main] 
2020/03/16 22:38:41.872 INFO [PinotFSBenchmarkDriver] [main] 0: took 21 ms to listFiles. directory_3 (1000 segments)
2020/03/16 22:38:41.890 INFO [PinotFSBenchmarkDriver] [main] 1: took 18 ms to listFiles. directory_3 (1000 segments)
2020/03/16 22:38:41.906 INFO [PinotFSBenchmarkDriver] [main] 2: took 16 ms to listFiles. directory_3 (1000 segments)
2020/03/16 22:38:41.925 INFO [PinotFSBenchmarkDriver] [main] 3: took 19 ms to listFiles. directory_3 (1000 segments)
2020/03/16 22:38:41.942 INFO [PinotFSBenchmarkDriver] [main] 4: took 15 ms to listFiles. directory_3 (1000 segments)
2020/03/16 22:38:41.942 INFO [PinotFSBenchmarkDriver] [main] 
2020/03/16 22:38:42.089 INFO [PinotFSBenchmarkDriver] [main] 0: took 147 ms to listFiles. directory_4 (10000 segments)
2020/03/16 22:38:42.191 INFO [PinotFSBenchmarkDriver] [main] 1: took 102 ms to listFiles. directory_4 (10000 segments)
2020/03/16 22:38:42.361 INFO [PinotFSBenchmarkDriver] [main] 2: took 169 ms to listFiles. directory_4 (10000 segments)
2020/03/16 22:38:42.461 INFO [PinotFSBenchmarkDriver] [main] 3: took 99 ms to listFiles. directory_4 (10000 segments)
2020/03/16 22:38:42.566 INFO [PinotFSBenchmarkDriver] [main] 4: took 105 ms to listFiles. directory_4 (10000 segments)
2020/03/16 22:38:42.566 INFO [PinotFSBenchmarkDriver] [main] 
2020/03/16 22:38:42.566 INFO [PinotFSBenchmarkDriver] [main] ========= List Files ==========
2020/03/16 22:38:42.567 INFO [PinotFSBenchmarkDriver] [main] Created file:/Users/snlee/benchmark/base/listTest for list test...
2020/03/16 22:38:42.571 INFO [PinotFSBenchmarkDriver] [main] Took 5 ms to create 10 segments.
2020/03/16 22:38:42.571 INFO [PinotFSBenchmarkDriver] [main] 0: took 0 ms to listFiles.
2020/03/16 22:38:42.572 INFO [PinotFSBenchmarkDriver] [main] 1: took 1 ms to listFiles.
2020/03/16 22:38:42.572 INFO [PinotFSBenchmarkDriver] [main] 2: took 0 ms to listFiles.
2020/03/16 22:38:42.572 INFO [PinotFSBenchmarkDriver] [main] 3: took 0 ms to listFiles.
2020/03/16 22:38:42.573 INFO [PinotFSBenchmarkDriver] [main] 4: took 1 ms to listFiles.
2020/03/16 22:38:42.573 INFO [PinotFSBenchmarkDriver] [main] 
========= Uploads and Downloads ==========
2020/03/16 22:38:42.573 INFO [PinotFSBenchmarkDriver] [main] Created file:/Users/snlee/benchmark/base/copyFiles for copy test...
2020/03/16 22:38:42.713 INFO [PinotFSBenchmarkDriver] [main] 0: took 138 ms to copyFromLocal, fileSize: 100 MB.
2020/03/16 22:38:42.850 INFO [PinotFSBenchmarkDriver] [main] 1: took 137 ms to copyFromLocal, fileSize: 100 MB.
2020/03/16 22:38:42.979 INFO [PinotFSBenchmarkDriver] [main] 2: took 129 ms to copyFromLocal, fileSize: 100 MB.
2020/03/16 22:38:43.096 INFO [PinotFSBenchmarkDriver] [main] 3: took 116 ms to copyFromLocal, fileSize: 100 MB.
2020/03/16 22:38:43.206 INFO [PinotFSBenchmarkDriver] [main] 4: took 110 ms to copyFromLocal, fileSize: 100 MB.
2020/03/16 22:38:43.206 INFO [PinotFSBenchmarkDriver] [main] 
2020/03/16 22:38:43.329 INFO [PinotFSBenchmarkDriver] [main] 0: took 123 ms to copyToLocal, fileSize: 100 MB.
2020/03/16 22:38:43.440 INFO [PinotFSBenchmarkDriver] [main] 1: took 110 ms to copyToLocal, fileSize: 100 MB.
2020/03/16 22:38:43.553 INFO [PinotFSBenchmarkDriver] [main] 2: took 112 ms to copyToLocal, fileSize: 100 MB.
2020/03/16 22:38:43.668 INFO [PinotFSBenchmarkDriver] [main] 3: took 115 ms to copyToLocal, fileSize: 100 MB.
2020/03/16 22:38:43.782 INFO [PinotFSBenchmarkDriver] [main] 4: took 114 ms to copyToLocal, fileSize: 100 MB.
2020/03/16 22:38:43.783 INFO [PinotFSBenchmarkDriver] [main] 
2020/03/16 22:38:43.890 INFO [PinotFSBenchmarkDriver] [main] 0: took 107 ms to copy, fileSize: 100 MB.
2020/03/16 22:38:43.993 INFO [PinotFSBenchmarkDriver] [main] 1: took 102 ms to copy, fileSize: 100 MB.
2020/03/16 22:38:44.103 INFO [PinotFSBenchmarkDriver] [main] 2: took 109 ms to copy, fileSize: 100 MB.
2020/03/16 22:38:44.212 INFO [PinotFSBenchmarkDriver] [main] 3: took 108 ms to copy, fileSize: 100 MB.
2020/03/16 22:38:44.320 INFO [PinotFSBenchmarkDriver] [main] 4: took 108 ms to copy, fileSize: 100 MB.
2020/03/16 22:38:44.321 INFO [PinotFSBenchmarkDriver] [main] 
2020/03/16 22:38:44.321 INFO [PinotFSBenchmarkDriver] [main] 0: took 0 ms to rename, fileSize: 100 MB.
2020/03/16 22:38:44.321 INFO [PinotFSBenchmarkDriver] [main] 1: took 0 ms to rename, fileSize: 100 MB.
2020/03/16 22:38:44.322 INFO [PinotFSBenchmarkDriver] [main] 2: took 1 ms to rename, fileSize: 100 MB.
2020/03/16 22:38:44.322 INFO [PinotFSBenchmarkDriver] [main] 3: took 0 ms to rename, fileSize: 100 MB.
2020/03/16 22:38:44.322 INFO [PinotFSBenchmarkDriver] [main] 4: took 0 ms to rename, fileSize: 100 MB.
2020/03/16 22:38:44.322 INFO [PinotFSBenchmarkDriver] [main] 
2020/03/16 22:38:44.334 INFO [PinotFSBenchmarkDriver] [main] 0: took 12 ms to delete, fileSize: 100 MB.
2020/03/16 22:38:44.345 INFO [PinotFSBenchmarkDriver] [main] 1: took 10 ms to delete, fileSize: 100 MB.
2020/03/16 22:38:44.358 INFO [PinotFSBenchmarkDriver] [main] 2: took 13 ms to delete, fileSize: 100 MB.
2020/03/16 22:38:44.370 INFO [PinotFSBenchmarkDriver] [main] 3: took 12 ms to delete, fileSize: 100 MB.
2020/03/16 22:38:44.383 INFO [PinotFSBenchmarkDriver] [main] 4: took 12 ms to delete, fileSize: 100 MB.
2020/03/16 22:38:46.620 INFO [PinotFSBenchmarkDriver] [main] Working directories have been cleaned up successfully.

@codecov-io
Copy link

codecov-io commented Mar 17, 2020 •
edited

Codecov Report

Merging #5160 into master will decrease coverage by 21.09%.
The diff coverage is n/a.

Impacted file tree graph

@@              Coverage Diff              @@
##             master    #5160       +/-   ##
=============================================
- Coverage     66.02%   44.93%   -21.10%     
=============================================
  Files          1050     1052        +2     
  Lines         54117    54170       +53     
  Branches       8071     8078        +7     
=============================================
- Hits          35729    24339    -11390     
- Misses        15734    27732    +11998     
+ Partials       2654     2099      -555
Impacted Files Coverage Δ Complexity Δ
...ava/org/apache/pinot/spi/data/MetricFieldSpec.java 0.00% <0.00%> (-100.00%) 0.00% <0.00%> (ø%)
.../org/apache/pinot/spi/data/DimensionFieldSpec.java 0.00% <0.00%> (-100.00%) 0.00% <0.00%> (ø%)
.../org/apache/pinot/spi/data/readers/FileFormat.java 0.00% <0.00%> (-100.00%) 0.00% <0.00%> (ø%)
...va/org/apache/pinot/common/config/FieldConfig.java 0.00% <0.00%> (-100.00%) 0.00% <0.00%> (ø%)
.../org/apache/pinot/common/http/MultiGetRequest.java 0.00% <0.00%> (-100.00%) 0.00% <0.00%> (ø%)
.../org/apache/pinot/common/lineage/SegmentGroup.java 0.00% <0.00%> (-100.00%) 0.00% <0.00%> (ø%)
.../apache/pinot/spi/utils/retry/BaseRetryPolicy.java 0.00% <0.00%> (-100.00%) 0.00% <0.00%> (ø%)
...apache/pinot/spi/stream/StreamConsumerFactory.java 0.00% <0.00%> (-100.00%) 0.00% <0.00%> (ø%)
...apache/pinot/spi/stream/StreamMessageMetadata.java 0.00% <0.00%> (-100.00%) 0.00% <0.00%> (ø%)
...g/apache/pinot/server/api/resources/ErrorInfo.java 0.00% <0.00%> (-100.00%) 0.00% <0.00%> (ø%)
... and 574 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update c81a656...ca924cf. Read the comment docs.

mcvsubbu
mcvsubbu reviewed Mar 17, 2020
View reviewed changes
pinot-tools/src/main/java/org/apache/pinot/tools/filesystem/PinotFSBenchmarkDriver.java Outdated Show resolved Hide resolved
pinot-tools/src/main/java/org/apache/pinot/tools/filesystem/PinotFSBenchmarkDriver.java Outdated Show resolved Hide resolved
pinot-tools/src/main/java/org/apache/pinot/tools/filesystem/PinotFSBenchmarkDriver.java Outdated

private static final int DEFAULT_NUM_SEGMENTS_FOR_LIST_TEST = 1000;
private static final int DEFAULT_DATA_SIZE_IN_MB_FOR_COPY_TEST = 1024; // 1GB
private static final int DEFAULT_RETRY = 5; // 5
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This does not seem to be retry count (I was associating the term retry with failures). DEFAULT_NUM_OPS?

pinot-tools/src/main/java/org/apache/pinot/tools/filesystem/PinotFSBenchmarkDriver.java Outdated Show resolved Hide resolved
pinot-tools/src/main/java/org/apache/pinot/tools/filesystem/PinotFSBenchmarkDriver.java Outdated Show resolved Hide resolved
mcvsubbu
mcvsubbu approved these changes Mar 20, 2020
View reviewed changes
Copy link
Contributor

@mcvsubbu mcvsubbu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm,
it may be better to add the other defaiults in arguments as well (num ops and num directories for file list)

@snleee snleee merged commit ed235d2 into apache:master Mar 26, 2020
@snleee snleee deleted the filesystem-benchmark branch March 26, 2020 01:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@mcvsubbu mcvsubbu mcvsubbu approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
3 participants
@snleee @codecov-io @mcvsubbu