Optimize Read Channels #940

Maithem · 2017-09-11T22:58:57Z

Use a single channel per segment for uncached reads.

coveralls · 2017-09-11T23:21:09Z

Changes Unknown when pulling ba6a205 on optReads into ** on master**.

corfudb-performance · 2017-09-12T00:07:56Z

Results automatically generated by CorfuDB Benchmark Framework to assess the performance of this pull request for commit ba6a205.

*** 0.0% transaction FAILURE rate for NonConflictingTx+Scan workload, 1 threads, Disk mode
*** 0.0% transaction FAILURE rate for NonConflictingTx+Scan workload, 5 threads, Disk mode
*** 0.0% transaction FAILURE rate for NonConflictingTx+Scan workload, 10 threads, Disk mode
*** 0.0% transaction FAILURE rate for NonConflictingTx+Iterator workload, 1 threads, Disk mode
*** 0.0% transaction FAILURE rate for NonConflictingTx+Iterator workload, 5 threads, Disk mode
*** 0.0% transaction FAILURE rate for NonConflictingTx+Iterator workload, 10 threads, Disk mode
*** 0.0% transaction FAILURE rate for NonConflictingTx workload, 1 threads, Disk mode
*** 0.0% transaction FAILURE rate for NonConflictingTx workload, 5 threads, Disk mode
*** 0.0% transaction FAILURE rate for NonConflictingTx workload, 10 threads, Disk mode

An interactive dashboard with Pull Request Performance Metrics for ALL cluster types and numbers of threads in run, is available at:
Pull Request #940 Graphs

no2chem

LGTM

Use a single channel per segment for to read uncached entries.

Maithem · 2017-12-13T03:40:50Z

@no2chem Rebased.

corfu-repo · 2017-12-13T05:07:32Z

infrastructure/src/main/java/org/corfudb/infrastructure/log/SegmentHandle.java

+            try {
+                channel.force(true);
+                channel.close();
+                channel = null;


Remove this useless assignment to local variable "channel".

corfu-repo · 2017-12-13T05:07:32Z

infrastructure/src/main/java/org/corfudb/infrastructure/log/SegmentHandle.java

+    @NonNull
+    String fileName;
+
+    private Map<Long, AddressMetaData> knownAddresses = new ConcurrentHashMap();


Remove the "knownAddresses" field and declare it as a local variable in the relevant methods.

corfu-repo · 2017-12-13T05:07:33Z

SonarQube analysis reported 5 issues

1 major
4 minor

Watch the comments in this conversation to review them.

1 extra issue

Note: The following issues were found on lines that were not modified in the pull request. Because these issues can't be reported as line comments, they are summarized here:

StreamLogFiles.java#L245: Remove this unnecessary cast to "Collection".

corfu-repo · 2017-12-13T05:07:33Z

infrastructure/src/main/java/org/corfudb/infrastructure/log/SegmentHandle.java

+    String fileName;
+
+    private Map<Long, AddressMetaData> knownAddresses = new ConcurrentHashMap();
+    private Set<Long> trimmedAddresses = Collections.newSetFromMap(new ConcurrentHashMap<>());


Remove the "trimmedAddresses" field and declare it as a local variable in the relevant methods.

corfu-repo · 2017-12-13T05:07:33Z

infrastructure/src/main/java/org/corfudb/infrastructure/log/SegmentHandle.java

+
+    private Map<Long, AddressMetaData> knownAddresses = new ConcurrentHashMap();
+    private Set<Long> trimmedAddresses = Collections.newSetFromMap(new ConcurrentHashMap<>());
+    private Set<Long> pendingTrims = Collections.newSetFromMap(new ConcurrentHashMap<>());


Remove the "pendingTrims" field and declare it as a local variable in the relevant methods.

codecov · 2017-12-13T05:08:04Z

Codecov Report

Merging #940 into master will increase coverage by 0.16%.
The diff coverage is 88.13%.

@@            Coverage Diff             @@
##           master     #940      +/-   ##
==========================================
+ Coverage   66.85%   67.01%   +0.16%     
==========================================
  Files         211      212       +1     
  Lines        9856     9853       -3     
  Branches      986      983       -3     
==========================================
+ Hits         6589     6603      +14     
+ Misses       2887     2871      -16     
+ Partials      380      379       -1

Impacted Files	Coverage Δ
.../org/corfudb/infrastructure/log/SegmentHandle.java	`84.37% <84.37%> (ø)`
...org/corfudb/infrastructure/log/StreamLogFiles.java	`83.62% <92.59%> (+0.03%)`	⬆️
...org/corfudb/runtime/view/LayoutManagementView.java	`50% <0%> (-2.35%)`	⬇️
...rg/corfudb/runtime/object/VersionLockedObject.java	`89.67% <0%> (+0.54%)`	⬆️
...main/java/org/corfudb/runtime/view/LayoutView.java	`84.84% <0%> (+0.75%)`	⬆️
.../org/corfudb/protocols/wireprotocol/IMetadata.java	`86.95% <0%> (+2.89%)`	⬆️
...fudb/infrastructure/orchestrator/Orchestrator.java	`76.81% <0%> (+4.34%)`	⬆️
.../org/corfudb/runtime/object/CorfuCompileProxy.java	`68.78% <0%> (+6.93%)`	⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 287ada9...5db09d7. Read the comment docs.

corfudb-performance · 2017-12-13T05:16:39Z

Results automatically generated by CorfuDB Benchmark Framework to assess the performance of this pull request for commit 5db09d7.

*** 0.0% transaction FAILURE rate for NonConflictingTx+Scan workload, 1 threads, Disk mode
*** 0.0% transaction FAILURE rate for NonConflictingTx+Scan workload, 5 threads, Disk mode
*** 0.0% transaction FAILURE rate for NonConflictingTx+Scan workload, 10 threads, Disk mode
*** 0.0% transaction FAILURE rate for NonConflictingTx workload, 1 threads, Disk mode
*** 0.0% transaction FAILURE rate for NonConflictingTx workload, 5 threads, Disk mode
*** 0.0% transaction FAILURE rate for NonConflictingTx workload, 10 threads, Disk mode

An interactive dashboard with Pull Request Performance Metrics for ALL cluster types and numbers of threads in run, is available at:
Pull Request #940 Graphs

Maithem added the typ: enhancement label Sep 11, 2017

Maithem requested review from no2chem and medhavidhawan September 11, 2017 22:58

no2chem added this to the 0.2.0 milestone Dec 5, 2017

no2chem mentioned this pull request Dec 5, 2017

Memory mapped read/address-space implementation #1040

Closed

4 tasks

no2chem assigned Maithem Dec 5, 2017

no2chem approved these changes Dec 12, 2017

View reviewed changes

Optimize Read Channels

5db09d7

Use a single channel per segment for to read uncached entries.

Maithem force-pushed the optReads branch from ba6a205 to 5db09d7 Compare December 13, 2017 03:40

corfu-repo reviewed Dec 13, 2017

View reviewed changes

Maithem merged commit 4acfbe4 into master Dec 13, 2017

Maithem deleted the optReads branch December 13, 2017 07:30

Maithem mentioned this pull request Dec 14, 2017

Orchestrator Service is causing builds to fail #1073

Closed

Maithem restored the optReads branch December 14, 2017 05:33

Maithem mentioned this pull request Dec 14, 2017

Fix Orchestrator Query #1084

Merged

4 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Optimize Read Channels #940

Optimize Read Channels #940

Maithem commented Sep 11, 2017

coveralls commented Sep 11, 2017

corfudb-performance commented Sep 12, 2017

no2chem left a comment

Maithem commented Dec 13, 2017

corfu-repo Dec 13, 2017

corfu-repo Dec 13, 2017

corfu-repo commented Dec 13, 2017

corfu-repo Dec 13, 2017

corfu-repo Dec 13, 2017

codecov bot commented Dec 13, 2017 •

edited

Loading

corfudb-performance commented Dec 13, 2017

Optimize Read Channels #940

Optimize Read Channels #940

Conversation

Maithem commented Sep 11, 2017

coveralls commented Sep 11, 2017

corfudb-performance commented Sep 12, 2017

no2chem left a comment

Choose a reason for hiding this comment

Maithem commented Dec 13, 2017

corfu-repo Dec 13, 2017

Choose a reason for hiding this comment

corfu-repo Dec 13, 2017

Choose a reason for hiding this comment

corfu-repo commented Dec 13, 2017

1 extra issue

corfu-repo Dec 13, 2017

Choose a reason for hiding this comment

corfu-repo Dec 13, 2017

Choose a reason for hiding this comment

codecov bot commented Dec 13, 2017 • edited Loading

Codecov Report

corfudb-performance commented Dec 13, 2017

codecov bot commented Dec 13, 2017 •

edited

Loading