Add block download progress to metrics UI #2484

str4d · 2017-06-26T23:17:19Z

No description provided.

daira

utACK modulo comments.

daira · 2017-07-03T10:14:12Z

src/metrics.cpp

+        double averageSpacing = (targetSpacing + checkpointSpacing) / 2;
+        int netheight = medianHeight + ((GetTime() - tipmediantime) / averageSpacing);
+        // Round to nearest ten to reduce noise
+        netheight = (netheight / 10) * 10;


This is not rounding to the nearest 10; it's rounding down to a multiple of 10.

daira · 2017-07-03T10:23:05Z

src/metrics.cpp

+        // checkpoint, and use that to estimate the current network height.
+        auto chainParams = Params();
+        auto checkpointData = chainParams.Checkpoints();
+        int medianHeight = height - 5;


Use CBlockIndex::nMedianTimeSpan / 2 instead of 5.

str4d · 2017-07-11T14:51:49Z

@daira comments addressed.

daira

utACK. I'm not sure what our testing policy is for changes that obviously only affect non-essential output; do we need to test this @nathan-at-least ?

nathan-at-least

Requested change: unit tests of arithmetic.

nathan-at-least · 2017-07-25T21:13:10Z

src/metrics.cpp

+        int netheight = medianHeight + ((GetTime() - tipmediantime) / averageSpacing);
+        // Round to nearest ten to reduce noise
+        netheight = ((netheight + 5) / 10) * 10;
+        int downloadPercent = height * 100 / netheight;


Imagine there were some bug in the arithmetic. Could we factor out the "sensor" code (which reads from blockchain state, ex IsInitialBlockDownload() and chainParams.Checkpoints, arithmetic functional code, and display code, and then unittest at least the arithmetic functional code?

Then if there were an arithmetic bug, that would imply either we missed a test specification, or the specification was buggy.

@str4d will write a unit test for this.

str4d · 2017-07-26T00:52:56Z

Refactored the code and added unit tests. I force-pushed to squash this into the existing commit.

daira

ut(ACK+cov)

ebfull · 2017-07-26T15:19:19Z

src/metrics.cpp

+    // We average the target spacing with the observed spacing to the last
+    // checkpoint, and use that to estimate the current network height.
+    int medianHeight = height - CBlockIndex::nMedianTimeSpan / 2;
+    double checkpointSpacing = (double (tipmediantime - timeLastCheckpoint)) / (medianHeight - heightLastCheckpoint);


Under what conditions could this potentially divide by zero?

Also, can checkpointSpacing become negative in pathological cases?

double division-by-zero is well-defined (for IEEE arithmetic) to produce infinity or -infinity. Dividing a double by an infinity gives zero.

Provided we have floating point non-stop mode enabled (via feholdexcept or fesetexceptflag). I assumed we did because we use infinities in the fee handling code.

I suggest avoiding the issue by using:

double checkpointSpacing = max(1.0d, (double (tipmediantime - timeLastCheckpoint)) / max(1.0d, double (medianHeight - heightLastCheckpoint)));

(and adding tests that exercise the clamping).

@ebfull raises a good point. I have addressed this in a different way: while the current height is <= the last checkpoint, estimate the checkpoint spacing using the interval [genesis, lastCheckpoint] instead of [lastCheckpoint, tip].

nathan-at-least

utACK.

There are some pathological cases and I'm not sure if they are reachable.

If not, an improvement would be to add precondition assertions to guard against undefined behavior (if that is indeed C++ divide by zero behavior). Otherwise test cases for them and error recovery would be good. I don't see either of those as requirements for merging this.

nathan-at-least · 2017-07-27T15:27:57Z

src/metrics.cpp

+    // We average the target spacing with the observed spacing to the last
+    // checkpoint, and use that to estimate the current network height.
+    int medianHeight = height - CBlockIndex::nMedianTimeSpan / 2;
+    double checkpointSpacing = (double (tipmediantime - timeLastCheckpoint)) / (medianHeight - heightLastCheckpoint);


Also, can checkpointSpacing become negative in pathological cases?

nathan-at-least · 2017-07-27T15:29:39Z

src/metrics.cpp

+    int medianHeight = height - CBlockIndex::nMedianTimeSpan / 2;
+    double checkpointSpacing = (double (tipmediantime - timeLastCheckpoint)) / (medianHeight - heightLastCheckpoint);
+    double averageSpacing = (targetSpacing + checkpointSpacing) / 2;
+    int netheight = medianHeight + ((GetTime() - tipmediantime) / averageSpacing);


Under what conditions could averageSpacing == 0? Three cases to consider:

targetSpacing == checkpointSpacing == 0

One is negative (which seem unexpected).

The other is negative.

nathan-at-least · 2017-07-27T15:30:27Z

src/metrics.cpp

+        Checkpoints::GetTotalBlocksEstimate(checkpointData),
+        checkpointData.nTimeLastCheckpoint,
+        chainParams.GetConsensus().nPowTargetSpacing);
+}


I personally like factoring out the "I/O" into this outer function. What's your opinion @str4d?

I like it too.

Could you clarify what you mean? If you mean pulling the stdio calls from the parent into here, I would like to point out that having them in their current function makes aligning the UI significantly easier, as all the common calls are together (in this case, inside printStats()).

nathan-at-least · 2017-07-27T15:37:19Z

src/metrics.cpp

+        height, tipmediantime,
+        Checkpoints::GetTotalBlocksEstimate(checkpointData),
+        checkpointData.nTimeLastCheckpoint,
+        chainParams.GetConsensus().nPowTargetSpacing);


Answering my edge-case questions above:

nPowTargetSpacing is defined in one place as an int64_t and is our target block time and it would be quite unlikely we wouldn't notice changing this to negative.

I haven't determined what the pathological conditions for checkpointData.nTimeLastCheckpoint or Checkpoints::GetTotalBlockEstimate() would be.

I suspect the worst case behavior for those pathological conditions would be just incorrect display, although it sounds like division by 0 can be undefined behavior.

daira · 2017-07-28T06:06:19Z

Note that we're behind schedule on the release and this is the last feature PR. It's in danger of getting kicked out if the edge case issues are not resolved today (Friday).

Corrections are to the median block times, which were generated by subtracting CBlockIndex::nMedianTimeSpan / 2 from the block height and then multiplying by the target spacing. GetMedianTimePast() takes an array sorted by std::sort() and returns element CBlockIndex::nMedianTimeSpan / 2, meaning that if CBlockIndex::nMedianTimeSpan is odd (which it is), there is an out-by-one error in the subtraction.

daira · 2017-07-29T04:54:56Z

ut(ACK+cov)

daira · 2017-07-29T04:55:15Z

@zkbot r+

zkbot · 2017-07-29T04:55:15Z

📌 Commit 92bfde0 has been approved by daira

zkbot · 2017-07-29T04:55:20Z

⌛ Testing commit 92bfde0 with merge 10acd8a...

Add block download progress to metrics UI

zkbot · 2017-07-29T05:56:26Z

☀️ Test successful - pr-merge
Approved by: daira
Pushing 10acd8a to master...

str4d added A-metrics-screen Area: Metrics screen usability labels Jun 26, 2017

str4d added this to the 1.0.11 Release milestone Jun 26, 2017

daira removed this from the 1.0.11 Release milestone Jul 3, 2017

daira approved these changes Jul 3, 2017

View reviewed changes

str4d force-pushed the block-download-completion branch from 92bea1d to 4d9c9e0 Compare July 11, 2017 14:51

daira approved these changes Jul 11, 2017

View reviewed changes

daira added this to the 1.0.11 Release milestone Jul 25, 2017

daira assigned str4d Jul 25, 2017

nathan-at-least suggested changes Jul 25, 2017

View reviewed changes

daira assigned str4d and unassigned str4d Jul 25, 2017

Add block download progress to metrics UI

047aec1

str4d force-pushed the block-download-completion branch from 4d9c9e0 to 047aec1 Compare July 26, 2017 00:51

daira approved these changes Jul 26, 2017

View reviewed changes

ebfull reviewed Jul 26, 2017

View reviewed changes

nathan-at-least approved these changes Jul 27, 2017

View reviewed changes

str4d added 2 commits July 28, 2017 10:45

Improve network height estimation

92bfde0

tromer mentioned this pull request Jul 28, 2017

Report additional status/alert conditions via metrics screen and RPC #2206

Open

zkbot added a commit that referenced this pull request Jul 29, 2017

Auto merge of #2484 - str4d:block-download-completion, r=daira

10acd8a

Add block download progress to metrics UI

zkbot merged commit 92bfde0 into zcash:master Jul 29, 2017

str4d deleted the block-download-completion branch August 3, 2017 13:56

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add block download progress to metrics UI #2484

Add block download progress to metrics UI #2484

str4d commented Jun 26, 2017

daira left a comment

daira Jul 3, 2017

daira Jul 3, 2017

str4d commented Jul 11, 2017

daira left a comment

nathan-at-least left a comment

nathan-at-least Jul 25, 2017

daira Jul 25, 2017

str4d commented Jul 26, 2017

daira left a comment

ebfull Jul 26, 2017

nathan-at-least Jul 27, 2017

daira Jul 27, 2017 •

edited

Loading

daira Jul 28, 2017

daira Jul 28, 2017

str4d Jul 28, 2017

nathan-at-least left a comment

nathan-at-least Jul 27, 2017

nathan-at-least Jul 27, 2017

nathan-at-least Jul 27, 2017

daira Jul 28, 2017

str4d Jul 28, 2017

nathan-at-least Jul 27, 2017

daira commented Jul 28, 2017 •

edited

Loading

daira commented Jul 29, 2017

daira commented Jul 29, 2017

zkbot commented Jul 29, 2017

zkbot commented Jul 29, 2017

zkbot commented Jul 29, 2017

Add block download progress to metrics UI #2484

Add block download progress to metrics UI #2484

Conversation

str4d commented Jun 26, 2017

daira left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

str4d commented Jul 11, 2017

daira left a comment

Choose a reason for hiding this comment

nathan-at-least left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

str4d commented Jul 26, 2017

daira left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

daira Jul 27, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

nathan-at-least left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

daira commented Jul 28, 2017 • edited Loading

daira commented Jul 29, 2017

daira commented Jul 29, 2017

zkbot commented Jul 29, 2017

zkbot commented Jul 29, 2017

zkbot commented Jul 29, 2017

daira Jul 27, 2017 •

edited

Loading

daira commented Jul 28, 2017 •

edited

Loading