New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

RPC: Introduce getblockstats to plot things #10757

Merged
merged 4 commits into from May 23, 2018

Conversation

@jtimon
Member

jtimon commented Jul 6, 2017

It returns per block statistics about several things. It should be easy to add more if people think of other things to add or remove some if I went too far (but once written, why not keep it? EDIT: answer: not to test or maintain them).

The currently available options are: minfee,maxfee,totalfee,minfeerate,maxfeerate,avgfee,avgfeerate,txs,ins,outs (EDIT: see updated list in the rpc call documentation)

For the x axis, one can use height or block.nTime (I guess I could add mediantime if there's interest [EDIT: nobody showed interest but I implemented mediantime nonetheless, in fact there's no distinction between x or y axis anymore, that's for the caller to judge]).

To calculate fees, -txindex is required.

@jtimon jtimon force-pushed the jtimon:b15-rpc-plotter branch from 457c30d to 53ea4de Jul 7, 2017

@clarkmoody

This comment has been minimized.

clarkmoody commented Jul 7, 2017

Some ideas for additions:

  • Non-fee total output amount
  • Coinbase reward
  • Money supply including this block
  • Transaction weight txweight (it can be derived from existing fields, however)

I would prefer to see both time and mediantime returned, since they are available.

Should we return non-independent fields, such as avgfee when also including totalfee and txs?

I find that for bitcoin-related data, the median is often more useful than the average of a distribution. Including medianweight, medianfee, medianfeerate, medianoutput etc would expose these useful quantities to the user.

@jnewbery

This comment has been minimized.

Member

jnewbery commented Jul 7, 2017

but once written, why not keep it?

Because more code => more bugs and more maintenance effort. I prefer:

If it's not really needed, why add it?

This is perhaps a nice-to-have, but since #8704, getblock can return all transactions in a block (without requiring txindex). Those can then be parsed and analysed offline.

Is there a compelling use-case I'm missing here? This seems like a feature only a small subset of users would be interested in, in which case an offline tools seems more appropriate.

Sorry - not meaning to be negative, but my default reaction to new RPCs/arguments tends towards NACK unless I can see a compelling and widespread use-case.

@clarkmoody

This comment has been minimized.

clarkmoody commented Jul 7, 2017

This is perhaps a nice-to-have, but since #8704, getblock can return all transactions in a block (without requiring txindex). Those can then be parsed and analysed offline.

This code pulls each transaction input's previous outpoint in order to compute transaction fees. Replicating that in RPC would require thousands of calls for most blocks.

@jnewbery

This comment has been minimized.

Member

jnewbery commented Jul 7, 2017

This code pulls each transaction input's previous outpoint

Ah yes, of course. Concept ACK in that case. Doing this with getblock / getrawtransaction is infeasible.

EDIT: I'm going to reverse myself again: I don't think +700 lines is worthwhile for something with limited usage for most users. I'm -0 on this.

@jtimon

This comment has been minimized.

Member

jtimon commented Jul 7, 2017

Because more code => more bugs and more maintenance effort. I prefer:

Sure, but I mean, removing for example the avgfee or avgfeerate won't safe much code or testing code, just a few lines. Forget I said this, if there's specific functions to remove because nobody will want them, let's remove those and focus on the ones people want. Adding specific things only a few people want can also happen in their own branches, so it's no big deal.

The only use case is gather statistics, presumably to plot things, create charts. That is, at least, compelling to me, but I don't think that will have widespread usage. I also don't think all rpc calls have it. Is getchaintxstats, for example, a widespread use case?

If that's enough reason not to merge this, it's fine, I can maintain it as a separate branch that I periodically rebase, it is simple enough, so that won't be a big deal. On the other hand, if I can get it reviewed and merged it'll be less work for me in the long run and I also get the review.

Non-fee total output amount
Coinbase reward

Sounds good.

Money supply including this block

Mhmm, it would be simpler to calculate here from start to end here than from genesis. But it's pretty trivial to write a function in any language that returns the total supply for a given height without access to any historic data. Unless you are talking about discounting op_return outputs or something like that. I don't think this is very interesting here. Perhaps that can be done in getchaintxstats ?

Transaction weight txweight (it can be derived from existing fields, however)

In fact I'm using weight for everything. I should s/size/weight/ and probably also show size separately.
Maybe separate feerates in by weight and serialize size? I don't know...

I would prefer to see both time and mediantime returned, since they are available.

Yeah, the mediantime takes a little bit longer to be calculated but not much and one can always disable anything. In fact, the height and time shouldn't be treated in any special way for being "the x axis" and should be allowed to be disabled like the rest.

Should we return non-independent fields, such as avgfee when also including totalfee and txs?

This is a good question. This is mostly what I meant by "why not if it's this easy?".
But yeah, I guess non-independent are good candidates to be removed.

re median: yeah, that sounds interesting too, good idea!

@clarkmoody

This comment has been minimized.

clarkmoody commented Jul 7, 2017

Mhmm, it would be simpler to calculate here from start to end here than from genesis. But it's pretty trivial to write a function in any language that returns the total supply for a given height without access to any historic data. Unless you are talking about discounting op_return outputs or something like that. I don't think this is very interesting here. Perhaps that can be done in getchaintxstats ?

I was thinking of the more trivial version, rather than the supply - provably_unspendable version, so keeping that as external code makes more sense. Maintaining the sum of spendable outputs against block height is a much more ambitious idea, and it may make sense in the future. However, it is probably out of scope of this PR.

@promag

Partial review, suggestion to use std::set.

Nit, rename allowed_plot_values to valid_plot_values.
Nit, rename getperblockstats to just getblockstats?

UniValue getperblockstats(const JSONRPCRequest& request)
{
std::string str_allowed_plot_values = "minfee,maxfee,totalfee,minfeerate,maxfeerate,avgfee,avgfeerate,txs,ins,outs";

This comment has been minimized.

@promag

promag Jul 7, 2017

Member

Remove.

UniValue getperblockstats(const JSONRPCRequest& request)
{
std::string str_allowed_plot_values = "minfee,maxfee,totalfee,minfeerate,maxfeerate,avgfee,avgfeerate,txs,ins,outs";
std::vector<std::string> allowed_plot_values;

This comment has been minimized.

@promag

promag Jul 7, 2017

Member
std::set<std::string> allowed_plot_values = {"minfee", "maxfee", "..."};
{
std::string str_allowed_plot_values = "minfee,maxfee,totalfee,minfeerate,maxfeerate,avgfee,avgfeerate,txs,ins,outs";
std::vector<std::string> allowed_plot_values;
boost::split(allowed_plot_values, str_allowed_plot_values, boost::is_any_of(","));

This comment has been minimized.

@promag

promag Jul 7, 2017

Member

Remove.

"\nArguments:\n"
"1. \"start\" (numeric, required) The height of the block that starts the window.\n"
"2. \"end\" (numeric, optional) The height of the block that ends the window (default: current tip).\n"
"3. \"plotvalues\" (string, optional) Values to plot (comma separated), default(all): " + str_allowed_plot_values +

This comment has been minimized.

@promag

promag Jul 7, 2017

Member
"..." + boost::join(allowed_plot_values, ",")
"3. \"plotvalues\" (string, optional) Values to plot (comma separated), default(all): " + str_allowed_plot_values +
"\nResult:\n"
"{\n"
"}\n"

This comment has been minimized.

@promag

promag Jul 7, 2017

Member

Missing result example.

"}\n"
"\nExamples:\n"
+ HelpExampleCli("getperblockstats", "1000 1000 \"minfeerate,avgfeerate\"")
+ HelpExampleRpc("getperblockstats", "1000 1000 \"maxfeerate,avgfeerate\"")

This comment has been minimized.

@promag

promag Jul 7, 2017

Member

Duplicate.

This comment has been minimized.

@jtimon

jtimon Jul 8, 2017

Member

One is HelpExampleCli and the other is HelpExampleRpc

This comment has been minimized.

@promag

promag Jul 8, 2017

Member

Ops sorry, overlooked it.

if (request.params.size() > 2) {
str_plot_values = request.params[2].get_str();
}
std::vector<std::string> plot_values;

This comment has been minimized.

@promag

promag Jul 7, 2017

Member
std::set<std::string> plot_values;
if (request.params.size() > 2) {
  boost::split(plot_values, request.params[2].get_str(), boost::is_any_of(","));

  // only validate in this case
  // ... 
} else {
  plot_values = allowed_plot_values;
}
}
}
static bool IsAllowedPlotValue(const std::string& plot_value, std::vector<std::string>& allowed_plot_values)

This comment has been minimized.

@promag

promag Jul 7, 2017

Member

Remove.

std::vector<std::string> plot_values;
boost::split(plot_values, str_plot_values, boost::is_any_of(","));
for (const std::string plot_value : plot_values) {
if (!IsAllowedPlotValue(plot_value, allowed_plot_values)) {

This comment has been minimized.

@promag

promag Jul 7, 2017

Member
if (allowed_plot_values.count(plot_value) == 0) {

This comment has been minimized.

@jtimon

jtimon Jul 8, 2017

Member

duh, I was so much over-complicating things so much for no good reason...thank you!

@jtimon jtimon force-pushed the jtimon:b15-rpc-plotter branch from 53ea4de to c316a9f Jul 8, 2017

@jtimon

This comment has been minimized.

Member

jtimon commented Jul 8, 2017

Thanks again for the great feedback!

@promag I think I solved all your nits except for #10757 (comment)

@clarkmoody I think I added most of your suggestions, explicitly excluding anything that involved accumulations neither from height=1 nor from height=start.
The former potentially implies a world of complexity and the latter can be trivially calculated on the visual side: I would completely discard any accumulator redundancy in this rpc beforehand.

And for the rest of the redundancies, @jnewbery and @clarkmoody - thanks again for pointing it out -, it's never too late to remove them before merging like a trivial squash and it's never too soon to start saying which ones you would bikesay* out first. Also bikesay the names for the curves and even the order in the list (duplicated for c++ and python).

In the meantime, I embraced redundancy since, as said, it will be trivial for me to remove later. And also the pertinent optimizations to skip calculations when plot_values.count("minfee") == 0 or actually only when the extra calculation is more expensive than the searching in plot_values which is a set of strings.

For example, we have blockfees, reward, subsidy, complying with consensus rule reward == blockfees + subsidy. Only 2 of the 3 are necessary, at least one is redundant. My personal preference is removing either subsidy or reward or subsidy, but not blockfees. But at said once written there's no problem with me in just making sure their tests don't surprise me until we decide which ones didn't deserve it.

Which one seems bikesaying in principle. But not in this case.
blockfees/total_fees serves for other calculations like avgfeerate. Let's not remove that one, just rename it.

But it is more interesting to propose new ones than to rename or vote for removal IMO. I believe the most interesting addition to this point was utxo_size_inc, which would welcomed some review from people who measures sizes more carefully like @sipa , since this doesn't use GetSerializeSize for Coin intentionally, independently of the optimization to read Coin if available in the utxo before calling RpcGetTx. I'm still not sure what to do with pre/post segwit feerates, does anybody care about the pre ones? which one needs the scale factor? none?

REM CalculateTruncatedMedian doesn't need to be a template at this point, but there's no harm being static IMO

EDIT: still some TODOs, mostly documentation and pending decisions

@promag

This comment has been minimized.

Member

promag commented Jul 8, 2017

@jtimon no problem. There are some nits to fix but I'll review more in depth later.

@@ -20,20 +20,22 @@ CFeeRate::CFeeRate(const CAmount& nFeePaid, size_t nBytes_)
nSatoshisPerK = 0;
}
CAmount CFeeRate::GetFee(size_t nBytes_) const
CAmount CFeeRate::GetTruncatedFee(size_t nBytes_) const

This comment has been minimized.

@promag

promag Jul 10, 2017

Member

Drop _? Same below.

This comment has been minimized.

@jtimon

jtimon Jul 11, 2017

Member

Well, yes, conserving the old name only saves 1 line of extra disruption. But I guess if we're touching the variable name we should use the new style. just bytes?

@@ -685,6 +686,22 @@ UniValue getblockheader(const JSONRPCRequest& request)
return blockheaderToJSON(pblockindex);
}
static void ReadBlockCheckPruned(const CBlockIndex* pblockindex, CBlock& block)

This comment has been minimized.

@promag

promag Jul 10, 2017

Member

Keep argument order as ReadBlockFromDisk? Is there a convention for where the output arguments should be?

This comment has been minimized.

@jtimon

jtimon Jul 10, 2017

Member

Not that I know of, but your proposed change sounds good to me.

{
T median;
size_t size = scores.size();
std::sort(scores.begin(), scores.end());

This comment has been minimized.

@promag

promag Jul 10, 2017

Member

Nit, could sort only after size == 1 case.

}
// outpoint (needed for the utxo index) + nHeight + fCoinBase
static const size_t PER_UTXO_OVERHEAD = sizeof(COutPoint) + sizeof(uint32_t) + sizeof(bool);

This comment has been minimized.

@promag

promag Jul 10, 2017

Member

static constexpr ...

This comment has been minimized.

@jtimon

jtimon Jul 10, 2017

Member

what's the gain? https://stackoverflow.com/a/41132221/935325 says it's the same...

This comment has been minimized.

@ryanofsky

ryanofsky Oct 31, 2017

Contributor

what's the gain? https://stackoverflow.com/a/41132221/935325 says it's the same...

Gain is just that constexpr is more descriptive, and that stackoverflow answer isn't really correct. const and constexpr aren't identical even for integers, for example const is valid here:

static const int X = rand();

where constexpr would not be:

static constexpr int X = rand();

Also that stackoverflow answer is relying on special case treatment for integers that doesn't apply to other types like floats. constexpr is just better for declaring compile time constants that const, so we should prefer it.

ReadBlockCheckPruned(pindex, block);
for (const auto& tx : block.vtx) {

This comment has been minimized.

@promag

promag Jul 10, 2017

Member

Remove empty line.

map_stats[plot_value] = UniValue(UniValue::VARR);
}
CBlockIndex* pindex = chainActive[end];

This comment has been minimized.

@promag

promag Jul 10, 2017

Member

Remove.

CBlockIndex* pindex = chainActive[end];
for (int i = end; i >= start; i--) {
UpdateBlockStats(pindex, plot_values, map_stats);

This comment has been minimized.

@promag

promag Jul 10, 2017

Member
UpdateBlockStats(chainActive[i], ...);

This comment has been minimized.

@jtimon

jtimon Jul 10, 2017

Member

This will be slightly less efficient, no?

This comment has been minimized.

@promag

promag Jul 11, 2017

Member

I guess it takes few more cycles but non critical code should be cleaner?

"\nArguments:\n"
"1. \"start\" (numeric, required) The height of the block that starts the window.\n"
"2. \"end\" (numeric, optional) The height of the block that ends the window (default: current tip).\n"
"3. \"plotvalues\" (string, optional) Values to plot (comma separated), default(all): " + boost::join(valid_plot_values, ",") +

This comment has been minimized.

@promag

promag Jul 10, 2017

Member

Replace plotvalues with stats? Also, 3rd argument could be object options?

This comment has been minimized.

@jtimon

jtimon Jul 10, 2017

Member

Isn't the string simpler?

This comment has been minimized.

@promag

promag Jul 11, 2017

Member

Ignore options suggestion.

int end;
if (request.params.size() > 1) {
end = request.params[1].get_int();
if (end < 0 || end > chainActive.Height()) {

This comment has been minimized.

@promag

promag Jul 10, 2017

Member

Nit, negative block could mean end = height - end to avoid early blocks (not new concept here I believe)?

This comment has been minimized.

@jtimon

jtimon Jul 10, 2017

Member

Mhmm, interesting. To be clear you mean start=-10 end=200 would be equivalent to start=190 end=200, right?

This comment has been minimized.

@promag

promag Jul 11, 2017

Member

No, I meant negative values are relative to the tip. To get the stats for the last 10 blocks you would pass start = -10 without querying the current block height.

@jtimon

This comment has been minimized.

Member

jtimon commented Jul 11, 2017

Here are some images generated using this branch in combination with (WIP): https://github.com/jtimon/rpc-explorer

GUI detail:

screenshot_plotter

GUI detail zoom:

screenshot_plotter_zoom

Hide some:

plotter1

Hide more:

plotter2

Fees:

plotterfees

Utxo size increase:

plotter_utxo

@jtimon jtimon changed the title from RPC: Introduce getperblockstats to plot things to RPC: Introduce getblockstats to plot things Jul 11, 2017

@jtimon jtimon force-pushed the jtimon:b15-rpc-plotter branch 3 times, most recently from 79d9907 to 90c7264 Jul 12, 2017

@jtimon

This comment has been minimized.

Member

jtimon commented Jul 12, 2017

Without the documentation for the result it was impossible to distinguish a weird choice to spring discussion from an implementation mistake. Removed the other TODO comments.
Coded more pending suggestions by @promag (hopefully all pending ones? if not, please insist) with some extra bikeshedding derived from s/plotvalues/stats/ and adapt tests to start and end being allowed to be negative.

More cleanups can be done, specially in the tests if we go further with #10757 (comment) and not calculate in inverse order (there's no point if we don't get the slight optimization).

@jnewbery jnewbery referenced this pull request Jul 13, 2017

Closed

Add histunspent RPC #10804

@jtimon jtimon force-pushed the jtimon:b15-rpc-plotter branch from 90c7264 to 33d2bda Aug 24, 2017

@jtimon

This comment has been minimized.

Member

jtimon commented Aug 24, 2017

Needed rebase.
If somebody made a web for it, it may be interesting to show number of segwit txs too http://segwit.5gbfree.com/countsegwit

@forklol

This comment has been minimized.

forklol commented Aug 29, 2017

Just wanted to say that this would be massively helpful to track statistics. I hope this finds it's way into a release soon.

@jtimon jtimon force-pushed the jtimon:b15-rpc-plotter branch 3 times, most recently from 8bc1d6a to b2d93e4 Aug 31, 2017

@jtimon

This comment has been minimized.

Member

jtimon commented Aug 31, 2017

Reversed the order of the values to the natural one, since as discussed the optimization of doing fetching the blocks in reverse order is not worth the loss in clarity of the code.
Added segwit tx counter stat, and also the total size and weight for those txs (txs that at least have one sw input, txs sending to sw outputs don't count).

@trippysalmon

This comment has been minimized.

trippysalmon commented Sep 1, 2017

Perhaps a better name for <stat>_old is <stat>_virtual, _virt or _v. Or perhaps prepend it with "v" just like the tx size in the output of getrawtransaction (vsize).

For example:

avgfeerate_old becomes vavgfeerate
maxfeerate_old becomes vmaxfeerate
medianfeerate_old becomes vmedianfeerate

etc.

@trippysalmon

This comment has been minimized.

trippysalmon commented Sep 3, 2017

I just finished calling getblockstats on every block in the chain and saving it into a database. I didn't encounter any issues and the performance is quite good (100-1000ms per "full" block on an i7 6900k /w 32gb ram + nvme ssd).

Btw, if anyone is interested in the dataset I can share it. Just convo me at freenode irc (nick: "trippysalmon"). It includes some other stats as well, like rolling average hashrates.

@jtimon jtimon force-pushed the jtimon:b15-rpc-plotter branch from 0d326d6 to de63105 May 1, 2018

@jtimon

This comment has been minimized.

Member

jtimon commented May 1, 2018

@ajtowns now it seems to work 24 hours after generating the data

@TheBlueMatt yeah, it looks simpler now without using CFeeRate or CFeeRate::GetTruncatedFee. Thanks

Independently of that, if we want more precision for feerates (say, move from sat/vbyte to sat/vKB or whatever), now it's the right time to decide so.
I guess we could also support arbitrary precision with an optional feerates_precision parameter that defaults to 1 (ie sat/vbyte), or 1000 (ie sat/vKB) or 1024 or whatever.
Or we can simply leave the latter for later if anybody asks, but it's a simple change if people want more precision.

}
std::set<std::string> stats;
if (request.params.size() > 1 && !request.params[1].isNull()) {

This comment has been minimized.

@TheBlueMatt

TheBlueMatt May 1, 2018

Contributor

You dont need the size() check.

} else {
const std::string strHash = request.params[0].get_str();
const uint256 hash(uint256S(strHash));
pindex = LookupBlockIndex(hash);

This comment has been minimized.

@TheBlueMatt

TheBlueMatt May 1, 2018

Contributor

I believe if the hash is invalid GetBlockChecked will crash trying to deref the blockindex nullptr.

This comment has been minimized.

@promag

promag May 15, 2018

Member

This is fixed.

@jtimon jtimon force-pushed the jtimon:b15-rpc-plotter branch from de63105 to 3bc5e78 May 7, 2018

Refactor: RPC: Separate GetBlockChecked() from getblock()
This does not change functionality

@jtimon jtimon force-pushed the jtimon:b15-rpc-plotter branch from 3bc5e78 to 4523b5d May 7, 2018

@jtimon

This comment has been minimized.

Member

jtimon commented May 7, 2018

Fixed last 2 nits. Added a test for when blocks aren't found.
Also needed rebase.

assert_raises_rpc_error(-5, 'Block not found', self.nodes[0].getblockstats,
hash_or_height='000000000019d6689c085ae165831e934ff763ae46a2a6c172b3f1b60a8ce26f')

This comment has been minimized.

@promag

promag May 7, 2018

Member

Linter error:

W293 blank line contains whitespace.

This comment has been minimized.

@jtimon

jtimon May 8, 2018

Member

Fixed

@jnewbery jnewbery referenced this pull request May 8, 2018

Closed

Some ACKs missing from PRs #12

@jtimon jtimon force-pushed the jtimon:b15-rpc-plotter branch 2 times, most recently from cedac47 to 563eee9 May 8, 2018

@promag

This comment has been minimized.

Member

promag commented May 15, 2018

Lightly tested ACK 563eee9.

@kallewoof

utACK 563eee9 w/ minor nits

" ,...\n"
" ]\n"
"\nResult: (all values are in reverse order height-wise)\n"
"{ (json object)\n"

This comment has been minimized.

@kallewoof

kallewoof May 17, 2018

Member

Due to two escapes in below lines, this needs to remove two spaces for alignment.

const uint256 hash(uint256S(strHash));
pindex = LookupBlockIndex(hash);
if (!pindex) {
throw JSONRPCError(RPC_INVALID_ADDRESS_OR_KEY, "Block not found");

This comment has been minimized.

@kallewoof

kallewoof May 17, 2018

Member

Just a note here. In

if (fHavePruned && !(pblockindex->nStatus & BLOCK_HAVE_DATA) && pblockindex->nTx > 0)
throw JSONRPCError(RPC_MISC_ERROR, "Block not available (pruned data)");
if (!ReadBlockFromDisk(block, pblockindex, Params().GetConsensus()))
// Block not found on disk. This could be because we have the block
// header in our index but don't have the block (for example if a
// non-whitelisted node sends us an unrequested long chain of valid
// blocks, we add the headers to our index, but don't accept the
// block).
throw JSONRPCError(RPC_MISC_ERROR, "Block not found on disk");
RPC_MISC_ERROR is used, while RPC_INVALID_ADDRESS_OR_KEY is used here. Not sure which is better, but I think they should both use the same error type. (This PR touches the other one so could easily tweak, I think.)

This comment has been minimized.

@jimpo

jimpo May 17, 2018

Contributor

I disagree. If the header is not found, that means the request was invalid, hence RPC_INVALID ADDRESS_OR KEY (key loosely taken to mean request argument, I take it). Whereas if the header is known but the block is not known, I do consider that to be a different error.

In #13144, I introduce RPC_DATA_UNAVAILABLE, which may be more appropriate than RPC_MISC in the latter case.

totalfee += txfee;
// New feerate uses satoshis per virtual byte instead of per serialized byte
CAmount feerate = weight ? (txfee * WITNESS_SCALE_FACTOR) / weight : 0;

This comment has been minimized.

@kallewoof

kallewoof May 17, 2018

Member

May be ok, but I think (txfee * WITNESS_SCALE_FACTOR) / (weight + WITNESS_SCALE_FACTOR - 1) is precisely accurate while the above will have truncation errors sometimes.

This comment has been minimized.

@jimpo

jimpo May 17, 2018

Contributor

That equation has truncated errors. Consider a tx with weight 3 and fee 100. The vsize is 1, so the fee rate should be 1, but your equation would give (100 * 4) / 6 = 66.

You point out correctly though that vsize is computed incorrectly here by rounding down rather than up. This should use GetVirtualTransactionSize().

This comment has been minimized.

@jtimon

jtimon May 17, 2018

Member

Sorry, I don't follow. This is using GetTransactionWeight() which does ::GetSerializeSize(tx, SER_NETWORK, PROTOCOL_VERSION | SERIALIZE_TRANSACTION_NO_WITNESS) * (WITNESS_SCALE_FACTOR - 1) + ::GetSerializeSize(tx, SER_NETWORK, PROTOCOL_VERSION)

GetVirtualTransactionSize does (std::max(nWeight, nSigOpCost * nBytesPerSigOp) + WITNESS_SCALE_FACTOR - 1) / WITNESS_SCALE_FACTOR

But I'm not sure why would we prefer that here.

This comment has been minimized.

@kallewoof

kallewoof May 18, 2018

Member

Sorry, I think I got things mixed up.

This comment has been minimized.

@ajtowns

ajtowns May 18, 2018

Contributor

The fee rate for a tx with weight 3 and fee 100 is the same as if there were 100 such tx's. That would have weight 300 or vsize 75, and fee 10,000, for a fee rate of 10,000/75 or 133.3, not 1, 100, or 66. :) Formula looks fine to me, though I guess estimatefee uses fee/vkbyte rather than fee/byte, which would be (txfee * WITNESS_SCALE_FACTOR * 1000) / weight.

This comment has been minimized.

@jtimon

jtimon May 22, 2018

Member

Well, we can move from sat/vbyte to sat/vkbyte, sure. I t should be a simple change if people prefer that.

@ajtowns

ACK 563eee9 modulo nits

"\nIt won't work for some heights with pruning.\n"
"\nIt won't work without -txindex for utxo_size_inc, *fee or *feerate stats.\n"
"\nArguments:\n"
"1. \"hash_or_height\" (string or numeric, required) The block hash or height of the target block. If height, negative values count back from the current tip\n"

This comment has been minimized.

@ajtowns

ajtowns May 18, 2018

Contributor

But, negative values don't count back from the current tip, they give an error?

static UniValue getblockstats(const JSONRPCRequest& request)
{
if (request.fHelp || request.params.size() < 1 || request.params.size() > 4)

This comment has been minimized.

@ajtowns

ajtowns May 18, 2018

Contributor

Multiline if statement should have braces.

" \"time\", (string, optional) Selected statistic\n"
" ,...\n"
" ]\n"
"\nResult: (all values are in reverse order height-wise)\n"

This comment has been minimized.

@ajtowns

ajtowns May 18, 2018

Contributor

"(all values are in reverse order height-wise)" is outdated now that this only does one block's stats, I think? If it's not, it doesn't make sense to me...

jtimon and others added some commits Jun 3, 2017

Tests: Test new getblockstats RPC
Includes commit from Anthony Towns @ajtowns:

Tests: Save and load block and corresponding expected statistics

@jtimon jtimon force-pushed the jtimon:b15-rpc-plotter branch from 563eee9 to 41d0476 May 22, 2018

@jtimon

This comment has been minimized.

Member

jtimon commented May 22, 2018

Fixed help nits (also made other improvements, some other things in the help were ugly).

@laanwj laanwj merged commit 41d0476 into bitcoin:master May 23, 2018

1 check passed

continuous-integration/travis-ci/pr The Travis CI build passed
Details

laanwj added a commit that referenced this pull request May 23, 2018

Merge #10757: RPC: Introduce getblockstats to plot things
41d0476 Tests: Add data file (Anthony Towns)
4cbfb6a Tests: Test new getblockstats RPC (Jorge Timón)
35e77a0 RPC: Introduce getblockstats (Jorge Timón)
cda8e36 Refactor: RPC: Separate GetBlockChecked() from getblock() (Jorge Timón)

Pull request description:

  It returns per block statistics about several things. It should be easy to add more if people think of other things to add or remove some if I went too far (but once written, why not keep it? EDIT: answer: not to test or maintain them).

  The currently available options are: minfee,maxfee,totalfee,minfeerate,maxfeerate,avgfee,avgfeerate,txs,ins,outs (EDIT: see updated list in the rpc call documentation)

  For the x axis, one can use height or block.nTime (I guess I could add mediantime if there's interest [EDIT: nobody showed interest but I implemented mediantime nonetheless, in fact there's no distinction between x or y axis anymore, that's for the caller to judge]).

  To calculate fees, -txindex is required.

Tree-SHA512: 2b2787a3c7dc4a11df1fce62c8a4c748f5347d7f7104205d5f0962ffec1e0370c825b49fd4d58ce8ce86bf39d8453f698bcd46206eea505f077541ca7d59b18c

@fanquake fanquake removed this from Blockers in High-priority for review May 24, 2018

laanwj added a commit that referenced this pull request Jun 8, 2018

Merge #13259: refactoring: add a method for determining if a block is…
… pruned or not

e9a1881 refactor: add a function for determining if a block is pruned or not (Karl-Johan Alm)

Pull request description:

  The check for whether a block is pruned or not is sufficiently obscure that it deserves a macro. It is also used in 2 places, ~~with more coming, e.g. #10757~~ (turns out it was a move, not an addition).

Tree-SHA512: b9aeb60663e1d1196df5371d5aa00b32ff5d4cdea6a77e4b566f28115cce09570c18e45e4b81a4033f67c4135c8e32c027f67bae3b75c2ea4564285578a3f4dd
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment