[Qt] Add interactive mempool graph #8550

jonasschnelli · Aug 19, 2016

Includes #8501

At the moment, the mempool graph is not very prominently placed (next to the debug window).

Features:

interactive graph with options for tx count, dynamic memory usage and minRelayFee/KB
collects stats in the background, changing the timespan will directly redraw (unlike our bandwidth graph)
Dynamic size drawing, window can be resized
The mempool graph is a QWidget which means, it could be placed together with other graphs in a combine multi-graph view (screensaver approach)

MarcoFalke · Aug 19, 2016

Concept ACK. Will test later.

jonasschnelli · Aug 19, 2016

Added the missing button PNGs now to git. This should fix the compile issue.

luke-jr · Aug 19, 2016

Nice. Some people might want to see actual byte-size, weights, etc; probably will overflow the one-line-per-item paradigm quickly - maybe have a dropdown box to add stuff?

jonasschnelli · Aug 19, 2016

Yes. Dropdown box could make sense. Adding a second graph below or on the right that just plots different data would probably also look good.

MarcoFalke · Aug 19, 2016

Nit: Fonts appear larger on my system. Otherwise this looks great!

Nit: Missing value. I think this is in satoshis. What about using BTC?

I can't find this icon

Indeed,... this does not exists yet. Will create one.

http://typicons.com/ has some chart icons ;)

Please remember to keep in mind licensing. typicons are CC-BY-3.0, which is a non-free license.

Huh, I recall he gave you permission to dual-license as MIT?

dae0a89

@luke-jr: keep in mind that we have permission from the author to distribute them under MIT: #6367 (comment)

Ah, forgot about that. Thanks for the reminder.

jonasschnelli · Aug 20, 2016

Switched to "Arial" as font and added a auto-size-adjustment.
Also added the "charts" icon for the menu.

MarcoFalke · Aug 20, 2016

Nit: run optimize-pngs and mention the icons in assets_attribution?

jonasschnelli · Aug 20, 2016

Optimized the pngs and added them to the assets attribution file.

Mendeleyef · Aug 20, 2016

Concept ACK.

isle2983 · Aug 21, 2016

Nit1: The graph window is always drawn on top of the main window, even when the main window is active. It makes it annoying to use the rest of the GUI while this is open.

Nit2: The quantity labels for the Dynamic Memory Usage values on the left hand side of the graph are cramped and cut off some of the characters. See "500.00 KB" in the screenshot. It is like that for larger values once it gets into the 100MBs too.

rebroad · Aug 24, 2016

I'm not sure what I'm doing wrong but I've merged this and the graph is empty, and has been for an hour now... :-s i.e. Dynamic Memory Usage is stuck on 0 bytes, Amount of Txs also 0, MinRelayFee also zero.

jonasschnelli · Aug 24, 2016

@rebroad: what does getmempoolinfo tells you? Are you in sync on mainnet (or testnet)?

rebroad · Aug 25, 2016

@jonasschnelli problem solved - I had blocksonly=1 in the config file!! It's working really nicely now - great job!! I love how it can be resized and the period of time changed without the graph resetting (unlike the network traffic graph!).

fanquake · Aug 26, 2016

OS X screenshots:

luke-jr · Aug 27, 2016

Using images for basically-a-checkbox gives a non-native look on every system. It looks like we can just colour a QCheckBox?

paveljanik · Sep 7, 2016

Rebase needed.

Concept ACK

Agree about checkboxes...

jonasschnelli · Sep 9, 2016

Rebased on top of #8501

@jonasschnelli What do you think about using QCheckBox with setPallette() as suggested by @luke-jr ?

I think Checkboxes are appropriate for now.

But we should consider using non-standard UI elements to improve the look and feel. I know, developers hate this. :-)

But have a look at #5896. I think people liked it because the look and feel was different then just "Arial 12 and standard UI elements".

Non-standard UI elements ruin the look and feel...

Not sure. That's probably a matter of taste.
The thing why I'd like to keep the non-standard UI elements is, that I'm using a QGraphicScene at this point. Adding a QWidget (checkbox) at this point breaks the flexibility of reusing the QGraphicScene and draw it in any scale.

I think the checkbox could be different then the one thats currently used. And it probably should be draw internally and not use a PNG.

Because it's a matter of taste is precisely why it should use standard UI elements. The standard element should do all the drawing. If the user doesn't like their standard widgets, they can change them.

paveljanik · Sep 15, 2016

paveljanik reviewed Sep 15, 2016

View changes

Can you please do something with this long line? It is there ~3 times and does a simple thing - worth a better method to add a sample?

Hmm.. I think this line is reasonable? How would you improve it?

Pass const ref. mempool only and do the rest in the method?

I like the current layering concept. IMO its preferable that the stats.cpp module has no knowledge over the txmempool.h's classes, etc., I prefer to keep it as "dump" as possible.

paveljanik · Sep 15, 2016

paveljanik reviewed Sep 15, 2016

View changes

-Wshadow warning:

qt/mempoolstats.cpp:351:16: warning: declaration shadows a local variable [-Wshadow]
        qint64 step = secsTotal/amountOfLinesV;
               ^
qt/mempoolstats.cpp:240:11: note: previous declaration is here
    qreal step = maxwidth/(double)vSamples.size();
          ^
1 warning generated.

paveljanik · Sep 15, 2016

paveljanik reviewed Sep 15, 2016

View changes

-Wshadow warning:

stats/rpc_stats.cpp:65:42: warning: declaration shadows a variable in the global namespace [-Wshadow]
void RegisterStatsRPCCommands(CRPCTable& tableRPC)
                                         ^
./rpc/server.h:172:18: note: previous declaration is here
extern CRPCTable tableRPC;
                 ^
1 warning generated.

paveljanik · Sep 15, 2016

I really like this!

Suggestions:

the menu item should read Mempool statistics (small s) to match other items in the menu.
the window title should be the same (both title bar and top header - I think it is there only to balance the "legenda" on the right...) ;-)
the graph itself in the window is not centered - it is a bit on the left side here.
Esc should close the window.

paveljanik · Sep 25, 2016

Rebase needed.

Hmm, github shows "Conflicting files"...

rebroad · Oct 16, 2016

I've been testing this under various window managers. On mwm the title of the mempool window says "bitcoin-qt". Also, it's not its own window like the debug window is, but a child of the main bitcoin-qt window, so it cannot be minimised on its own, and is minimised whenever the main window is minimised.

jonasschnelli · Oct 17, 2016

Thanks @rebroad.
I have plans to completely overhaul this PR during this week.

rebroad · Oct 22, 2016

@jonasschnelli Needed a little rebasing... d7857e1

laanwj · Oct 26, 2016

Concept ACK, looks nice!

fanquake · Nov 7, 2016

ping @jonasschnelli Did you get to overhauling this PR?

Victorsueca · Nov 13, 2016

ACK 7a7d631
Tested on Windows x64
Nit: Some elements overlay with my system font size and default window size
Maybe we should add this as a Debug window tab?

fanquake · Nov 24, 2016

Post overhaul OS X screenshots:

jonasschnelli · Nov 25, 2016

Removed the custom checkbox icons.
Thanks to QGraphicsProxyWidget, it's still flexible in Size.

jonasschnelli · Nov 25, 2016

This is how it looks now on OSX:

paveljanik · Nov 25, 2016

paveljanik reviewed Nov 25, 2016

View changes

No button_* files now.

paveljanik · Nov 25, 2016

@jonasschnelli Please address my comments here: #8550 (comment)

luke-jr · Nov 26, 2016

@jonasschnelli luke-jr/bitcoin@ee121aa

luke-jr · Nov 27, 2016

I think the click-the-human-readable-timeframe was better, but I replaced it with a slider to match our bandwidth graph. Probably they should both be the same, whatever we go with (ie, replace the sliders together, in a subsequent PR).

gmaxwell · Dec 4, 2016

More graph things are cool. But I think this data is basically worthless to users, if someone goes and makes and gives you a zillion minrelayfee transactions that will never get confirmed ... the graph will read maximum, but nothing has really changed for the user or the network.

A stacked chart of mempool bytes at different feerates would be much more interesting and useful to users like how sipa's chart displays the utxo set http://bitcoin.sipa.be/utxo_size.png as it would make it possible to tell things that will get confirmed (bottom stacks of the graph) from noise (higher stacks).

rebroad · Dec 5, 2016

@gmaxwell Showing memory utilization, especially in relation to minfeerelay is useful for anyone wanting to refine the algorithm for setting minfeerelay.

I have set maxmempool to 144, i.e. 24 hours worth of 1MB blocks - in order to give me an indication of the minimum fee needed to keep a tx in the mempool for at least 24 hours - although not an indication for how much the fee needs to be to get it confirmed within 24 hours, admittedly.

jonasschnelli · Dec 5, 2016

I like the idea of the stacked chart. However I wanted to start with a simple graph to get in the fundamental stuff (stats core classes, drawing stuff). At the moment, the only insight into the mempool from the GUI you can get is the amount of transactions. IMO step after step.

laanwj · Dec 8, 2016

However I wanted to start with a simple graph to get in the fundamental stuff (stats core classes, drawing stuff).

I tend to agree - one step at a time, scope creep before the initial merge will likely mean that this will never happen at all.
(needs rebase)

rebroad · Dec 11, 2016

@laanwj spot on re scope creep. In this sense, I'd say ACK to merging it now.

The screenshot attached is a typical example of what it looks like after running for a week or so. One thing that becomes apparent is that the horizontal graph label points become useless - i.e. once it goes over 24 hours of duration, it would make sense to switch to date instead of time.

luke-jr · Dec 21, 2016

luke-jr reviewed Dec 21, 2016

View changes

Doing it this way doesn't update -help...

paveljanik · Feb 22, 2017

Rebase needed.

I really really would like to see this included in the GUI in the next version!

luke-jr · Feb 23, 2017

FWIW, rebased (at least part of the way) in my stats_rpc-0.14 and stats_qt-0.14 branches.

jonasschnelli · Feb 23, 2017

This is based on #8501 (core change with additional RPC command) which has to get right first.
Another solution would be to not depend on a core statistics "module" and therefore collect the stats in a GUI container only. Maybe this would be the better first step.

jtimon · Feb 23, 2017

Concept ACK needs rebase

DATSEC · Mar 8, 2017

The graphics are good, the design does not forget
Here you can see the different designs of graphs
http://www.bittbox.com/showcase/30-flat-analytics-ui-dashboards

I like this one

https://dribbble.com/shots/1719845-Statistics-General-trends

laanwj · Jun 6, 2017

Seems the discussion here bled out.

I'd say please review the code, and then get it merged. Visual details etc can be changed later. Real-time statistics is one reason why people like to run their own nodes.

laanwj · Jun 28, 2017

Needs rebase.

Sjors · Nov 9, 2017

Concept ACK.

I can't compile this (log gist). I'll try again after the rebase.

A stacked chart of mempool bytes at different feerates would be much more interesting and useful

Agreed.

It would be nice to have a little chart icon in the bottom right corner of the wallet (in a future PR).

That said, I also agree with:

Visual details etc can be changed later.

jonasschnelli · Nov 9, 2017

This is based on #8501 and the question is, should it be Qt only (no Qt free stats collector) or should we try to get #8501 done?
(will rebase both PRs soon)

jonasschnelli · Nov 22, 2017

Rebased.
It's currently not Qt4 compatible due to a lambda signal connect (travis reported / working on it).

laanwj · Nov 23, 2017

It's currently not Qt4 compatible due to a lambda signal connect (travis reported / working on it).

If it's a lot of work, feel free to leave "nice but not critical" features such as this out when compiling with Qt4.

jonasschnelli added the GUI label Aug 19, 2016

morcos referenced this pull request Dec 5, 2016
Open
Add mempool statistics collector #8501

luke-jr added a commit to bitcoinknots/bitcoin that referenced this pull request Dec 21, 2016

[Qt] Add interactive mempool graph
Github-Pull: #8550 Rebased-From: 6fcd92c

71bfbf0

luke-jr added a commit to bitcoinknots/bitcoin that referenced this pull request Dec 21, 2016

[SquashMe] use default QCheckBox instead of custom image
Github-Pull: #8550 Rebased-From: 251ee28

67c1eb8

MarcoFalke referenced this pull request Mar 8, 2017
Closed
Great plan and design a bitcoin wallet #9948

jonasschnelli referenced this pull request May 16, 2017
Open
Network Traffic/Reset delayed action #10406

+    +    QDateTime fromDateTime = toDateTime.addSecs(-timeFilter); //-1h
+    +    if (timeFilter == 0)
+    +    {
+    +        // disable filter if timeFilter == 0
+    +        toDateTime.setTime_t(0);
+    +        fromDateTime.setTime_t(0);
+    +    }
+    +
+    +    mempoolSamples_t vSamples = clientModel->getMempoolStatsInRange(fromDateTime, toDateTime);
+    +
+    +    // set the values into the overview labels
+    +    if (vSamples.size())
+    +    {
+    +        dynMemUsageValueItem->setPlainText(GUIUtil::formatBytes(vSamples.back().dynMemUsage));
+    +        txCountValueItem->setPlainText(QString::number(vSamples.back().txCount));
+    +        minFeeValueItem->setPlainText(QString::number(vSamples.back().minFeePerK));

@@ -351,6 +354,11 @@ void BitcoinGUI::createActions()
          // initially disable the debug window menu item
          openRPCConsoleAction->setEnabled(false);
+    +    showMempoolStatsAction = new QAction(platformStyle->TextColorIcon(":/icons/mempoolstats"), tr("&Mempool Statistics"), this);

@@ -234,6 +237,11 @@ RES_ICONS = \
        qt/res/icons/bitcoin.ico \
        qt/res/icons/bitcoin_testnet.ico \
        qt/res/icons/bitcoin.png \
+    +  qt/res/icons/button_off.png \
+    +  qt/res/icons/button_on_blue.png \

@@ -2837,6 +2844,9 @@ bool static ConnectTip(CValidationState& state, const CChainParams& chainparams,
              for(unsigned int i=0; i < pblock->vtx.size(); i++)
                  txChanged.push_back(std::make_tuple(pblock->vtx[i], pindexNew, i));
+        +    // update mempool stats
+        +    CStats::DefaultStats()->addMempoolSample(mempool.size(), mempool.DynamicMemoryUsage(), mempool.GetMinFee(GetArg("-maxmempool", DEFAULT_MAX_MEMPOOL_SIZE) * 1000000).GetFeePerK());

+        +        QGraphicsTextItem *item = scene->addText(drawTime.toString("HH:mm"), gridFont);
+        +        item->setPos(GRAPH_PADDING_LEFT+lX-(item->boundingRect().width()/2), bottom);
+        +        redrawItems.append(item);
+        +        qint64 step = secsTotal/amountOfLinesV;

bitcoin/bitcoin

Join GitHub today

[Qt] Add interactive mempool graph #8550

Conversation

jonasschnelli commented Aug 19, 2016

jonasschnelli added the GUI label Aug 19, 2016

MarcoFalke commented Aug 19, 2016

jonasschnelli commented Aug 19, 2016

luke-jr commented Aug 19, 2016

jonasschnelli commented Aug 19, 2016

MarcoFalke commented Aug 19, 2016

Show 1 comment Hide comment MarcoFalke commented on an outdated diff Aug 19, 2016

Show 7 comments Hide comments MarcoFalke and 2 others commented on an outdated diff Aug 20, 2016

jonasschnelli commented Aug 20, 2016

MarcoFalke commented Aug 20, 2016

jonasschnelli commented Aug 20, 2016

Mendeleyef commented Aug 20, 2016

isle2983 commented Aug 21, 2016

rebroad commented Aug 24, 2016 • edited Edited 1 time rebroad Aug 24, 2016

jonasschnelli commented Aug 24, 2016

rebroad commented Aug 25, 2016 • edited Edited 1 time rebroad Aug 25, 2016

fanquake commented Aug 26, 2016

luke-jr commented Aug 27, 2016

paveljanik commented Sep 7, 2016

jonasschnelli commented Sep 9, 2016

Show 5 comments Hide comments MarcoFalke and 2 others commented on an outdated diff Sep 9, 2016

paveljanik reviewed Sep 15, 2016 View changes

paveljanik reviewed Sep 15, 2016 View changes

paveljanik reviewed Sep 15, 2016 View changes

paveljanik commented Sep 15, 2016 • edited Edited 1 time paveljanik Sep 15, 2016

paveljanik commented Sep 25, 2016

rebroad commented Oct 16, 2016

jonasschnelli commented Oct 17, 2016

rebroad commented Oct 22, 2016

laanwj commented Oct 26, 2016

fanquake commented Nov 7, 2016

Victorsueca commented Nov 13, 2016 • edited Edited 1 time Victorsueca Nov 13, 2016

fanquake commented Nov 24, 2016

jonasschnelli commented Nov 25, 2016

jonasschnelli commented Nov 25, 2016

paveljanik reviewed Nov 25, 2016 View changes

paveljanik commented Nov 25, 2016

luke-jr commented Nov 26, 2016

luke-jr commented Nov 27, 2016

gmaxwell commented Dec 4, 2016

morcos referenced this pull request Dec 5, 2016

rebroad commented Dec 5, 2016

jonasschnelli commented Dec 5, 2016

laanwj commented Dec 8, 2016 • edited Edited 1 time laanwj Dec 8, 2016

rebroad commented Dec 11, 2016

luke-jr reviewed Dec 21, 2016 View changes

luke-jr added a commit to bitcoinknots/bitcoin that referenced this pull request Dec 21, 2016

luke-jr added a commit to bitcoinknots/bitcoin that referenced this pull request Dec 21, 2016

paveljanik commented Feb 22, 2017

luke-jr commented Feb 23, 2017

jonasschnelli commented Feb 23, 2017

jtimon commented Feb 23, 2017

MarcoFalke referenced this pull request Mar 8, 2017

DATSEC commented Mar 8, 2017

jonasschnelli referenced this pull request May 16, 2017

laanwj commented Jun 6, 2017 • edited Edited 1 time laanwj Jun 6, 2017

laanwj commented Jun 28, 2017

Sjors commented Nov 9, 2017

jonasschnelli commented Nov 9, 2017

jonasschnelli added some commits Aug 12, 2016

jonasschnelli commented Nov 22, 2017

laanwj commented Nov 23, 2017

MarcoFalke commented on an outdated diff Aug 19, 2016

MarcoFalke and 2 others commented on an outdated diff Aug 20, 2016

rebroad commented Aug 24, 2016 •

Edited 1 time

rebroad
Aug 24, 2016

rebroad commented Aug 25, 2016 •

Edited 1 time

rebroad
Aug 25, 2016

MarcoFalke and 2 others commented on an outdated diff Sep 9, 2016

paveljanik reviewed Sep 15, 2016

View changes

paveljanik reviewed Sep 15, 2016

View changes

paveljanik reviewed Sep 15, 2016

View changes

paveljanik commented Sep 15, 2016 •

Edited 1 time

paveljanik
Sep 15, 2016

Victorsueca commented Nov 13, 2016 •

Edited 1 time

Victorsueca
Nov 13, 2016

paveljanik reviewed Nov 25, 2016

View changes

laanwj commented Dec 8, 2016 •

Edited 1 time

laanwj
Dec 8, 2016

luke-jr reviewed Dec 21, 2016

View changes

laanwj commented Jun 6, 2017 •

Edited 1 time

laanwj
Jun 6, 2017