[Qt] Add interactive mempool graph #8550

Open
wants to merge 4 commits into
from

Projects

None yet
@jonasschnelli
Member

Includes #8501

At the moment, the mempool graph is not very prominently placed (next to the debug window).
bildschirmfoto 2016-08-19 um 21 20 48

Features:

  • interactive graph with options for tx count, dynamic memory usage and minRelayFee/KB
  • collects stats in the background, changing the timespan will directly redraw (unlike our bandwidth graph)
  • Dynamic size drawing, window can be resized
  • The mempool graph is a QWidget which means, it could be placed together with other graphs in a combine multi-graph view (screensaver approach)

bildschirmfoto 2016-08-19 um 21 28 09

@jonasschnelli jonasschnelli added the GUI label Aug 19, 2016
@MarcoFalke
Member

Concept ACK. Will test later.

@jonasschnelli
Member

Added the missing button PNGs now to git. This should fix the compile issue.

@luke-jr
Member
luke-jr commented Aug 19, 2016

Nice. Some people might want to see actual byte-size, weights, etc; probably will overflow the one-line-per-item paradigm quickly - maybe have a dropdown box to add stuff?

@jonasschnelli
Member

Yes. Dropdown box could make sense. Adding a second graph below or on the right that just plots different data would probably also look good.

@MarcoFalke
Member

Nit: Fonts appear larger on my system. Otherwise this looks great!

screenshot from 2016-08-19 23-15-51

@MarcoFalke MarcoFalke commented on the diff Aug 19, 2016
src/qt/mempoolstats.cpp
+ QDateTime fromDateTime = toDateTime.addSecs(-timeFilter); //-1h
+ if (timeFilter == 0)
+ {
+ // disable filter if timeFilter == 0
+ toDateTime.setTime_t(0);
+ fromDateTime.setTime_t(0);
+ }
+
+ mempoolSamples_t vSamples = clientModel->getMempoolStatsInRange(fromDateTime, toDateTime);
+
+ // set the values into the overview labels
+ if (vSamples.size())
+ {
+ dynMemUsageValueItem->setPlainText(GUIUtil::formatBytes(vSamples.back().dynMemUsage));
+ txCountValueItem->setPlainText(QString::number(vSamples.back().txCount));
+ minFeeValueItem->setPlainText(QString::number(vSamples.back().minFeePerK));
@MarcoFalke
MarcoFalke Aug 19, 2016 Member

Nit: Missing value. I think this is in satoshis. What about using BTC?

@MarcoFalke MarcoFalke and 2 others commented on an outdated diff Aug 20, 2016
src/qt/bitcoingui.cpp
@@ -351,6 +354,11 @@ void BitcoinGUI::createActions()
// initially disable the debug window menu item
openRPCConsoleAction->setEnabled(false);
+ showMempoolStatsAction = new QAction(platformStyle->TextColorIcon(":/icons/mempoolstats"), tr("&Mempool Statistics"), this);
@MarcoFalke
MarcoFalke Aug 20, 2016 Member

I can't find this icon

@jonasschnelli
jonasschnelli Aug 20, 2016 Member

Indeed,... this does not exists yet. Will create one.

@MarcoFalke
MarcoFalke Aug 20, 2016 Member

http://typicons.com/ has some chart icons ;)

@luke-jr
luke-jr Aug 20, 2016 Member

Please remember to keep in mind licensing. typicons are CC-BY-3.0, which is a non-free license.

@MarcoFalke
MarcoFalke Aug 20, 2016 Member

Huh, I recall he gave you permission to dual-license as MIT?

dae0a89

@jonasschnelli
jonasschnelli Aug 20, 2016 Member

@luke-jr: keep in mind that we have permission from the author to distribute them under MIT: #6367 (comment)

@luke-jr
luke-jr Aug 20, 2016 Member

Ah, forgot about that. Thanks for the reminder.

@jonasschnelli
Member

Switched to "Arial" as font and added a auto-size-adjustment.
Also added the "charts" icon for the menu.

@MarcoFalke
Member

Nit: run optimize-pngs and mention the icons in assets_attribution?

@jonasschnelli
Member

Optimized the pngs and added them to the assets attribution file.

@mcccs
mcccs commented Aug 20, 2016

Concept ACK.

@isle2983
Contributor

Nit1: The graph window is always drawn on top of the main window, even when the main window is active. It makes it annoying to use the rest of the GUI while this is open.

Nit2: The quantity labels for the Dynamic Memory Usage values on the left hand side of the graph are cramped and cut off some of the characters. See "500.00 KB" in the screenshot. It is like that for larger values once it gets into the 100MBs too.

mempoolstats

@rebroad
Contributor
rebroad commented Aug 24, 2016 edited

I'm not sure what I'm doing wrong but I've merged this and the graph is empty, and has been for an hour now... :-s i.e. Dynamic Memory Usage is stuck on 0 bytes, Amount of Txs also 0, MinRelayFee also zero.

@jonasschnelli
Member

@rebroad: what does getmempoolinfo tells you? Are you in sync on mainnet (or testnet)?

@rebroad
Contributor
rebroad commented Aug 25, 2016 edited

@jonasschnelli problem solved - I had blocksonly=1 in the config file!! It's working really nicely now - great job!! I love how it can be resized and the period of time changed without the graph resetting (unlike the network traffic graph!).

@fanquake
Contributor

OS X screenshots:
init
more-than-10

@luke-jr
Member
luke-jr commented Aug 27, 2016

Using images for basically-a-checkbox gives a non-native look on every system. It looks like we can just colour a QCheckBox?

@paveljanik
Contributor

Rebase needed.

Concept ACK

Agree about checkboxes...

@jonasschnelli
Member

Rebased on top of #8501

@MarcoFalke MarcoFalke and 2 others commented on an outdated diff Sep 9, 2016
src/Makefile.qt.include
@@ -234,6 +237,11 @@ RES_ICONS = \
qt/res/icons/bitcoin.ico \
qt/res/icons/bitcoin_testnet.ico \
qt/res/icons/bitcoin.png \
+ qt/res/icons/button_off.png \
+ qt/res/icons/button_on_blue.png \
@MarcoFalke
MarcoFalke Sep 9, 2016 Member

@jonasschnelli What do you think about using QCheckBox with setPallette() as suggested by @luke-jr ?

@jonasschnelli
jonasschnelli Sep 15, 2016 Member

I think Checkboxes are appropriate for now.

But we should consider using non-standard UI elements to improve the look and feel. I know, developers hate this. :-)

But have a look at #5896. I think people liked it because the look and feel was different then just "Arial 12 and standard UI elements".

@luke-jr
luke-jr Nov 24, 2016 Member

Non-standard UI elements ruin the look and feel...

@jonasschnelli
jonasschnelli Nov 24, 2016 Member

Not sure. That's probably a matter of taste.
The thing why I'd like to keep the non-standard UI elements is, that I'm using a QGraphicScene at this point. Adding a QWidget (checkbox) at this point breaks the flexibility of reusing the QGraphicScene and draw it in any scale.

I think the checkbox could be different then the one thats currently used. And it probably should be draw internally and not use a PNG.

@luke-jr
luke-jr Nov 24, 2016 Member

Because it's a matter of taste is precisely why it should use standard UI elements. The standard element should do all the drawing. If the user doesn't like their standard widgets, they can change them.

src/main.cpp
@@ -2837,6 +2844,9 @@ bool static ConnectTip(CValidationState& state, const CChainParams& chainparams,
for(unsigned int i=0; i < pblock->vtx.size(); i++)
txChanged.push_back(std::make_tuple(pblock->vtx[i], pindexNew, i));
+ // update mempool stats
+ CStats::DefaultStats()->addMempoolSample(mempool.size(), mempool.DynamicMemoryUsage(), mempool.GetMinFee(GetArg("-maxmempool", DEFAULT_MAX_MEMPOOL_SIZE) * 1000000).GetFeePerK());
@paveljanik
paveljanik Sep 15, 2016 Contributor

Can you please do something with this long line? It is there ~3 times and does a simple thing - worth a better method to add a sample?

@jonasschnelli
jonasschnelli Sep 15, 2016 Member

Hmm.. I think this line is reasonable? How would you improve it?

@paveljanik
paveljanik Sep 15, 2016 Contributor

Pass const ref. mempool only and do the rest in the method?

@jonasschnelli
jonasschnelli Sep 15, 2016 Member

I like the current layering concept. IMO its preferable that the stats.cpp module has no knowledge over the txmempool.h's classes, etc., I prefer to keep it as "dump" as possible.

+ QGraphicsTextItem *item = scene->addText(drawTime.toString("HH:mm"), gridFont);
+ item->setPos(GRAPH_PADDING_LEFT+lX-(item->boundingRect().width()/2), bottom);
+ redrawItems.append(item);
+ qint64 step = secsTotal/amountOfLinesV;
@paveljanik
paveljanik Sep 15, 2016 Contributor

-Wshadow warning:

qt/mempoolstats.cpp:351:16: warning: declaration shadows a local variable [-Wshadow]
        qint64 step = secsTotal/amountOfLinesV;
               ^
qt/mempoolstats.cpp:240:11: note: previous declaration is here
    qreal step = maxwidth/(double)vSamples.size();
          ^
1 warning generated.
+
+};
+
+void RegisterStatsRPCCommands(CRPCTable& tableRPC)
@paveljanik
paveljanik Sep 15, 2016 Contributor

-Wshadow warning:

stats/rpc_stats.cpp:65:42: warning: declaration shadows a variable in the global namespace [-Wshadow]
void RegisterStatsRPCCommands(CRPCTable& tableRPC)
                                         ^
./rpc/server.h:172:18: note: previous declaration is here
extern CRPCTable tableRPC;
                 ^
1 warning generated.
@paveljanik
Contributor
paveljanik commented Sep 15, 2016 edited

I really like this!

Suggestions:

  1. the menu item should read Mempool statistics (small s) to match other items in the menu.
  2. the window title should be the same (both title bar and top header - I think it is there only to balance the "legenda" on the right...) ;-)
  3. the graph itself in the window is not centered - it is a bit on the left side here.
  4. Esc should close the window.
@paveljanik
Contributor

Rebase needed.

Hmm, github shows "Conflicting files"...

@rebroad
Contributor
rebroad commented Oct 16, 2016

I've been testing this under various window managers. On mwm the title of the mempool window says "bitcoin-qt". Also, it's not its own window like the debug window is, but a child of the main bitcoin-qt window, so it cannot be minimised on its own, and is minimised whenever the main window is minimised.

@jonasschnelli
Member

Thanks @rebroad.
I have plans to completely overhaul this PR during this week.

@rebroad
Contributor
rebroad commented Oct 22, 2016

@jonasschnelli Needed a little rebasing... d7857e1

@laanwj
Member
laanwj commented Oct 26, 2016

Concept ACK, looks nice!

@fanquake
Contributor
fanquake commented Nov 7, 2016

ping @jonasschnelli Did you get to overhauling this PR?

@Victorsueca
Victorsueca commented Nov 13, 2016 edited

ACK 7a7d631
Tested on Windows x64
Nit: Some elements overlay with my system font size and default window size
Maybe we should add this as a Debug window tab?
GUI Screenshot

@fanquake
Contributor

Post overhaul OS X screenshots:
screen shot
screen shot 3
screen shot 5

jonasschnelli added some commits Aug 12, 2016
@jonasschnelli jonasschnelli Add mempool statistics collector 706da3e
@jonasschnelli jonasschnelli [Qt] Add interactive mempool graph 6fcd92c
@jonasschnelli jonasschnelli [RPC] Use JSON array for mempool samples 9bcf505
@jonasschnelli
Member

Removed the custom checkbox icons.
Thanks to QGraphicsProxyWidget, it's still flexible in Size.

@jonasschnelli jonasschnelli [SquashMe] use default QCheckBox instead of custom image 251ee28
@jonasschnelli
Member

This is how it looks now on OSX:
bildschirmfoto 2016-11-25 um 11 38 32

@@ -79,6 +80,7 @@ Comment: Modifications of Stephan Hutchings Typicons
Files: src/qt/res/icons/about.png
src/qt/res/icons/bitcoin.*
+ src/qt/res/icons/button_*
@paveljanik
paveljanik Nov 25, 2016 Contributor

No button_* files now.

@paveljanik
Contributor

@jonasschnelli Please address my comments here: #8550 (comment)

@luke-jr
Member
luke-jr commented Nov 27, 2016

I think the click-the-human-readable-timeframe was better, but I replaced it with a slider to match our bandwidth graph. Probably they should both be the same, whatever we go with (ie, replace the sliders together, in a subsequent PR).

@gmaxwell
Member
gmaxwell commented Dec 4, 2016

More graph things are cool. But I think this data is basically worthless to users, if someone goes and makes and gives you a zillion minrelayfee transactions that will never get confirmed ... the graph will read maximum, but nothing has really changed for the user or the network.

A stacked chart of mempool bytes at different feerates would be much more interesting and useful to users like how sipa's chart displays the utxo set http://bitcoin.sipa.be/utxo_size.png as it would make it possible to tell things that will get confirmed (bottom stacks of the graph) from noise (higher stacks).

@rebroad
Contributor
rebroad commented Dec 5, 2016

@gmaxwell Showing memory utilization, especially in relation to minfeerelay is useful for anyone wanting to refine the algorithm for setting minfeerelay.

I have set maxmempool to 144, i.e. 24 hours worth of 1MB blocks - in order to give me an indication of the minimum fee needed to keep a tx in the mempool for at least 24 hours - although not an indication for how much the fee needs to be to get it confirmed within 24 hours, admittedly.

@jonasschnelli
Member

I like the idea of the stacked chart. However I wanted to start with a simple graph to get in the fundamental stuff (stats core classes, drawing stuff). At the moment, the only insight into the mempool from the GUI you can get is the amount of transactions. IMO step after step.

@laanwj
Member
laanwj commented Dec 8, 2016 edited

However I wanted to start with a simple graph to get in the fundamental stuff (stats core classes, drawing stuff).

I tend to agree - one step at a time, scope creep before the initial merge will likely mean that this will never happen at all.
(needs rebase)

@rebroad
Contributor
rebroad commented Dec 11, 2016

screenshot at 2016-12-11 10 34 09

@laanwj spot on re scope creep. In this sense, I'd say ACK to merging it now.

The screenshot attached is a typical example of what it looks like after running for a week or so. One thing that becomes apparent is that the horizontal graph label points become useless - i.e. once it goes over 24 hours of duration, it would make sense to switch to date instead of time.

@@ -148,6 +148,9 @@ void OptionsModel::Init(bool resetSettings)
if (!SoftSetArg("-lang", settings.value("language").toString().toStdString()))
addOverriddenOption("-lang");
+ // Enable statistics by default
+ SoftSetBoolArg("-statsenable", true);
@luke-jr
luke-jr Dec 21, 2016 Member

Doing it this way doesn't update -help...

@luke-jr luke-jr added a commit to bitcoinknots/bitcoin that referenced this pull request Dec 21, 2016
@jonasschnelli @luke-jr jonasschnelli + luke-jr [Qt] Add interactive mempool graph
Github-Pull: #8550
Rebased-From: 6fcd92c
71bfbf0
@luke-jr luke-jr added a commit to bitcoinknots/bitcoin that referenced this pull request Dec 21, 2016
@jonasschnelli @luke-jr jonasschnelli + luke-jr [SquashMe] use default QCheckBox instead of custom image
Github-Pull: #8550
Rebased-From: 251ee28
67c1eb8
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment