Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature] Metric Dashboard #205

Merged
20 commits merged into from
Feb 18, 2022
Merged

[Feature] Metric Dashboard #205

20 commits merged into from
Feb 18, 2022

Conversation

apoorv-2204
Copy link
Contributor

-implementation of requested changes

lib/archethic/metrics/poller.ex Outdated Show resolved Hide resolved
lib/archethic/metrics/poller.ex Outdated Show resolved Hide resolved
lib/archethic/metrics/poller.ex Outdated Show resolved Hide resolved
assets/js/metric_config.js Outdated Show resolved Hide resolved
@apoorv-2204
Copy link
Contributor Author

apoorv-2204 commented Jan 27, 2022

METRIC LIST :

Sno Metrics Community -understands? Taken/worked/code
1 Node location on map Yes NO
2 Transaction-pe-second Yes Yes
Db_ duration   Yes
4 "archethic_mining_full_transaction_validation_duration" yes Yes
5 POW y y
6 "archethic_contract_parsing_duration" Y Y
7 Orcale_usd_price Y Y
8 Oracle_eur_price Y Y
9 archethic_p2p_send_message_duration N Y
10 total_number_of_transactions Y Y
11 Tokenomics Y Y
12 "archethic_election_validation_nodes_duration" N 1
13 "archethic_election_storage_nodes_duration" N 1
14 "archethic_mining_pending_transaction_validation_duration" N 1
15 "archethic_mining_fetch_context_duration" N 1
16 "archethic_self_repair_duration" N 1
17 "vm_total_run_queue_lengths_io" N 1
18 "vm_total_run_queue_lengths_cpu" N 1
19 "vm_total_run_queue_lengths_total" N 1
20 "vm_system_counts_process_count" N 1
21 "vm_system_counts_port_count" N 1
22 "vm_system_counts_atom_count" N 1
23 "vm_memory_total" N 1
24 "vm_memory_system" N 1
25 "vm_memory_processes_used" N 1
26 "vm_memory_processes" N 1
27 "vm_memory_ets" N 1
28 "vm_memory_code" N 1
29 "vm_memory_binary" N 1
30 "vm_memory_atom_used" 0 1
31 "vm_memory_atom" 0 1
32 "db_get_transaction_duration" 0 0
33 "db_get_transaction_chain_duration" 0 0
34 "db_write_transaction_duration" 0 0
35 "db_write_transaction_chain_duration" 0 0
36 "p2p_send_message_GetLastTransactionAddress_duration" 0 0
37 "p2p_send_message_ReplicateTransaction_duration" 0 0
38 "p2p_send_message_GetTransactionInputs_duration" 0 0
39 "p2p_send_message_NotifyEndOfNodeSync_duration" 0 0
40 "p2p_send_message_GetTransactionChain_duration" 0 0
41 "p2p_send_message_GetStorageNonce_duration" 0 0
42 "p2p_send_message_GetTransactionSummary_duration" 0 0
43 "p2p_send_message_CrossValidationDone_duration" 0 0
44 "p2p_send_message_GetBeaconSummaries_duration" 0 0
45 "p2p_send_message_NewTransaction_duration" 0 0
46 "p2p_send_message_GetP2PView_duration" 0 0
47 "p2p_send_message_NewBeaconTransaction_duration" 0 0
48 "p2p_send_message_NotifyLastTransactionAddress_duration" 0 0
49 "p2p_send_message_AcknowledgeStorage_duration" 0 0
50 "p2p_send_message_Ping_duration" 0 0
51 "p2p_send_message_StartMining_duration" 0 0
52 "p2p_send_message_AddMiningContext_duration" 0 0
53 "p2p_send_message_ListNodes_duration" 0 0
54 "p2p_send_message_GetUnspentOutputs_duration" 0 0
55 "p2p_send_message_GetBootstrappingNodes_duration" 0 0
56 "p2p_send_message_GetTransaction_duration" 0 0
       

@ghost
Copy link

ghost commented Jan 27, 2022

Full transaction Validation Duration (card/graph/guage) => maybe a graph or histogram
P2p _ send _ message _ duration => maybe a gauge
Proof of Work => maybe a gauge

@apoorv-2204 apoorv-2204 closed this Feb 3, 2022
@apoorv-2204 apoorv-2204 reopened this Feb 3, 2022
@internet-zero internet-zero assigned ghost Feb 14, 2022
@internet-zero internet-zero added the feature New feature request label Feb 14, 2022
@internet-zero internet-zero changed the title Metric Dashboard PR2 [Feature] Metric Dashboard PR2 Feb 14, 2022
@ghost
Copy link

ghost commented Feb 14, 2022

I got some UI issue when you click on the line in the charts, it disappears.

Also I think you should indicate in the UI than the update is every 5 seconds, and the polling is performed for nodes in the network, with aggregation of metrics.

def network_collector() do
data = Enum.filter(retrieve_network_metrics(), fn {key, _val} -> req_metrics(key) end)
data = Enum.into(data, %{})
Map.put(data, "tps", DB.get_latest_tps())
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the TPS should come from the metrics and not only from the DB.
This one is computed for an entire day or self-repair interval, which may not reflects the real load

Copy link
Contributor Author

@apoorv-2204 apoorv-2204 Feb 15, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

### Calculating The Transaction per second

archethic_mining_full_transaction_validation_duration_bucket{le="0.01"} 0
archethic_mining_full_transaction_validation_duration_bucket{le="0.025"} 0
archethic_mining_full_transaction_validation_duration_bucket{le="0.05"} 470
archethic_mining_full_transaction_validation_duration_bucket{le="0.1"} 1219
archethic_mining_full_transaction_validation_duration_bucket{le="0.5"} 1471
archethic_mining_full_transaction_validation_duration_bucket{le="0.8"} 1510
archethic_mining_full_transaction_validation_duration_bucket{le="1"} 1511
archethic_mining_full_transaction_validation_duration_bucket{le="1.2"} 1512
archethic_mining_full_transaction_validation_duration_bucket{le="1.5"} 1512
archethic_mining_full_transaction_validation_duration_bucket{le="2"} 1512
archethic_mining_full_transaction_validation_duration_bucket{le="2.5"} 1512
archethic_mining_full_transaction_validation_duration_bucket{le="3"} 1512
archethic_mining_full_transaction_validation_duration_bucket{le="5"} 1512
archethic_mining_full_transaction_validation_duration_bucket{le="10"} 1512
archethic_mining_full_transaction_validation_duration_bucket{le="+Inf"} 1512
archethic_mining_full_transaction_validation_duration_sum 162.63586202400006
archethic_mining_full_transaction_validation_duration_count 1512

So to calculate Transaction per second :
archethic_mining_full_transaction_validation_duration_sum 162.63586202400006
archethic_mining_full_transaction_validation_duration_count 1512

total no of events in less than 10 ms is 1512
So total number of events =1512
and total time 162 ms
.162 seconds --> 1512 txns
in 1s --> 1512/ .162
==> 9333 txn/s
is this maths correct ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hi Sir , please mention how to calculate Transaction Per second ? from /metrics endpoint . .???
@samuel-uniris

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

1512 is the number of transaction validation which take less than 10s, from the last polling interval (i.e 5 seconds)
162 is total of the observed values (sum of the milliseconds)
So the TPS would be: 1512/162: 9.33

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @samuel-uniris Sir,
I have to add all the txn per second of all the nodes. (and not do 1 by average total validation time of network.)

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you can compute the TPS for the all the nodes and then do the average

Copy link
Contributor Author

@apoorv-2204 apoorv-2204 Feb 15, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi Sir,
Averaging TPS shows average TPS across nodes . It shows

Average number of transaction per seconds for a node that is doing in the network , while in network all the combined nodes all are validating/processing TXN simultaneously per second is rather a huge number ?

Perhaps w.r.t ARCH consensus doing AVG of TPS is or , rather a SUM is meaningful ?

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think SUM will be more meaningful, as it will end up to be a bigger number. Because for now, all the nodes will be involved in the validation, so summing the TPS would create an inconsistent state I think.

For example:
Node1: 5 TPS
Node2: 5 TPS
Node3: 5 TPS

With SUM we will say the network can ingest 15 TPS, while in reality the network only manage 5 transactions but for different nodes, so the AVG may show more accurate values, I think.

Even if we are using sharding, several nodes are involved for a given transaction and not one transaction by node.

@apoorv-2204
Copy link
Contributor Author

apoorv-2204 commented Feb 15, 2022

I got some UI issue when you click on the line in the charts, it disappears.

Also I think you should indicate in the UI than the update is every 5 seconds, and the polling is performed for nodes in the network, with aggregation of metrics.

Hi Sir ,Pardon I didn't get you the second part. should indicate in the UI than the update is every 5 seconds ``do you want me to put tooltip showing those details?
@samuel-uniris

@ghost
Copy link

ghost commented Feb 15, 2022

We can put a headline to explain what is this screen

@ghost ghost merged commit 3d3e5da into archethic-foundation:master Feb 18, 2022
@apoorv-2204 apoorv-2204 changed the title [Feature] Metric Dashboard PR2 [Feature] Metric Dashboard Aug 16, 2022
This pull request was closed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature New feature request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants