Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Collect EOS-VM-OC memory usage from the mainnet #1159

Closed
Tracked by #149
linh2931 opened this issue May 11, 2023 · 4 comments
Closed
Tracked by #149

Collect EOS-VM-OC memory usage from the mainnet #1159

linh2931 opened this issue May 11, 2023 · 4 comments
Assignees

Comments

@linh2931
Copy link
Member

This is one of work items identified in eosnetworkfoundation/product#149 for eosnetworkfoundation/product#161.

To configure a reasonable memory slice count for EOC-VM-OC, we need to understand typical memory usages.
This work will use a January 2022 snapshot from the mainnet, instrument EOS-VM-OC code to keep track and report
the largest number of linear memory the current code uses, start nodeos from the snapshot and sync with a mainnet peer node,
collect memory usage, and find max and 95 percentile of number of linear memory pages used by all contract codes.

@linh2931
Copy link
Member Author

Will also collect EVM contract memory usage.

@linh2931
Copy link
Member Author

The steps for this are

  1. Instrument OC code
  • In libraries/chain/include/eosio/chain/webassembly/eos-vm-oc/eos-vm-oc.h
      int64_t current_linear_memory_pages; //-1 if no memory
+   int64_t largest_linear_memory_pages;
  • In libraries/chain/webassembly/runtimes/eos-vm-oc/executor.cpp,
      cb->globals = globals;
+   cb->largest_linear_memory_pages = cb->current_linear_memory_pages;
...
+
+   if (context.get_receiver() == "eosio.evm"_n)
+      std::cout << "EVM_OC_LARGEST_PAGES " << cb->largest_linear_memory_pages << std::endl;
+   else
+      std::cout << "NATIVE_OC_LARGEST_PAGES " << cb->largest_linear_memory_pages << std::endl;
  • In libraries/chain/webassembly/runtimes/eos-vm-oc/gs_seg_helpers.c
+   if (cb_ptr->current_linear_memory_pages > cb_ptr->largest_linear_memory_pages) {
+      cb_ptr->largest_linear_memory_pages = cb_ptr->current_linear_memory_pages;
+   }
  1. Download a snapshot from the mainnet.
  2. Resync to mainnet starting from the snapshot and save nodeos log to a file.
  3. grep EVM_OC_LARGEST_PAGES and NATIVE_OC_LARGEST_PAGES from the log file
  4. Process the data

@linh2931
Copy link
Member Author

Below are result from April 12 mainnet snapshot and syncing to May 23.

For EVM contracts, 1 MB (16 pages), covers 88% of actions. 2 MB will cover 95% actions.

EVM Contracts

page  occurrences  accumulated
163   2           74101
157   1           74099
154   1           74098
137   1           74097
136   6           74096
135   2           74090
134   2           74088
132   2           74086
129   1           74084
125   2           74083
124   5           74081
123   21          74076
118   1           74055
115   1           74054
114   35          74053
113   9           74018
109   6           74009
105   2           74003
97    2           74001
96    3           73999
95    5           73996
94    1           73991
92    1           73990
91    1           73989
86    2           73988
83    1           73986
81    2           73985
80    4           73983
79    1           73979
76    15          73978
75    6           73963
72    2           73957
71    20          73955
70    4           73935
69    2           73931
68    145         73929
67    1           73784
66    42          73783
65    27          73741
64    5           73714
63    2           73709
62    3           73707
61    16          73704
60    11          73688
59    15          73677
58    97          73662
57    12          73565
56    21          73553
55    31          73532
54    23          73501
52    20          73478
51    37          73458
50    21          73421
49    384         73400
48    233         73016
47    97          72783
46    189         72686
45    30          72497
44    387         72467
43    107         72080
42    10          71973
41    15          71963
40    12          71948
39    4           71936
38    93          71932
37    304         71839
36    185         71535
35    47          71350
34    55          71303
33    148         71248
32    114         71100
31    764         70986
30    74          70222
29    68          70148
28    111         70080
27    364         69969
26    238         69605
25    482         69367
24    487         68885
23    263         68398
22    739         68135
21    467         67396
20    894         66929
19    500         66035
18    50          65535
17    40          65485
16    258         65445
15    1421        65187
14    6008        63766
13    7392        57758
12    1841        50366
11    6941        48525
10    4926        41584
9     2398        36658
8     470         34260
7     4936        33790
6     1248        28854
5     456         27606
4     2387        27150
3     10835       24763
2     13928       13928

Native contracts

page  occurrences  accumulated
94    11          62976083
93    1419        62976072
84    1430        62974653
58    1           62973223
57    109         62973222
56    212         62973113
55    32          62972901
54    5           62972869
53    4           62972864
52    137         62972860
51    275         62972723
50    299         62972448
49    279         62972149
48    490         62971870
47    236         62971380
46    91          62971144
45    74          62971053
44    73          62970979
43    110         62970906
42    73          62970796
41    93          62970723
40    90          62970630
39    62          62970540
38    82          62970478
37    84          62970396
36    277         62970312
35    282         62970035
34    156         62969753
33    206         62969597
32    332         62969391
31    273         62969059
30    298         62968786
29    283         62968488
28    296         62968205
27    342         62967909
26    334         62967567
25    1547        62967233
24    326         62965686
23    424         62965360
22    464         62964936
21    547         62964472
20    695         62963925
19    499         62963230
18    944         62962731
17    955         62961787
16    1321        62960832
15    1598        62959511
14    1957        62957913
13    2916        62955956
12    2447        62953040
11    2857        62950593
10    21824       62947736
9     7197        62925912
8     14716       62918715
7     13134       62903999
6     17605       62890865
5     28754       62873260
4     47869       62844506
3     140945      62796637
2     485981      62655692
1     62169460    62169711
-1    251         251

@linh2931
Copy link
Member Author

Information has been collected. Close this issue. Will collect new on on a future date if needed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Archived in project
Development

No branches or pull requests

4 participants