Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

perf: use application:get_env/3 instead of application:get_env/1 #41

Merged
merged 1 commit into from Feb 8, 2024

Conversation

SergeTupchiy
Copy link

application:get_env/1 calls ets:match/2 on ac_tab and can be very inefficient.

`application:get_env/1` calls `ets:match/2` on `ac_tab` and can be very inefficient.
@SergeTupchiy
Copy link
Author

Some eprof stats from emqx_resource_buffer_worker before this fix:

****** Process <0.5384.0>    -- 0.11 % of profiled time *** 
FUNCTION                                                                                    CALLS        %     TIME  [uS / CALLS]
--------                                                                                    -----  -------     ----  [----------]
...
emqx_placeholder:proc_tmpl/3                                                                12564     0.44     5990  [      0.48]
replayq:pop/5                                                                               11530     0.44     5997  [      0.52]
erlang:monitor/2                                                                             6282     0.45     6049  [      0.96]
mc_worker_api:assign_id/1                                                                    3141     0.45     6068  [      1.93]
erlang:external_size/1                                                                       5765     0.46     6148  [      1.07]
lists:all/2                                                                                  3141     0.47     6278  [      2.00]
emqx_bridge_mongodb_connector:on_query/3                                                     3141     0.47     6366  [      2.03]
mongo_id_server:object_id/0                                                                  3141     0.49     6633  [      2.11]
emqx_resource_buffer_worker:aggregate_counters/2                                             8957     0.53     7156  [      0.80]
erlang:list_to_binary/1                                                                      6282     0.57     7726  [      1.23]
erts_internal:map_next/3                                                                    12132     0.58     7769  [      0.64]
maps:try_next/2                                                                             52880     0.62     8342  [      0.16]
maps:update_with/4                                                                           8906     0.63     8450  [      0.95]
gproc:get_value/2                                                                            5765     0.64     8601  [      1.49]
emqx_utils_json:safe_decode/2                                                               15705     0.66     8921  [      0.57]
emqx_resource_buffer_worker:do_flush/2                                                       3141     0.66     8953  [      2.85]
emqx_bridge_mongodb_connector:'-create_mapping_of_references_to_tuple_values/1-fun-0-'/3    43974     0.69     9276  [      0.21]
emqx_placeholder:'-proc_tmpl/3-fun-0-'/3                                                    37692     0.69     9282  [      0.25]
emqx_placeholder:lookup_var/2                                                               31410     0.72     9747  [      0.31]
emqx_metrics_worker:get_pterm/1                                                              6380     0.79    10623  [      1.67]
erlang:binary_to_existing_atom/2                                                            15705     0.80    10783  [      0.69]
erlang:send/2                                                                                8905     0.87    11698  [      1.31]
erlang:spawn/3                                                                               3141     0.89    11994  [      3.82]
erlang:send/3                                                                                6282     0.90    12079  [      1.92]
gproc_pool:pick_near/3                                                                       5765     1.00    13532  [      2.35]
emqx_resource_buffer_worker:extract_connector_id/1                                           6282     1.11    15020  [      2.39]
emqx_resource_buffer_worker:flush/1                                                          5815     1.17    15768  [      2.71]
lists:map_1/2                                                                               46598     1.20    16191  [      0.35]
rand:seed_put/1                                                                              5766     1.25    16791  [      2.91]
ets:next/2                                                                                   5765     1.39    18781  [      3.26]
emqx_resource_buffer_worker:cancel_flush_timer/1                                             5815     1.44    19476  [      3.35]
erlang:demonitor/2                                                                          12564     1.45    19545  [      1.56]
binary:split/3                                                                               6282     1.52    20493  [      3.26]
maps:fold_1/4                                                                               64978     1.56    20996  [      0.32]
jiffy:nif_decode_init/2                                                                      3141     2.46    33207  [     10.57]
ets:lookup/2                                                                                20624     2.49    33548  [      1.63]
gproc:lookup_value/1                                                                         3141     2.53    34097  [     10.86]
jiffy:decode/2                                                                              18846     3.00    40396  [      2.14]
emqx_placeholder:lookup/2                                                                   15705     3.31    44649  [      2.84]
mc_topology:get_pool/3                                                                       3141     3.32    44742  [     14.24]
emqx_placeholder:do_one_lookup/2                                                            31410     3.60    48480  [      1.54]
ets:lookup_element/3                                                                        12097     3.65    49196  [      4.07]
gen:do_call/4                                                                               12564     7.45   100349  [      7.99]
ets:match/2                                                                                  3141     9.75   131384  [     41.83]
----------------------------------------------------------------------------------------  -------  -------  -------  [----------]
Total:                                                                                    1430744  100.00%  1347828  [      0.94]

@SergeTupchiy
Copy link
Author

SergeTupchiy commented Feb 8, 2024

~40K/s pubs, 4min duration test on 1 EMQX 5.5.0 otp26 node and single mongodb instance (both are c5.4xlarge):
otp-26

~40K/s pubs, 5min duration test on 1 EMQX 5.5.0 otp26 node and single mongodb instance (both are c5.4xlarge) patched with this fix:
emqx-550-otp26-get-env-3

Copy link

@thalesmg thalesmg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🥇

@SergeTupchiy SergeTupchiy merged commit b791892 into emqx:master Feb 8, 2024
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants