Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor the calculation to avoid reduntant time conversions #2545

Merged
merged 1 commit into from Feb 4, 2020

Conversation

NelsonVides
Copy link
Collaborator

On shaper.erl, the arithmetics are done by converting to and from native-second time units, which shows on the profiling statistics I gathered last time:

for a scenario like "5k users sending a 1-to-1 message per second"

FUNCTION                              CALLS        %      TIME  [uS / CALLS]
--------                              -----  -------      ----  [----------]
erlang:monotonic_time/0                5999     0.01      1022  [      0.17]
erlang:convert_time_unit/3            11998     0.04      8580  [      0.72]

We see that convert_time_unit/3 is called twice as often as monotonic_time/0, which is exactly what shaper:update/2 does.

On fprof, we can see that, of the accumulated 130us that shaper:update/2 takes, 20us of them are spent on erlang:convert_time_unit/3, plus 4us on erlang:monotonic_time/0, while

{[{{ejabberd_receiver,process_data,2},         1409,  130.061,   51.471}],     
 { {shaper,update,2},                          1409,  130.061,   51.471},     %
 [{{erlang,convert_time_unit,3},               2818,   34.792,   20.457},      
  {{lager_config,get,2},                       1409,   15.401,   10.091},      
  {{erlang,whereis,1},                         2818,    9.563,    9.563},      
  {{erlang,max,2},                             2818,    9.154,    9.154},      
  {{erlang,monotonic_time,0},                  1409,    4.882,    4.882},      
  {{erlang,min,2},                             1409,    4.798,    4.798}]}.    

My main idea to optimise this, is this quote from the erlang:monotonic_time/1 docs:

Same as calling erlang:convert_time_unit(erlang:monotonic_time(), native, Unit), however optimized for commonly used Units.

@mongoose-im
Copy link
Collaborator

mongoose-im commented Dec 2, 2019

7504.1 / Erlang 22.0 / small_tests / 64fa7e4
Reports root / small


7504.2 / Erlang 22.0 / internal_mnesia / 64fa7e4
Reports root/ big
OK: 1274 / Failed: 0 / User-skipped: 138 / Auto-skipped: 0


7504.3 / Erlang 22.0 / odbc_mssql_mnesia / 64fa7e4
Reports root/ big
OK: 2493 / Failed: 0 / User-skipped: 199 / Auto-skipped: 0


7504.4 / Erlang 22.0 / mysql_redis / 64fa7e4
Reports root/ big
OK: 2488 / Failed: 0 / User-skipped: 204 / Auto-skipped: 0


7504.7 / Erlang 22.0 / elasticsearch_and_cassandra_mnesia / 64fa7e4
Reports root/ big
OK: 328 / Failed: 0 / User-skipped: 28 / Auto-skipped: 0


7504.6 / Erlang 22.0 / ldap_mnesia / 64fa7e4
Reports root/ big
OK: 1239 / Failed: 0 / User-skipped: 173 / Auto-skipped: 0


7504.5 / Erlang 22.0 / riak_mnesia / 64fa7e4
Reports root/ big
OK: 1407 / Failed: 0 / User-skipped: 145 / Auto-skipped: 0


7504.9 / Erlang 21.3 / pgsql_mnesia / 64fa7e4
Reports root/ big / small
OK: 2506 / Failed: 0 / User-skipped: 186 / Auto-skipped: 0

@codecov
Copy link

codecov bot commented Dec 2, 2019

Codecov Report

Merging #2545 into master will increase coverage by 0.01%.
The diff coverage is 100%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #2545      +/-   ##
==========================================
+ Coverage   78.99%   79.01%   +0.01%     
==========================================
  Files         343      343              
  Lines       29494    29491       -3     
==========================================
+ Hits        23298    23301       +3     
+ Misses       6196     6190       -6
Impacted Files Coverage Δ
src/shaper.erl 100% <100%> (+5.88%) ⬆️
src/ejabberd_receiver.erl 81.5% <100%> (ø) ⬆️
src/elasticsearch/mongoose_elasticsearch.erl 76.92% <0%> (-7.7%) ⬇️
src/mam/mod_mam_rdbms_prefs.erl 92.59% <0%> (-3.71%) ⬇️
src/rdbms/mongoose_rdbms.erl 69.74% <0%> (-2.06%) ⬇️
src/mam/mod_mam_elasticsearch_arch.erl 89.81% <0%> (-1.86%) ⬇️
src/mam/mod_mam_utils.erl 83.49% <0%> (-0.67%) ⬇️
src/ejabberd_c2s.erl 87.36% <0%> (-0.08%) ⬇️
src/mod_muc_log.erl 77.69% <0%> (ø) ⬆️
src/mod_bosh_socket.erl 81.51% <0%> (+0.33%) ⬆️
... and 2 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 21fa934...35850d5. Read the comment docs.

Copy link
Contributor

@michalwski michalwski left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you squash these 2 commits into one?

@NelsonVides
Copy link
Collaborator Author

Could you squash these 2 commits into one?

Done :)

@mongoose-im
Copy link
Collaborator

mongoose-im commented Dec 2, 2019

7510.1 / Erlang 22.0 / small_tests / a90f120
Reports root / small


7510.2 / Erlang 22.0 / internal_mnesia / a90f120
Reports root/ big
OK: 1274 / Failed: 0 / User-skipped: 138 / Auto-skipped: 0


7510.3 / Erlang 22.0 / odbc_mssql_mnesia / a90f120
Reports root/ big
OK: 2493 / Failed: 0 / User-skipped: 199 / Auto-skipped: 0


7510.4 / Erlang 22.0 / mysql_redis / a90f120
Reports root/ big
OK: 2488 / Failed: 0 / User-skipped: 204 / Auto-skipped: 0


7510.7 / Erlang 22.0 / elasticsearch_and_cassandra_mnesia / a90f120
Reports root/ big
OK: 328 / Failed: 0 / User-skipped: 28 / Auto-skipped: 0


7510.5 / Erlang 22.0 / riak_mnesia / a90f120
Reports root/ big
OK: 1407 / Failed: 0 / User-skipped: 145 / Auto-skipped: 0


7510.6 / Erlang 22.0 / ldap_mnesia / a90f120
Reports root/ big
OK: 1239 / Failed: 0 / User-skipped: 173 / Auto-skipped: 0


7510.9 / Erlang 21.3 / pgsql_mnesia / a90f120
Reports root/ big / small
OK: 2506 / Failed: 0 / User-skipped: 186 / Auto-skipped: 0

Copy link
Member

@fenek fenek left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The PR looks fine but we need a separate PR first that would introduce tests that focus on shapers verification.

@NelsonVides NelsonVides added WIP 🚧 and removed ready labels Dec 3, 2019
@mongoose-im
Copy link
Collaborator

mongoose-im commented Feb 4, 2020

7885.1 / Erlang 22.0 / small_tests / 4e51e93
Reports root / small


7885.2 / Erlang 22.0 / internal_mnesia / 4e51e93
Reports root/ big
OK: 1378 / Failed: 0 / User-skipped: 155 / Auto-skipped: 0


7885.4 / Erlang 22.0 / mysql_redis / 4e51e93
Reports root/ big
OK: 2608 / Failed: 0 / User-skipped: 205 / Auto-skipped: 0


7885.3 / Erlang 22.0 / odbc_mssql_mnesia / 4e51e93
Reports root/ big
OK: 2613 / Failed: 0 / User-skipped: 200 / Auto-skipped: 0


7885.5 / Erlang 22.0 / riak_mnesia / 4e51e93
Reports root/ big
OK: 1511 / Failed: 0 / User-skipped: 162 / Auto-skipped: 0


7885.7 / Erlang 22.0 / elasticsearch_and_cassandra_mnesia / 4e51e93
Reports root/ big
OK: 328 / Failed: 0 / User-skipped: 28 / Auto-skipped: 0


7885.6 / Erlang 22.0 / ldap_mnesia / 4e51e93
Reports root/ big
OK: 1343 / Failed: 0 / User-skipped: 190 / Auto-skipped: 0


7885.9 / Erlang 21.3 / pgsql_mnesia / 4e51e93
Reports root/ big / small
OK: 2626 / Failed: 0 / User-skipped: 187 / Auto-skipped: 0

@michalwski michalwski merged commit ffd9706 into master Feb 4, 2020
@michalwski michalwski deleted the perf/time_calls_in_shaper branch February 4, 2020 14:51
@michalwski michalwski added this to the 3.6.1 milestone Feb 11, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants