Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

C2s/stream management #3796

Merged
merged 25 commits into from
Oct 19, 2022
Merged

C2s/stream management #3796

merged 25 commits into from
Oct 19, 2022

Conversation

NelsonVides
Copy link
Collaborator

@NelsonVides NelsonVides commented Oct 5, 2022

The first few commits fix things not necessarily related to implementing stream management but come as a side necessity:

  • Now that user_send_packet runs with more kinds of packets, like for stream management, metrics would fail to match a non-standard stanza.
  • Stream management will need to reroute messages as a connection is being closed, but for that, the session must have already been closed in the session manager, otherwise rerouting would result in an infinite loop, so for that we separate a new kind of event after the session is closed.
  • c2s hooks are more open, and also consistent, about the parameters that a c2s event can take. Now they all take the full information about the c2s event, like, its event type and content, consistently.
  • The socket_send key in the c2s acc had the problem that it would bypass all the processing for user_receive_packet, which would then break stream management, so I had to update the usages of socket_send to a new route key, that would trigger a route event and do all the processing related, bringing it closer to how older code used to trigger a ejabberd_router:route/4 call.
  • Tests for stream management where breaking when the session is not mandatory and so other messages can be received out of order, so I had to extend c2s to recognise this state in the state machine definition, and make it configurable whether this will be used or not. Can also help with backwards compatibility, where two listeners with different configurations can be spawned and clients can be migrated to connect to the new one as they are configured not to required session establishment any more. On the other hand we could also just relax the tests for stream management and keep a single configuration with a single listener.

Now, for stream management, it registers on the following events:

  • user_send_packet: counts received packets from the user.
  • user_receive_packet: counts and buffers packets that are to be delivered to the user. This needs to be run late in the hook pipeline, as a different hook might have decided to block routing the packet.
  • user_send_xmlel: This captures foreign packets, those that are not standard stanzas, like, stream management packets themselves, and acts on them. They can be run outside of the XMPP session establishment order, like in the case of resumption.
  • foreign_event: captures gen_statem:calls where a different process is asking to resume this current session, and timeouts when the buffer has been full for long enough, to decide if SM has to take control or not.
  • user_stop_request, user_socket_closed, user_socket_error: they're handled by the same handler, and it gives SM a chance to take control over the c2s process by pushing itself as a new gen_statem callback module.
  • user_terminate: if the session is being closed because SM says so, stop all handling and don't give other modules the chance to break SM closing.
  • reroute_unacked_messages: after the session has been closed, try to reroute all the buffered messages.

@codecov
Copy link

codecov bot commented Oct 5, 2022

Codecov Report

Base: 70.36% // Head: 70.77% // Increases project coverage by +0.41% 🎉

Coverage data is based on head (abf9cbb) compared to base (e3a74ae).
Patch coverage: 90.40% of modified lines in pull request are covered.

Additional details and impacted files
@@                   Coverage Diff                    @@
##           feature/mongoose_c2s    #3796      +/-   ##
========================================================
+ Coverage                 70.36%   70.77%   +0.41%     
========================================================
  Files                       536      536              
  Lines                     34879    35209     +330     
========================================================
+ Hits                      24541    24919     +378     
+ Misses                    10338    10290      -48     
Impacted Files Coverage Δ
src/c2s/mongoose_c2s_listener.erl 85.71% <ø> (-1.47%) ⬇️
src/config/mongoose_config_spec.erl 100.00% <ø> (ø)
src/stream_management/mod_stream_management.erl 89.39% <89.19%> (+29.77%) ⬆️
src/mod_presence.erl 85.57% <90.00%> (-0.38%) ⬇️
src/c2s/mongoose_c2s.erl 72.77% <90.16%> (+3.97%) ⬆️
src/c2s/mongoose_c2s_acc.erl 100.00% <100.00%> (+18.51%) ⬆️
src/c2s/mongoose_c2s_hooks.erl 93.93% <100.00%> (+0.39%) ⬆️
src/c2s/mongoose_c2s_stanzas.erl 80.00% <100.00%> (+1.05%) ⬆️
src/metrics/mongoose_metrics_hooks.erl 97.75% <100.00%> (+0.02%) ⬆️
src/mod_ping.erl 98.43% <100.00%> (+0.13%) ⬆️
... and 26 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report at Codecov.
📢 Do you have feedback about the report comment? Let us know in this issue.

@mongoose-im

This comment was marked as outdated.

@mongoose-im

This comment was marked as outdated.

@mongoose-im

This comment was marked as outdated.

@mongoose-im

This comment was marked as outdated.

@mongoose-im

This comment was marked as outdated.

@mongoose-im

This comment was marked as outdated.

@mongoose-im

This comment was marked as outdated.

@mongoose-im

This comment was marked as outdated.

@NelsonVides NelsonVides force-pushed the c2s/stream_management branch 2 times, most recently from 55d1c0d to 3f9814d Compare October 12, 2022 20:01
@mongoose-im

This comment was marked as outdated.

@NelsonVides NelsonVides force-pushed the c2s/stream_management branch 2 times, most recently from 611f41f to d0f4bcd Compare October 12, 2022 20:17
@mongoose-im

This comment was marked as outdated.

@mongoose-im

This comment was marked as outdated.

@NelsonVides NelsonVides force-pushed the c2s/stream_management branch 2 times, most recently from 2a2a32f to 8f47f49 Compare October 12, 2022 22:42
User-send-packet is currently passing many more stanza types that this
won't match on. It needs a future refactor to use the more granular
hooks.
Some things might be required to happen while the session is still
registered, like distributing presences, while others when it is not
anymore, like rerouting stream-management buffers, which otherwise
would be rerouted to itself in an infinite loop.
@mongoose-im

This comment was marked as outdated.

@NelsonVides NelsonVides marked this pull request as ready for review October 13, 2022 15:06
@mongoose-im

This comment was marked as outdated.

@mongoose-im

This comment was marked as outdated.

@mongoose-im

This comment was marked as outdated.

src/c2s/mongoose_c2s.erl Outdated Show resolved Hide resolved
@mongoose-im
Copy link
Collaborator

mongoose-im commented Oct 14, 2022

small_tests_24 / small_tests / a538b53
Reports root / small


small_tests_25 / small_tests / a538b53
Reports root / small


ldap_mnesia_24 / ldap_mnesia / a538b53
Reports root/ big
OK: 1545 / Failed: 0 / User-skipped: 686 / Auto-skipped: 0


ldap_mnesia_25 / ldap_mnesia / a538b53
Reports root/ big
OK: 1545 / Failed: 0 / User-skipped: 686 / Auto-skipped: 0


pgsql_mnesia_24 / pgsql_mnesia / a538b53
Reports root/ big
OK: 3720 / Failed: 0 / User-skipped: 89 / Auto-skipped: 0


dynamic_domains_pgsql_mnesia_24 / pgsql_mnesia / a538b53
Reports root/ big
OK: 3544 / Failed: 0 / User-skipped: 78 / Auto-skipped: 0


internal_mnesia_25 / internal_mnesia / a538b53
Reports root/ big
OK: 1634 / Failed: 0 / User-skipped: 597 / Auto-skipped: 0


elasticsearch_and_cassandra_25 / elasticsearch_and_cassandra_mnesia / a538b53
Reports root/ big
OK: 1968 / Failed: 0 / User-skipped: 598 / Auto-skipped: 0


riak_mnesia_24 / riak_mnesia / a538b53
Reports root/ big
OK: 1826 / Failed: 1 / User-skipped: 590 / Auto-skipped: 0

pubsub_SUITE:tree+basic:publish_only_retract_items_scope_test
{error,{{badmatch,false},
    [{pubsub_tools,check_response,2,
             [{file,"/home/circleci/project/big_tests/tests/pubsub_tools.erl"},
            {line,491}]},
     {pubsub_tools,receive_response,3,
             [{file,"/home/circleci/project/big_tests/tests/pubsub_tools.erl"},
            {line,481}]},
     {pubsub_tools,receive_and_check_response,4,
             [{file,"/home/circleci/project/big_tests/tests/pubsub_tools.erl"},
            {line,471}]},
     {escalus_story,story,4,
            [{file,"/home/circleci/project/big_tests/_build/default/lib/escalus/src/escalus_story.erl"},
             {line,72}]},
     {test_server,ts_tc,3,[{file,"test_server.erl"},{line,1783}]},
     {test_server,run_test_case_eval1,6,
            [{file,"test_server.erl"},{line,1292}]},
     {test_server,run_test_case_eval,9,
            [{file,"test_server.erl"},{line,1224}]}]}}

Report log


dynamic_domains_pgsql_mnesia_25 / pgsql_mnesia / a538b53
Reports root/ big
OK: 3544 / Failed: 0 / User-skipped: 78 / Auto-skipped: 0


dynamic_domains_mysql_redis_25 / mysql_redis / a538b53
Reports root/ big
OK: 3518 / Failed: 0 / User-skipped: 104 / Auto-skipped: 0


pgsql_mnesia_25 / pgsql_mnesia / a538b53
Reports root/ big
OK: 3720 / Failed: 0 / User-skipped: 89 / Auto-skipped: 0


dynamic_domains_mssql_mnesia_25 / odbc_mssql_mnesia / a538b53
Reports root/ big
OK: 3544 / Failed: 0 / User-skipped: 78 / Auto-skipped: 0


mssql_mnesia_25 / odbc_mssql_mnesia / a538b53
Reports root/ big
OK: 3720 / Failed: 0 / User-skipped: 89 / Auto-skipped: 0


mysql_redis_25 / mysql_redis / a538b53
Reports root/ big
OK: 3706 / Failed: 0 / User-skipped: 103 / Auto-skipped: 0

Copy link
Member

@chrzaszcz chrzaszcz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, I just added some minor comments.

big_tests/tests/sm_SUITE.erl Outdated Show resolved Hide resolved
big_tests/tests/sm_helper.erl Outdated Show resolved Hide resolved
src/c2s/mongoose_c2s.erl Outdated Show resolved Hide resolved
src/c2s/mongoose_c2s_hooks.erl Show resolved Hide resolved
src/stream_management/mod_stream_management.erl Outdated Show resolved Hide resolved
src/stream_management/mod_stream_management.erl Outdated Show resolved Hide resolved
src/stream_management/mod_stream_management.erl Outdated Show resolved Hide resolved
src/stream_management/mod_stream_management.erl Outdated Show resolved Hide resolved
src/stream_management/mod_stream_management.erl Outdated Show resolved Hide resolved
src/stream_management/mod_stream_management.erl Outdated Show resolved Hide resolved
@mongoose-im
Copy link
Collaborator

mongoose-im commented Oct 17, 2022

small_tests_24 / small_tests / d6b0235
Reports root / small


small_tests_25 / small_tests / d6b0235
Reports root / small


ldap_mnesia_24 / ldap_mnesia / d6b0235
Reports root/ big
OK: 1545 / Failed: 0 / User-skipped: 686 / Auto-skipped: 0


ldap_mnesia_25 / ldap_mnesia / d6b0235
Reports root/ big
OK: 1545 / Failed: 0 / User-skipped: 686 / Auto-skipped: 0


pgsql_mnesia_24 / pgsql_mnesia / d6b0235
Reports root/ big
OK: 3720 / Failed: 0 / User-skipped: 89 / Auto-skipped: 0


dynamic_domains_pgsql_mnesia_24 / pgsql_mnesia / d6b0235
Reports root/ big
OK: 3544 / Failed: 0 / User-skipped: 78 / Auto-skipped: 0


internal_mnesia_25 / internal_mnesia / d6b0235
Reports root/ big
OK: 1634 / Failed: 0 / User-skipped: 597 / Auto-skipped: 0


elasticsearch_and_cassandra_25 / elasticsearch_and_cassandra_mnesia / d6b0235
Reports root/ big
OK: 1968 / Failed: 0 / User-skipped: 598 / Auto-skipped: 0


dynamic_domains_mysql_redis_25 / mysql_redis / d6b0235
Reports root/ big
OK: 3518 / Failed: 0 / User-skipped: 104 / Auto-skipped: 0


dynamic_domains_pgsql_mnesia_25 / pgsql_mnesia / d6b0235
Reports root/ big
OK: 3544 / Failed: 0 / User-skipped: 78 / Auto-skipped: 0


dynamic_domains_mssql_mnesia_25 / odbc_mssql_mnesia / d6b0235
Reports root/ big
OK: 3544 / Failed: 0 / User-skipped: 78 / Auto-skipped: 0


riak_mnesia_24 / riak_mnesia / d6b0235
Reports root/ big
OK: 1808 / Failed: 0 / User-skipped: 590 / Auto-skipped: 0


pgsql_mnesia_25 / pgsql_mnesia / d6b0235
Reports root/ big
OK: 3720 / Failed: 0 / User-skipped: 89 / Auto-skipped: 0


mysql_redis_25 / mysql_redis / d6b0235
Reports root/ big
OK: 3706 / Failed: 0 / User-skipped: 103 / Auto-skipped: 0


mssql_mnesia_25 / odbc_mssql_mnesia / d6b0235
Reports root/ big
OK: 3720 / Failed: 0 / User-skipped: 89 / Auto-skipped: 0

@mongoose-im
Copy link
Collaborator

mongoose-im commented Oct 19, 2022

small_tests_24 / small_tests / abf9cbb
Reports root / small


small_tests_25 / small_tests / abf9cbb
Reports root / small


ldap_mnesia_24 / ldap_mnesia / abf9cbb
Reports root/ big
OK: 1545 / Failed: 0 / User-skipped: 686 / Auto-skipped: 0


dynamic_domains_pgsql_mnesia_24 / pgsql_mnesia / abf9cbb
Reports root/ big
OK: 3544 / Failed: 0 / User-skipped: 78 / Auto-skipped: 0


ldap_mnesia_25 / ldap_mnesia / abf9cbb
Reports root/ big
OK: 1545 / Failed: 0 / User-skipped: 686 / Auto-skipped: 0


pgsql_mnesia_24 / pgsql_mnesia / abf9cbb
Reports root/ big
OK: 3720 / Failed: 0 / User-skipped: 89 / Auto-skipped: 0


internal_mnesia_25 / internal_mnesia / abf9cbb
Reports root/ big
OK: 1634 / Failed: 0 / User-skipped: 597 / Auto-skipped: 0


riak_mnesia_24 / riak_mnesia / abf9cbb
Reports root/ big
OK: 1808 / Failed: 0 / User-skipped: 590 / Auto-skipped: 0


elasticsearch_and_cassandra_25 / elasticsearch_and_cassandra_mnesia / abf9cbb
Reports root/ big
OK: 1968 / Failed: 0 / User-skipped: 598 / Auto-skipped: 0


dynamic_domains_pgsql_mnesia_25 / pgsql_mnesia / abf9cbb
Reports root/ big
OK: 3544 / Failed: 0 / User-skipped: 78 / Auto-skipped: 0


dynamic_domains_mysql_redis_25 / mysql_redis / abf9cbb
Reports root/ big
OK: 3518 / Failed: 0 / User-skipped: 104 / Auto-skipped: 0


pgsql_mnesia_25 / pgsql_mnesia / abf9cbb
Reports root/ big
OK: 3720 / Failed: 0 / User-skipped: 89 / Auto-skipped: 0


dynamic_domains_mssql_mnesia_25 / odbc_mssql_mnesia / abf9cbb
Reports root/ big
OK: 3544 / Failed: 0 / User-skipped: 78 / Auto-skipped: 0


mysql_redis_25 / mysql_redis / abf9cbb
Reports root/ big
OK: 3706 / Failed: 0 / User-skipped: 103 / Auto-skipped: 0


mssql_mnesia_25 / odbc_mssql_mnesia / abf9cbb
Reports root/ big
OK: 3720 / Failed: 0 / User-skipped: 89 / Auto-skipped: 0

@chrzaszcz chrzaszcz merged commit bbc6885 into feature/mongoose_c2s Oct 19, 2022
@chrzaszcz chrzaszcz deleted the c2s/stream_management branch October 19, 2022 12:17
@jacekwegr jacekwegr added this to the 6.1.0 milestone Apr 27, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants