Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use more granular c2s hooks #3955

Merged
merged 2 commits into from
Feb 20, 2023
Merged

Conversation

jacekwegr
Copy link
Contributor

@jacekwegr jacekwegr commented Feb 1, 2023

This PR changes the hooks used in the mod_event_pusher and mod_mam to more granular ones.

@mongoose-im

This comment was marked as outdated.

@codecov
Copy link

codecov bot commented Feb 1, 2023

Codecov Report

Base: 83.52% // Head: 83.47% // Decreases project coverage by -0.05% ⚠️

Coverage data is based on head (3f042fc) compared to base (8f20f7f).
Patch coverage: 83.33% of modified lines in pull request are covered.

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #3955      +/-   ##
==========================================
- Coverage   83.52%   83.47%   -0.05%     
==========================================
  Files         538      538              
  Lines       33953    33957       +4     
==========================================
- Hits        28358    28347      -11     
- Misses       5595     5610      +15     
Impacted Files Coverage Δ
.../event_pusher/mod_event_pusher_hook_translator.erl 88.57% <ø> (-2.86%) ⬇️
src/mongoose_stanza_api.erl 97.50% <80.00%> (-1.19%) ⬇️
src/mam/mod_mam_pm.erl 89.60% <100.00%> (ø)
src/pubsub/mod_pubsub_db.erl 57.14% <0.00%> (-28.58%) ⬇️
src/async_pools/mongoose_aggregator_worker.erl 63.33% <0.00%> (-5.01%) ⬇️
src/smart_markers/mod_smart_markers_rdbms.erl 91.66% <0.00%> (-3.34%) ⬇️
src/muc_light/mod_muc_light_db_mnesia.erl 91.57% <0.00%> (-1.06%) ⬇️
src/wpool/mongoose_wpool.erl 82.24% <0.00%> (-0.94%) ⬇️
src/rdbms/mongoose_rdbms.erl 65.26% <0.00%> (-0.71%) ⬇️
... and 4 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report at Codecov.
📢 Do you have feedback about the report comment? Let us know in this issue.

@mongoose-im

This comment was marked as outdated.

@jacekwegr jacekwegr marked this pull request as ready for review February 2, 2023 09:20
Copy link
Collaborator

@NelsonVides NelsonVides left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The two modules still subscribed to user_send_packet:

  • MAM: don't we want to move it to user_send_message as well?
  • event pusher: it is probably right, but, have we verified that down the line the event pusher only cares about messages? I'd be happy with a pointer to the line of code were we can know that :)

Base automatically changed from feature/mongoose_c2s to master February 6, 2023 14:55
@@ -22,7 +22,7 @@

-export([add_hooks/1, delete_hooks/1]).

-export([user_send_packet/3,
-export([user_send_message/3,
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not 100% sure if really want to change the hook type here.

@DenysGonchar
Copy link
Collaborator

What about other places, e.g. mam, inbox, mod_offline (actually all the places that use user_send_packet hook)? Even if we don't need to change anything there, that would be nice to see some analysis and conclusions.

@jacekwegr
Copy link
Contributor Author

The two modules still subscribed to user_send_packet:

* MAM: don't we want to move it to `user_send_message` as well?

* event pusher: it is probably right, but, have we verified that down the line the event pusher only cares about messages? I'd be happy with a pointer to the line of code were we can know that :)
  • I think that MAM is also used for querying for messages, and when you change the hook to user_send_message, it stops working, and tests like graphql_stanza_SUITE:admin_stanza_http:admin_mam:admin_get_last_messages are going to fail, so I decided to leave it as is.

  • Correct me if I'm wrong, but the records found in include/mod_event_pusher_events.hrl and which are used by src/event_pusher/mod_event_pusher_hook_translator.erl can suggest that it is used only for messages.

@jacekwegr
Copy link
Contributor Author

What about other places, e.g. mam, inbox, mod_offline (actually all the places that use user_send_packet hook)? Even if we don't need to change anything there, that would be nice to see some analysis and conclusions.

  • Changing the mod_mam hook to user_send_message causes some trouble while querying for messages, so I think it is better to leave it as is
  • mod_inbox, mod_presence, mod_smart_markers and mod_carboncopy were already changed to use user_send_message and user_receive_message in this PR C2s/refactor user send packet #3857
  • mod_ping uses user_send_packet, but it makes sense because it is not used for messages
  • mod_stream_management uses user_send_packet and user_receive_packet, which also makes sense to me because it is not used for handling messages
  • metrics use user_send_packet and user_receive_packet to then divide by the type of message, which seems fine

@NelsonVides
Copy link
Collaborator

ping, stream management, and metrics, are the ones that actually make sense they should capture all packets.

About the event_pusher, I also understood it cares only for messages: in the hook handler, it checks for chat_type and will act only when the type is chat, groupchat, headline, normal, or not defined. This means that it surely can't be:

About MAM, this one's weird, because as far as I understand https://xmpp.org/extensions/xep-0313.html it seems to me to require only messages, so I don't know why MAM would be concerned about the other types of messages. If a test fails I find that suspicious 🙄 MAM is also used for fetching messages, but the fetch happens in the IQ handler, here the hook is only capturing the messages that will be stored 🤔

So I'd only ask, do we want to investigate and fix that MAM failure in this PR or in some other case? If leaving that for later I'd approve this one for now 👌🏽

@mongoose-im
Copy link
Collaborator

mongoose-im commented Feb 17, 2023

small_tests_24 / small_tests / 3f042fc
Reports root / small


small_tests_25 / small_tests / 3f042fc
Reports root / small


ldap_mnesia_24 / ldap_mnesia / 3f042fc
Reports root/ big
OK: 2219 / Failed: 0 / User-skipped: 825 / Auto-skipped: 0


dynamic_domains_pgsql_mnesia_24 / pgsql_mnesia / 3f042fc
Reports root/ big
OK: 4169 / Failed: 0 / User-skipped: 88 / Auto-skipped: 0


ldap_mnesia_25 / ldap_mnesia / 3f042fc
Reports root/ big
OK: 2219 / Failed: 0 / User-skipped: 825 / Auto-skipped: 0


dynamic_domains_mysql_redis_25 / mysql_redis / 3f042fc
Reports root/ big
OK: 4143 / Failed: 0 / User-skipped: 114 / Auto-skipped: 0


dynamic_domains_pgsql_mnesia_25 / pgsql_mnesia / 3f042fc
Reports root/ big
OK: 4169 / Failed: 0 / User-skipped: 88 / Auto-skipped: 0


pgsql_mnesia_24 / pgsql_mnesia / 3f042fc
Reports root/ big
OK: 4549 / Failed: 0 / User-skipped: 97 / Auto-skipped: 0


dynamic_domains_mssql_mnesia_25 / odbc_mssql_mnesia / 3f042fc
Reports root/ big
OK: 4165 / Failed: 1 / User-skipped: 91 / Auto-skipped: 0

muc_SUITE:register:user_unregisters_nick_twice
{error,
  {{assertion_failed,assert,is_iq_result,
     [{xmlel,<<"iq">>,
        [{<<"type">>,<<"set">>},
         {<<"id">>,<<"df1b582a7e938deb17cfe739d2eee65d">>},
         {<<"to">>,<<"groupchats.domain.example.com">>}],
        [{xmlel,<<"query">>,
           [{<<"xmlns">>,<<"jabber:iq:register">>}],
           [{xmlel,<<"x">>,
            [{<<"xmlns">>,<<"jabber:x:data">>},
             {<<"type">>,<<"submit">>}],
            [{xmlel,<<"field">>,
               [{<<"type">>,<<"hidden">>},
                {<<"var">>,<<"FORM_TYPE">>}],
               [{xmlel,<<"value">>,[],
                  [{xmlcdata,<<"jabber:iq:register">>}]}]},
             {xmlel,<<"field">>,
               [{<<"type">>,<<"text-single">>},
                {<<"var">>,<<"nick">>}],
               [{xmlel,<<"value">>,[],
                  [{xmlcdata,
                     <<"thirdwitch1room-9daa74711a">>}]}]}]}]}]}],
     {xmlel,<<"iq">>,
       [{<<"from">>,<<"groupchats.domain.example.com">>},
        {<<"to">>,
         <<"alice_user_unregisters_nick_twice_2437@domain.example.com/res1">>},
        {<<"type">>,<<"error">>},
        {<<"id">>,<<"df1b582a7e938deb17cfe739d2eee65d">>}],
       [{xmlel,<<"query">>,
          [{<<"xmlns">>,<<"jabber:iq:register">>}],
          [{xmlel,<<"x">>,
             [{<<"xmlns">>,<<"jabber:x:data">>},
            {<<"type">>,<<"submit">>}],
             [{xmlel,<<"field">>,
              [{<<"type">>,<<"hidden">>},
               {<<"var">>,<<"FORM_TYPE">>}],
        ...

Report log


elasticsearch_and_cassandra_25 / elasticsearch_and_cassandra_mnesia / 3f042fc
Reports root/ big
OK: 2721 / Failed: 0 / User-skipped: 662 / Auto-skipped: 0


internal_mnesia_25 / internal_mnesia / 3f042fc
Reports root/ big
OK: 2360 / Failed: 1 / User-skipped: 683 / Auto-skipped: 0

graphql_session_SUITE:admin_session:admin_session_cli:admin_list_users_with_status
{error,
  {{assertEqual,
     [{module,graphql_session_SUITE},
      {line,591},
      {expression,"lists : sort ( ActualJIDs )"},
      {expected,
        [<<"alice_admin_list_users_with_status_912@localhost.bis/res1">>]},
      {value,[]}]},
   [{graphql_session_SUITE,check_users,2,
      [{file,
         "/home/circleci/project/big_tests/tests/graphql_session_SUITE.erl"},
       {line,591}]},
    {graphql_session_SUITE,admin_list_users_with_status_story,3,
      [{file,
         "/home/circleci/project/big_tests/tests/graphql_session_SUITE.erl"},
       {line,422}]},
    {escalus_story,story,4,
      [{file,
         "/home/circleci/project/big_tests/_build/default/lib/escalus/src/escalus_story.erl"},
       {line,72}]},
    {test_server,ts_tc,3,[{file,"test_server.erl"},{line,1782}]},
    {test_server,run_test_case_eval1,6,
      [{file,"test_server.erl"},{line,1291}]},
    {test_server,run_test_case_eval,9,
      [{file,"test_server.erl"},{line,1223}]}]}}

Report log


riak_mnesia_24 / riak_mnesia / 3f042fc
Reports root/ big
OK: 2559 / Failed: 0 / User-skipped: 654 / Auto-skipped: 0


pgsql_mnesia_25 / pgsql_mnesia / 3f042fc
Reports root/ big
OK: 4549 / Failed: 0 / User-skipped: 97 / Auto-skipped: 0


mssql_mnesia_25 / odbc_mssql_mnesia / 3f042fc
Reports root/ big
OK: 4546 / Failed: 0 / User-skipped: 100 / Auto-skipped: 0


mysql_redis_25 / mysql_redis / 3f042fc
Reports root/ big
OK: 4534 / Failed: 1 / User-skipped: 111 / Auto-skipped: 0

push_integration_SUITE:pubsub_less:pm_notifications_with_inbox:inbox_msg_reset_unread_count_fcm
{error,
  {{assertion_failed,assert_many,false,
     [is_presence,is_message,is_message],
     [{xmlel,<<"message">>,
        [{<<"from">>,
        <<"alice_inbox_msg_reset_unread_count_fcm_2794@localhost/res1">>},
         {<<"to">>,
        <<"bob_inbox_msg_reset_unread_count_fcm_2794@localhost">>},
         {<<"type">>,<<"chat">>},
         {<<"id">>,<<"5a4e63a7f338821b1875c9a0c0293e47">>}],
        [{xmlel,<<"body">>,[],[{xmlcdata,<<"FIRST MESSAGE">>}]},
         {xmlel,<<"delay">>,
           [{<<"xmlns">>,<<"urn:xmpp:delay">>},
          {<<"stamp">>,<<"2023-02-17T14:37:05.201184Z">>},
          {<<"from">>,<<"localhost">>}],
           [{xmlcdata,<<"Offline Storage">>}]}]},
      {xmlel,<<"presence">>,
        [{<<"from">>,
        <<"bob_inbox_msg_reset_unread_count_fcm_2794@localhost/res1">>},
         {<<"to">>,
        <<"bob_inbox_msg_reset_unread_count_fcm_2794@localhost/res1">>}],
        []}],
     "   <message from='alice_inbox_msg_reset_unread_count_fcm_2794@localhost/res1' to='bob_inbox_msg_reset_unread_count_fcm_2794@localhost' type='chat' id='5a4e63a7f338821b1875c9a0c0293e47'><body>FIRST MESSAGE</body><delay xmlns='urn:xmpp:delay' stamp='2023-02-17T14:37:05.201184Z' from='localhost'>Offline Storage</delay></message>   <presence from='bob_inbox_msg_reset_unread_count_fcm_2794@localhost/res1' to='bob_inbox_msg_reset_unread_count_fcm_2794@localhost/res1'/>"},
   [{escalus_new_assert,assert_true,2,
      [{file,
         "/home/circleci/project/big_tests/_buil...

Report log


dynamic_domains_mssql_mnesia_25 / odbc_mssql_mnesia / 3f042fc
Reports root/ big
OK: 4166 / Failed: 0 / User-skipped: 91 / Auto-skipped: 0


internal_mnesia_25 / internal_mnesia / 3f042fc
Reports root/ big
OK: 2361 / Failed: 0 / User-skipped: 683 / Auto-skipped: 0


mysql_redis_25 / mysql_redis / 3f042fc
Reports root/ big
OK: 4535 / Failed: 0 / User-skipped: 111 / Auto-skipped: 0

Copy link
Collaborator

@NelsonVides NelsonVides left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM 👌🏽

Copy link
Collaborator

@DenysGonchar DenysGonchar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks ok

@DenysGonchar DenysGonchar merged commit 5fa12c5 into master Feb 20, 2023
@DenysGonchar DenysGonchar deleted the use-more-granular-c2s-hooks branch February 20, 2023 11:44
@jacekwegr jacekwegr added this to the 6.1.0 milestone Apr 26, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants