Add retry logic into mongooseim task aggregator #3969

arcusfelis · 2023-02-20T18:03:00Z

This PR addresses "Failing inbox tests on mssql". A part of MIM-1823 ticket.

Proposed changes include:

Retry logic and a test

codecov · 2023-02-20T18:12:39Z

Codecov Report

Patch coverage: 85.18% and no project coverage change

Comparison is base (5fa12c5) 83.54% compared to head (110d8f9) 83.54%.

Additional details and impacted files

@@           Coverage Diff           @@
##           master    #3969   +/-   ##
=======================================
  Coverage   83.54%   83.54%           
=======================================
  Files         538      538           
  Lines       34007    34022   +15     
=======================================
+ Hits        28412    28425   +13     
- Misses       5595     5597    +2

Impacted Files	Coverage Δ
src/inbox/mod_inbox_rdbms_async.erl	`71.01% <0.00%> (+0.42%)`	⬆️
src/async_pools/mongoose_aggregator_worker.erl	`78.37% <92.00%> (+15.04%)`	⬆️
src/event_pusher/mod_event_pusher_sns.erl	`82.19% <0.00%> (-12.33%)`	⬇️
src/domain/service_domain_db.erl	`83.33% <0.00%> (-2.09%)`	⬇️
src/wpool/mongoose_wpool.erl	`82.24% <0.00%> (-0.94%)`	⬇️
src/rdbms/mongoose_rdbms.erl	`66.31% <0.00%> (-0.71%)`	⬇️
src/mod_muc_log.erl	`62.82% <0.00%> (ø)`
src/mod_roster.erl	`79.28% <0.00%> (+0.71%)`	⬆️
src/mod_caps.erl	`78.01% <0.00%> (+0.86%)`	⬆️
... and 1 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report at Codecov.
📢 Do you have feedback about the report comment? Let us know in this issue.

…ve coverage

pawlooss1

In general it looks good 👍 I left a few minor comments to consider. Also, CI won't allow to merge because of the coverage decrease. Please look into it.

pawlooss1 · 2023-02-27T11:39:22Z

src/async_pools/mongoose_aggregator_worker.erl

+    %% Reset number of retries
+    maybe_request_next2(State#state{retries_left = Total}).
+
+maybe_request_next2(#state{flush_elems = Acc, flush_queue = Queue} = State) ->


Maybe instead of creating a function with suffix 2 we could go with something like this:

maybe_request_next(#state{flush_elems = Acc, flush_queue = Queue, total_retries = Total, retries_left = Total} = State) -> case queue:out(Queue) of {{value, Key}, NewQueue} -> {Value, NewAcc} = maps:take(Key, Acc), NewState1 = State#state{flush_elems = NewAcc, flush_queue = NewQueue}, case make_async_request(Value, NewState1) of NewState2 = #state{async_request = no_request_pending} -> maybe_request_next(NewState2); NewState2 -> NewState2 end; {empty, _} -> State#state{async_request = no_request_pending} end; maybe_request_next(#state{total_retries = Total} = State) -> %% Reset number of retries maybe_request_next(State#state{retries_left = Total}).

What do you think?

I can use State1 = State#state{retries_left = Total}.
Don't like recursion here, makes code a bit harder.

src/async_pools/mongoose_aggregator_worker.erl

test/batches_SUITE.erl

arcusfelis · 2023-02-27T14:54:52Z

yeah, coverage decrease is annoying, because we didn't had a test for ignore return case.
So, now when I am touching these lines with (to reindent or move to another function), I am forced to add these missing tests. Maybe I should just touch more lines, it is how the original code got merged (sarcasm haha).

…her helpers similarly

mongoose-im · 2023-02-27T17:27:15Z

small_tests_24 / small_tests / 08a9a33
Reports root / small

small_tests_25 / small_tests / 08a9a33
Reports root / small

dynamic_domains_pgsql_mnesia_24 / pgsql_mnesia / 08a9a33
Reports root/ big
OK: 4169 / Failed: 0 / User-skipped: 88 / Auto-skipped: 0

ldap_mnesia_25 / ldap_mnesia / 08a9a33
Reports root/ big
OK: 2219 / Failed: 0 / User-skipped: 825 / Auto-skipped: 0

dynamic_domains_mysql_redis_25 / mysql_redis / 08a9a33
Reports root/ big
OK: 4143 / Failed: 0 / User-skipped: 114 / Auto-skipped: 0

dynamic_domains_pgsql_mnesia_25 / pgsql_mnesia / 08a9a33
Reports root/ big
OK: 4169 / Failed: 0 / User-skipped: 88 / Auto-skipped: 0

pgsql_mnesia_24 / pgsql_mnesia / 08a9a33
Reports root/ big
OK: 4548 / Failed: 1 / User-skipped: 97 / Auto-skipped: 0

push_integration_SUITE:pubsub_less:pm_notifications_with_inbox:inbox_msg_reset_unread_count_apns

{error,
  {{assertion_failed,assert_many,false,
     [is_presence,is_message,is_message],
     [{xmlel,<<"message">>,
        [{<<"from">>,
        <<"alice_inbox_msg_reset_unread_count_apns_2894@localhost/res1">>},
         {<<"to">>,
        <<"bob_inbox_msg_reset_unread_count_apns_2894@localhost">>},
         {<<"type">>,<<"chat">>},
         {<<"id">>,<<"4b464c6e210fee3d8db78336c0e6c20e">>}],
        [{xmlel,<<"body">>,[],[{xmlcdata,<<"FIRST MESSAGE">>}]},
         {xmlel,<<"delay">>,
           [{<<"xmlns">>,<<"urn:xmpp:delay">>},
          {<<"stamp">>,<<"2023-02-27T17:34:36.567605Z">>},
          {<<"from">>,<<"localhost">>}],
           [{xmlcdata,<<"Offline Storage">>}]}]},
      {xmlel,<<"presence">>,
        [{<<"from">>,
        <<"bob_inbox_msg_reset_unread_count_apns_2894@localhost/res1">>},
         {<<"to">>,
        <<"bob_inbox_msg_reset_unread_count_apns_2894@localhost/res1">>}],
        []}],
     "   <message from='alice_inbox_msg_reset_unread_count_apns_2894@localhost/res1' to='bob_inbox_msg_reset_unread_count_apns_2894@localhost' type='chat' id='4b464c6e210fee3d8db78336c0e6c20e'><body>FIRST MESSAGE</body><delay xmlns='urn:xmpp:delay' stamp='2023-02-27T17:34:36.567605Z' from='localhost'>Offline Storage</delay></message>   <presence from='bob_inbox_msg_reset_unread_count_apns_2894@localhost/res1' to='bob_inbox_msg_reset_unread_count_apns_2894@localhost/res1'/>"},
   [{escalus_new_assert,assert_true,2,
      [{file,
         "/home/circleci/project/big_tes...

Report log

dynamic_domains_mssql_mnesia_25 / odbc_mssql_mnesia / 08a9a33
Reports root/ big
OK: 4166 / Failed: 0 / User-skipped: 91 / Auto-skipped: 0

internal_mnesia_25 / internal_mnesia / 08a9a33
Reports root/ big
OK: 2361 / Failed: 0 / User-skipped: 683 / Auto-skipped: 0

elasticsearch_and_cassandra_25 / elasticsearch_and_cassandra_mnesia / 08a9a33
Reports root/ big
OK: 2721 / Failed: 0 / User-skipped: 662 / Auto-skipped: 0

pgsql_mnesia_25 / pgsql_mnesia / 08a9a33
Reports root/ big
OK: 4549 / Failed: 0 / User-skipped: 97 / Auto-skipped: 0

riak_mnesia_24 / riak_mnesia / 08a9a33
Reports root/ big
OK: 2559 / Failed: 0 / User-skipped: 654 / Auto-skipped: 0

mysql_redis_25 / mysql_redis / 08a9a33
Reports root/ big
OK: 4535 / Failed: 0 / User-skipped: 111 / Auto-skipped: 0

mssql_mnesia_25 / odbc_mssql_mnesia / 08a9a33
Reports root/ big
OK: 4546 / Failed: 0 / User-skipped: 100 / Auto-skipped: 0

pgsql_mnesia_24 / pgsql_mnesia / 08a9a33
Reports root/ big
OK: 4549 / Failed: 0 / User-skipped: 97 / Auto-skipped: 0

pawlooss1

LGTM 👍

mongoose-im · 2023-02-28T09:28:00Z

small_tests_24 / small_tests / 4139c9d
Reports root / small

small_tests_25 / small_tests / 4139c9d
Reports root / small

ldap_mnesia_24 / ldap_mnesia / 4139c9d
Reports root/ big
OK: 2219 / Failed: 0 / User-skipped: 825 / Auto-skipped: 0

dynamic_domains_pgsql_mnesia_24 / pgsql_mnesia / 4139c9d
Reports root/ big
OK: 4163 / Failed: 1 / User-skipped: 88 / Auto-skipped: 5

bosh_SUITE:essential_https:accept_higher_hold_value

{error,
  {{assertEqual,
     [{module,bosh_SUITE},
      {line,251},
      {expression,"get_bosh_sessions ( )"},
      {expected,[]},
      {value,
        [{bosh_session,<<"3e4dc42eeaf8b1c22b14c2daa68f9378b48535d2">>,
           <8839.10055.0>}]}]},
   [{bosh_SUITE,accept_higher_hold_value,1,
      [{file,"/home/circleci/project/big_tests/tests/bosh_SUITE.erl"},
       {line,251}]},
    {test_server,ts_tc,3,[{file,"test_server.erl"},{line,1783}]},
    {test_server,run_test_case_eval1,6,
      [{file,"test_server.erl"},{line,1292}]},
    {test_server,run_test_case_eval,9,
      [{file,"test_server.erl"},{line,1224}]}]}}

Report log

ldap_mnesia_25 / ldap_mnesia / 4139c9d
Reports root/ big
OK: 2219 / Failed: 0 / User-skipped: 825 / Auto-skipped: 0

dynamic_domains_pgsql_mnesia_25 / pgsql_mnesia / 4139c9d
Reports root/ big
OK: 4169 / Failed: 0 / User-skipped: 88 / Auto-skipped: 0

dynamic_domains_mysql_redis_25 / mysql_redis / 4139c9d
Reports root/ big
OK: 4143 / Failed: 0 / User-skipped: 114 / Auto-skipped: 0

dynamic_domains_mssql_mnesia_25 / odbc_mssql_mnesia / 4139c9d
Reports root/ big
OK: 4161 / Failed: 5 / User-skipped: 91 / Auto-skipped: 0

inbox_extensions_SUITE:regular:one_to_one:pagination:pagination_overrides_form

{error,
  {timeout_when_waiting_for_stanza,
    [{escalus_client,wait_for_stanza,
       [{client,
          <<"bob_pagination_overrides_form_1499@domain.example.com/res1">>,
          escalus_tcp,<0.13215.1>,
          [{event_manager,<0.13180.1>},
           {server,<<"domain.example.com">>},
           {username,<<"bOb_pagination_overrides_form_1499">>},
           {resource,<<"res1">>}],
          [{event_client,
             [{event_manager,<0.13180.1>},
            {server,<<"domain.example.com">>},
            {username,<<"bOb_pagination_overrides_form_1499">>},
            {resource,<<"res1">>}]},
           {resource,<<"res1">>},
           {username,<<"bob_pagination_overrides_form_1499">>},
           {server,<<"domain.example.com">>},
           {host,<<"localhost">>},
           {port,5222},
           {auth,{escalus_auth,auth_plain}},
           {wspath,undefined},
           {username,<<"bOb_pagination_overrides_form_1499">>},
           {server,<<"domain.example.com">>},
           {host,<<"localhost">>},
           {password,<<"makrolika">>},
           {stream_id,<<"8e8d9083f0f69555">>}]},
        5000],
       [{file,
          "/home/circleci/project/big_tests/_build/default/lib/escalus/src/escalus_client.erl"},
        {line,136}]},
     {inbox_helper,'-given_conversations_between/2-fun-1-',4,
       [{file,"/home/circleci/project/big_tests/tests/inbox_helper.erl"},
        {line,543}]},
     {lists,foldl,3,[{file,"lists.erl"},{line,1350}]},
     {inbox_e...

Report log

inbox_extensions_SUITE:regular:one_to_one:pagination:max_queries_can_fetch_ahead

{error,
  {timeout_when_waiting_for_stanza,
    [{escalus_client,wait_for_stanza,
       [{client,
          <<"bob_max_queries_can_fetch_ahead_1501@domain.example.com/res1">>,
          escalus_tcp,<0.13224.1>,
          [{event_manager,<0.13194.1>},
           {server,<<"domain.example.com">>},
           {username,<<"bOb_max_queries_can_fetch_ahead_1501">>},
           {resource,<<"res1">>}],
          [{event_client,
             [{event_manager,<0.13194.1>},
            {server,<<"domain.example.com">>},
            {username,<<"bOb_max_queries_can_fetch_ahead_1501">>},
            {resource,<<"res1">>}]},
           {resource,<<"res1">>},
           {username,<<"bob_max_queries_can_fetch_ahead_1501">>},
           {server,<<"domain.example.com">>},
           {host,<<"localhost">>},
           {port,5222},
           {auth,{escalus_auth,auth_plain}},
           {wspath,undefined},
           {username,<<"bOb_max_queries_can_fetch_ahead_1501">>},
           {server,<<"domain.example.com">>},
           {host,<<"localhost">>},
           {password,<<"makrolika">>},
           {stream_id,<<"009353d8bb7669e8">>}]},
        5000],
       [{file,
          "/home/circleci/project/big_tests/_build/default/lib/escalus/src/escalus_client.erl"},
        {line,136}]},
     {inbox_helper,'-given_conversations_between/2-fun-1-',4,
       [{file,"/home/circleci/project/big_tests/tests/inbox_helper.erl"},
        {line,543}]},
     {lists,foldl,3,[{file,"lists.erl"},{line,1350}]},
   ...

Report log

inbox_extensions_SUITE:regular:one_to_one:pagination:max_queries_can_be_limited

{error,
  {timeout_when_waiting_for_stanza,
    [{escalus_client,wait_for_stanza,
       [{client,
          <<"kate_max_queries_can_be_limited_1503@domain.example.com/res1">>,
          escalus_tcp,<0.13227.1>,
          [{event_manager,<0.13208.1>},
           {server,<<"domain.example.com">>},
           {username,<<"kate_max_queries_can_be_limited_1503">>},
           {resource,<<"res1">>}],
          [{event_client,
             [{event_manager,<0.13208.1>},
            {server,<<"domain.example.com">>},
            {username,<<"kate_max_queries_can_be_limited_1503">>},
            {resource,<<"res1">>}]},
           {resource,<<"res1">>},
           {username,<<"kate_max_queries_can_be_limited_1503">>},
           {server,<<"domain.example.com">>},
           {host,<<"localhost">>},
           {port,5222},
           {auth,{escalus_auth,auth_plain}},
           {wspath,undefined},
           {username,<<"kate_max_queries_can_be_limited_1503">>},
           {server,<<"domain.example.com">>},
           {host,<<"localhost">>},
           {password,<<"makrowe;p">>},
           {stream_id,<<"02af5f0c335c4116">>}]},
        5000],
       [{file,
          "/home/circleci/project/big_tests/_build/default/lib/escalus/src/escalus_client.erl"},
        {line,136}]},
     {inbox_helper,'-given_conversations_between/2-fun-1-',4,
       [{file,"/home/circleci/project/big_tests/tests/inbox_helper.erl"},
        {line,543}]},
     {lists,foldl_1,3,[{file,"lists.erl"},{line,1355}]},
 ...

Report log

inbox_extensions_SUITE:regular:one_to_one:pagination:can_paginate_forwards

{error,
  {timeout_when_waiting_for_stanza,
    [{escalus_client,wait_for_stanza,
       [{client,
          <<"kate_can_paginate_forwards_1498@domain.example.com/res1">>,
          escalus_tcp,<0.13225.1>,
          [{event_manager,<0.13173.1>},
           {server,<<"domain.example.com">>},
           {username,<<"kate_can_paginate_forwards_1498">>},
           {resource,<<"res1">>}],
          [{event_client,
             [{event_manager,<0.13173.1>},
            {server,<<"domain.example.com">>},
            {username,<<"kate_can_paginate_forwards_1498">>},
            {resource,<<"res1">>}]},
           {resource,<<"res1">>},
           {username,<<"kate_can_paginate_forwards_1498">>},
           {server,<<"domain.example.com">>},
           {host,<<"localhost">>},
           {port,5222},
           {auth,{escalus_auth,auth_plain}},
           {wspath,undefined},
           {username,<<"kate_can_paginate_forwards_1498">>},
           {server,<<"domain.example.com">>},
           {host,<<"localhost">>},
           {password,<<"makrowe;p">>},
           {stream_id,<<"5581033d654cd06d">>}]},
        5000],
       [{file,
          "/home/circleci/project/big_tests/_build/default/lib/escalus/src/escalus_client.erl"},
        {line,136}]},
     {inbox_helper,'-given_conversations_between/2-fun-1-',4,
       [{file,"/home/circleci/project/big_tests/tests/inbox_helper.erl"},
        {line,543}]},
     {lists,foldl_1,3,[{file,"lists.erl"},{line,1355}]},
     {inbox_extensions_SUI...

Report log

inbox_extensions_SUITE:regular:one_to_one:pagination:can_paginate_backwards

{error,
  {timeout_when_waiting_for_stanza,
    [{escalus_client,wait_for_stanza,
       [{client,
          <<"kate_can_paginate_backwards_1500@domain.example.com/res1">>,
          escalus_tcp,<0.13228.1>,
          [{event_manager,<0.13187.1>},
           {server,<<"domain.example.com">>},
           {username,<<"kate_can_paginate_backwards_1500">>},
           {resource,<<"res1">>}],
          [{event_client,
             [{event_manager,<0.13187.1>},
            {server,<<"domain.example.com">>},
            {username,<<"kate_can_paginate_backwards_1500">>},
            {resource,<<"res1">>}]},
           {resource,<<"res1">>},
           {username,<<"kate_can_paginate_backwards_1500">>},
           {server,<<"domain.example.com">>},
           {host,<<"localhost">>},
           {port,5222},
           {auth,{escalus_auth,auth_plain}},
           {wspath,undefined},
           {username,<<"kate_can_paginate_backwards_1500">>},
           {server,<<"domain.example.com">>},
           {host,<<"localhost">>},
           {password,<<"makrowe;p">>},
           {stream_id,<<"8b317060f6dcd3a9">>}]},
        5000],
       [{file,
          "/home/circleci/project/big_tests/_build/default/lib/escalus/src/escalus_client.erl"},
        {line,136}]},
     {inbox_helper,'-given_conversations_between/2-fun-1-',4,
       [{file,"/home/circleci/project/big_tests/tests/inbox_helper.erl"},
        {line,543}]},
     {lists,foldl_1,3,[{file,"lists.erl"},{line,1355}]},
     {inbox_extension...

Report log

internal_mnesia_25 / internal_mnesia / 4139c9d
Reports root/ big
OK: 2361 / Failed: 0 / User-skipped: 683 / Auto-skipped: 0

pgsql_mnesia_24 / pgsql_mnesia / 4139c9d
Reports root/ big
OK: 4549 / Failed: 0 / User-skipped: 97 / Auto-skipped: 0

riak_mnesia_24 / riak_mnesia / 4139c9d
Reports root/ big
OK: 2577 / Failed: 1 / User-skipped: 654 / Auto-skipped: 0

pubsub_SUITE:dag+basic:publish_only_retract_items_scope_test

{error,{{badmatch,false},
    [{pubsub_tools,check_response,2,
             [{file,"/home/circleci/project/big_tests/tests/pubsub_tools.erl"},
            {line,491}]},
     {pubsub_tools,receive_response,3,
             [{file,"/home/circleci/project/big_tests/tests/pubsub_tools.erl"},
            {line,481}]},
     {pubsub_tools,receive_and_check_response,4,
             [{file,"/home/circleci/project/big_tests/tests/pubsub_tools.erl"},
            {line,471}]},
     {pubsub_SUITE,'-publish_only_retract_items_scope_test/1-fun-0-',2,
             [{file,"/home/circleci/project/big_tests/tests/pubsub_SUITE.erl"},
            {line,723}]},
     {escalus_story,story,4,
            [{file,"/home/circleci/project/big_tests/_build/default/lib/escalus/src/escalus_story.erl"},
             {line,72}]},
     {test_server,ts_tc,3,[{file,"test_server.erl"},{line,1783}]},
     {test_server,run_test_case_eval1,6,
            [{file,"test_server.erl"},{line,1292}]},
     {test_server,run_test_case_eval,9,
            [{file,"test_server.erl"},{line,1224}]}]}}

Report log

pgsql_mnesia_25 / pgsql_mnesia / 4139c9d
Reports root/ big
OK: 4548 / Failed: 1 / User-skipped: 97 / Auto-skipped: 0

push_integration_SUITE:pubsub_less:pm_notifications_with_inbox:inbox_msg_reset_unread_count_fcm

{error,
  {{assertion_failed,assert_many,false,
     [is_presence,is_message,is_message],
     [{xmlel,<<"message">>,
        [{<<"from">>,
        <<"alice_inbox_msg_reset_unread_count_fcm_2982@localhost/res1">>},
         {<<"to">>,
        <<"bob_inbox_msg_reset_unread_count_fcm_2982@localhost">>},
         {<<"type">>,<<"chat">>},
         {<<"id">>,<<"677b4cfe3ba72d1f63f0047599304f5d">>}],
        [{xmlel,<<"body">>,[],[{xmlcdata,<<"FIRST MESSAGE">>}]},
         {xmlel,<<"delay">>,
           [{<<"xmlns">>,<<"urn:xmpp:delay">>},
          {<<"stamp">>,<<"2023-02-28T09:38:53.946378Z">>},
          {<<"from">>,<<"localhost">>}],
           [{xmlcdata,<<"Offline Storage">>}]}]},
      {xmlel,<<"presence">>,
        [{<<"from">>,
        <<"bob_inbox_msg_reset_unread_count_fcm_2982@localhost/res1">>},
         {<<"to">>,
        <<"bob_inbox_msg_reset_unread_count_fcm_2982@localhost/res1">>}],
        []}],
     "   <message from='alice_inbox_msg_reset_unread_count_fcm_2982@localhost/res1' to='bob_inbox_msg_reset_unread_count_fcm_2982@localhost' type='chat' id='677b4cfe3ba72d1f63f0047599304f5d'><body>FIRST MESSAGE</body><delay xmlns='urn:xmpp:delay' stamp='2023-02-28T09:38:53.946378Z' from='localhost'>Offline Storage</delay></message>   <presence from='bob_inbox_msg_reset_unread_count_fcm_2982@localhost/res1' to='bob_inbox_msg_reset_unread_count_fcm_2982@localhost/res1'/>"},
   [{escalus_new_assert,assert_true,2,
      [{file,
         "/home/circleci/project/big_tests/_buil...

Report log

mysql_redis_25 / mysql_redis / 4139c9d
Reports root/ big
OK: 4535 / Failed: 0 / User-skipped: 111 / Auto-skipped: 0

mssql_mnesia_25 / odbc_mssql_mnesia / 4139c9d
Reports root/ big
OK: 4546 / Failed: 0 / User-skipped: 100 / Auto-skipped: 0

dynamic_domains_pgsql_mnesia_24 / pgsql_mnesia / 4139c9d
Reports root/ big
OK: 4169 / Failed: 0 / User-skipped: 88 / Auto-skipped: 0

dynamic_domains_mssql_mnesia_25 / odbc_mssql_mnesia / 4139c9d
Reports root/ big
OK: 4163 / Failed: 3 / User-skipped: 91 / Auto-skipped: 0

graphql_server_SUITE:admin_http:clustering_http_tests:remove_node_test

{error,{{badmatch,{error,econnrefused}},
    [{rest_helper,fusco_request,7,
            [{file,"/home/circleci/project/big_tests/tests/rest_helper.erl"},
             {line,185}]},
     {rest_helper,make_request,1,
            [{file,"/home/circleci/project/big_tests/tests/rest_helper.erl"},
             {line,114}]},
     {rest_helper,make_request,1,
            [{file,"/home/circleci/project/big_tests/tests/rest_helper.erl"},
             {line,121}]},
     {graphql_server_SUITE,remove_node_test,1,
                 [{file,"/home/circleci/project/big_tests/tests/graphql_server_SUITE.erl"},
                {line,210}]},
     {test_server,ts_tc,3,[{file,"test_server.erl"},{line,1782}]},
     {test_server,run_test_case_eval1,6,
            [{file,"test_server.erl"},{line,1291}]},
     {test_server,run_test_case_eval,9,
            [{file,"test_server.erl"},{line,1223}]}]}}

Report log

graphql_server_SUITE:admin_http:clustering_http_tests:stop_node_test

{error,{{badmatch,{error,econnrefused}},
    [{rest_helper,fusco_request,7,
            [{file,"/home/circleci/project/big_tests/tests/rest_helper.erl"},
             {line,185}]},
     {rest_helper,make_request,1,
            [{file,"/home/circleci/project/big_tests/tests/rest_helper.erl"},
             {line,114}]},
     {rest_helper,make_request,1,
            [{file,"/home/circleci/project/big_tests/tests/rest_helper.erl"},
             {line,121}]},
     {graphql_server_SUITE,stop_node_test,1,
                 [{file,"/home/circleci/project/big_tests/tests/graphql_server_SUITE.erl"},
                {line,215}]},
     {test_server,ts_tc,3,[{file,"test_server.erl"},{line,1782}]},
     {test_server,run_test_case_eval1,6,
            [{file,"test_server.erl"},{line,1291}]},
     {test_server,run_test_case_eval,9,
            [{file,"test_server.erl"},{line,1223}]}]}}

Report log

muc_SUITE:register:user_submits_registration_form

{error,
  {{assertion_failed,assert,is_iq_result,
     [{xmlel,<<"iq">>,
        [{<<"type">>,<<"set">>},
         {<<"id">>,<<"8590eb0afcaa050f5f202105f4d2a0bb">>},
         {<<"to">>,<<"groupchats.domain.example.com">>}],
        [{xmlel,<<"query">>,
           [{<<"xmlns">>,<<"jabber:iq:register">>}],
           [{xmlel,<<"x">>,
            [{<<"xmlns">>,<<"jabber:x:data">>},
             {<<"type">>,<<"submit">>}],
            [{xmlel,<<"field">>,
               [{<<"type">>,<<"hidden">>},
                {<<"var">>,<<"FORM_TYPE">>}],
               [{xmlel,<<"value">>,[],
                  [{xmlcdata,<<"jabber:iq:register">>}]}]},
             {xmlel,<<"field">>,
               [{<<"type">>,<<"text-single">>},
                {<<"var">>,<<"nick">>}],
               [{xmlel,<<"value">>,[],
                  [{xmlcdata,
                     <<"thirdwitchroom-92ad6cb0e2">>}]}]}]}]}]}],
     {xmlel,<<"iq">>,
       [{<<"from">>,<<"groupchats.domain.example.com">>},
        {<<"to">>,
         <<"alice_user_submits_registration_form_2462@domain.example.com/res1">>},
        {<<"type">>,<<"error">>},
        {<<"id">>,<<"8590eb0afcaa050f5f202105f4d2a0bb">>}],
       [{xmlel,<<"query">>,
          [{<<"xmlns">>,<<"jabber:iq:register">>}],
          [{xmlel,<<"x">>,
             [{<<"xmlns">>,<<"jabber:x:data">>},
            {<<"type">>,<<"submit">>}],
             [{xmlel,<<"field">>,
              [{<<"type">>,<<"hidden">>},
               {<<"var">>,<<"FORM_TYPE">>}],
      ...

Report log

pgsql_mnesia_25 / pgsql_mnesia / 4139c9d
Reports root/ big
OK: 4549 / Failed: 0 / User-skipped: 97 / Auto-skipped: 0

dynamic_domains_mssql_mnesia_25 / odbc_mssql_mnesia / 4139c9d
Reports root/ big
OK: 4166 / Failed: 0 / User-skipped: 91 / Auto-skipped: 0

NelsonVides

Ok, seems good, convoluted usual retry logic, I'm fine with it. Just some wrong indentation going around 😇

NelsonVides · 2023-02-28T10:15:06Z

src/async_pools/mongoose_aggregator_worker.erl

+        ignore ->
+            {noreply, State};
+         next ->
+            {noreply, maybe_request_next(State)};
+         retry ->
+            {noreply, maybe_request_retry(ReqTask, State)}


Indentation is off here

mongoose-im · 2023-02-28T10:51:57Z

small_tests_24 / small_tests / 110d8f9
Reports root / small

small_tests_25 / small_tests / 110d8f9
Reports root / small

dynamic_domains_pgsql_mnesia_24 / pgsql_mnesia / 110d8f9
Reports root/ big
OK: 4169 / Failed: 0 / User-skipped: 88 / Auto-skipped: 0

ldap_mnesia_24 / ldap_mnesia / 110d8f9
Reports root/ big
OK: 2219 / Failed: 0 / User-skipped: 825 / Auto-skipped: 0

ldap_mnesia_25 / ldap_mnesia / 110d8f9
Reports root/ big
OK: 2219 / Failed: 0 / User-skipped: 825 / Auto-skipped: 0

dynamic_domains_mysql_redis_25 / mysql_redis / 110d8f9
Reports root/ big
OK: 4143 / Failed: 0 / User-skipped: 114 / Auto-skipped: 0

dynamic_domains_pgsql_mnesia_25 / pgsql_mnesia / 110d8f9
Reports root/ big
OK: 4169 / Failed: 0 / User-skipped: 88 / Auto-skipped: 0

dynamic_domains_mssql_mnesia_25 / odbc_mssql_mnesia / 110d8f9
Reports root/ big
OK: 4166 / Failed: 0 / User-skipped: 91 / Auto-skipped: 0

pgsql_mnesia_24 / pgsql_mnesia / 110d8f9
Reports root/ big
OK: 4549 / Failed: 0 / User-skipped: 97 / Auto-skipped: 0

internal_mnesia_25 / internal_mnesia / 110d8f9
Reports root/ big
OK: 2361 / Failed: 0 / User-skipped: 683 / Auto-skipped: 0

elasticsearch_and_cassandra_25 / elasticsearch_and_cassandra_mnesia / 110d8f9
Reports root/ big
OK: 2721 / Failed: 0 / User-skipped: 662 / Auto-skipped: 0

mysql_redis_25 / mysql_redis / 110d8f9
Reports root/ big
OK: 4535 / Failed: 0 / User-skipped: 111 / Auto-skipped: 0

riak_mnesia_24 / riak_mnesia / 110d8f9
Reports root/ big
OK: 2559 / Failed: 0 / User-skipped: 654 / Auto-skipped: 0

pgsql_mnesia_25 / pgsql_mnesia / 110d8f9
Reports root/ big
OK: 4549 / Failed: 0 / User-skipped: 97 / Auto-skipped: 0

mssql_mnesia_25 / odbc_mssql_mnesia / 110d8f9
Reports root/ big
OK: 4546 / Failed: 0 / User-skipped: 100 / Auto-skipped: 0

Add retry logic into mongooseim task aggregator

ac5a21c

arcusfelis marked this pull request as draft February 20, 2023 18:03

This comment was marked as outdated.

Sign in to view

Add batches_SUITE:retry_request_cancelled testcase

19f8dfd

This comment was marked as outdated.

Sign in to view

Add batches_SUITE:ignore_msg_when_waiting_for_reply testcase to impro…

74b047e

…ve coverage

This comment was marked as outdated.

Sign in to view

arcusfelis marked this pull request as ready for review February 26, 2023 18:27

arcusfelis requested a review from NelsonVides February 26, 2023 22:13

arcusfelis added the waiting-for-review label Feb 26, 2023

pawlooss1 reviewed Feb 27, 2023

View reviewed changes

arcusfelis added 3 commits February 27, 2023 16:50

Refactor maybe_request_next function

791dc30

Move new test cases right bellow the other ones and new helpers to ot…

b873355

…her helpers similarly

Add ignore_msg_when_waiting_for_reply testcase

08a9a33

Add retry_request_cancelled_in_verify_function testcase

4139c9d

pawlooss1 approved these changes Feb 28, 2023

View reviewed changes

NelsonVides reviewed Feb 28, 2023

View reviewed changes

Fix indention

110d8f9

pawlooss1 merged commit 4a97962 into master Feb 28, 2023

pawlooss1 deleted the mu-retry-aggr-tasks branch February 28, 2023 11:32

jacekwegr added this to the 6.1.0 milestone Apr 26, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add retry logic into mongooseim task aggregator #3969

Add retry logic into mongooseim task aggregator #3969

arcusfelis commented Feb 20, 2023 •

edited

This comment was marked as outdated.

codecov bot commented Feb 20, 2023 •

edited

This comment was marked as outdated.

This comment was marked as outdated.

pawlooss1 left a comment

pawlooss1 Feb 27, 2023

arcusfelis Feb 27, 2023

arcusfelis commented Feb 27, 2023

mongoose-im commented Feb 27, 2023 •

edited

pawlooss1 left a comment

mongoose-im commented Feb 28, 2023 •

edited

NelsonVides left a comment

NelsonVides Feb 28, 2023

mongoose-im commented Feb 28, 2023 •

edited

Add retry logic into mongooseim task aggregator #3969

Add retry logic into mongooseim task aggregator #3969

Conversation

arcusfelis commented Feb 20, 2023 • edited

This comment was marked as outdated.

codecov bot commented Feb 20, 2023 • edited

Codecov Report

This comment was marked as outdated.

This comment was marked as outdated.

pawlooss1 left a comment

Choose a reason for hiding this comment

pawlooss1 Feb 27, 2023

Choose a reason for hiding this comment

arcusfelis Feb 27, 2023

Choose a reason for hiding this comment

arcusfelis commented Feb 27, 2023

mongoose-im commented Feb 27, 2023 • edited

pawlooss1 left a comment

Choose a reason for hiding this comment

mongoose-im commented Feb 28, 2023 • edited

NelsonVides left a comment

Choose a reason for hiding this comment

NelsonVides Feb 28, 2023

Choose a reason for hiding this comment

mongoose-im commented Feb 28, 2023 • edited

arcusfelis commented Feb 20, 2023 •

edited

codecov bot commented Feb 20, 2023 •

edited

mongoose-im commented Feb 27, 2023 •

edited

mongoose-im commented Feb 28, 2023 •

edited

mongoose-im commented Feb 28, 2023 •

edited