CETS backend for mod_bosh #4050

arcusfelis · 2023-06-30T18:21:57Z

This PR addresses MIM-1943.
We want to add CETS backend for mod_bosh.
There was already existing backend support in this module, so this PR is small.

Proposed changes include:

Change SIDs generation to make them actually random. make_ref() UIDs are not cryptographically random.
backend module for CETS.

mongoose-im · 2023-06-30T18:31:57Z

small_tests_24 / small_tests / 57a29b5
Reports root / small

small_tests_25 / small_tests / 57a29b5
Reports root / small

small_tests_25_arm64 / small_tests / 57a29b5
Reports root / small

ldap_mnesia_24 / ldap_mnesia / 57a29b5
Reports root/ big
OK: 2224 / Failed: 0 / User-skipped: 838 / Auto-skipped: 0

dynamic_domains_pgsql_mnesia_24 / pgsql_mnesia / 57a29b5
Reports root/ big
OK: 4198 / Failed: 0 / User-skipped: 90 / Auto-skipped: 0

ldap_mnesia_25 / ldap_mnesia / 57a29b5
Reports root/ big
OK: 2224 / Failed: 0 / User-skipped: 838 / Auto-skipped: 0

dynamic_domains_mysql_redis_25 / mysql_redis / 57a29b5
Reports root/ big
OK: 4172 / Failed: 0 / User-skipped: 116 / Auto-skipped: 0

dynamic_domains_pgsql_mnesia_25 / pgsql_mnesia / 57a29b5
Reports root/ big
OK: 4198 / Failed: 0 / User-skipped: 90 / Auto-skipped: 0

dynamic_domains_mssql_mnesia_25 / odbc_mssql_mnesia / 57a29b5
Reports root/ big
OK: 4195 / Failed: 0 / User-skipped: 93 / Auto-skipped: 0

pgsql_mnesia_24 / pgsql_mnesia / 57a29b5
Reports root/ big
OK: 4571 / Failed: 0 / User-skipped: 99 / Auto-skipped: 0

internal_mnesia_25 / internal_mnesia / 57a29b5
Reports root/ big
OK: 2370 / Failed: 0 / User-skipped: 692 / Auto-skipped: 0

pgsql_mnesia_25 / pgsql_mnesia / 57a29b5
Reports root/ big
OK: 4571 / Failed: 0 / User-skipped: 99 / Auto-skipped: 0

elasticsearch_and_cassandra_25 / elasticsearch_and_cassandra_mnesia / 57a29b5
Reports root/ big
OK: 2719 / Failed: 4 / User-skipped: 524 / Auto-skipped: 154

graphql_server_SUITE:admin_cli:clustering_tests:remove_alive_from_cluster

{error,{{badrpc,nodedown},
    [{distributed_helper,rpc,
               [#{node => mongooseim3@localhost,
                timeout => 60000},
                mongoose_cluster,join,
                [mongooseim@localhost]],
               [{file,"/home/circleci/project/big_tests/tests/distributed_helper.erl"},
                {line,121}]},
     {graphql_server_SUITE,remove_alive_from_cluster,1,
                 [{file,"/home/circleci/project/big_tests/tests/graphql_server_SUITE.erl"},
                {line,197}]},
     {test_server,ts_tc,3,[{file,"test_server.erl"},{line,1782}]},
     {test_server,run_test_case_eval1,6,
            [{file,"test_server.erl"},{line,1291}]},
     {test_server,run_test_case_eval,9,
            [{file,"test_server.erl"},{line,1223}]}]}}

Report log

graphql_server_SUITE:admin_cli:clustering_tests:remove_node_test

{error,{#{expected_type => ok,
      response_code => {exit_status,3},
      what => invalid_response_code},
    [{graphql_helper,assert_response_code,2,
             [{file,"/home/circleci/project/big_tests/tests/graphql_helper.erl"},
              {line,256}]},
     {graphql_helper,get_ok_value,2,
             [{file,"/home/circleci/project/big_tests/tests/graphql_helper.erl"},
              {line,239}]},
     {graphql_server_SUITE,remove_node_test,1,
                 [{file,"/home/circleci/project/big_tests/tests/graphql_server_SUITE.erl"},
                {line,208}]},
     {test_server,ts_tc,3,[{file,"test_server.erl"},{line,1782}]},
     {test_server,run_test_case_eval1,6,
            [{file,"test_server.erl"},{line,1291}]},
     {test_server,run_test_case_eval,9,
            [{file,"test_server.erl"},{line,1223}]}]}}

Report log

graphql_server_SUITE:admin_cli:clustering_tests:stop_node_test

{error,{#{expected_type => ok,
      response_code => {exit_status,3},
      what => invalid_response_code},
    [{graphql_helper,assert_response_code,2,
             [{file,"/home/circleci/project/big_tests/tests/graphql_helper.erl"},
              {line,256}]},
     {graphql_helper,get_ok_value,2,
             [{file,"/home/circleci/project/big_tests/tests/graphql_helper.erl"},
              {line,239}]},
     {graphql_server_SUITE,stop_node_test,1,
                 [{file,"/home/circleci/project/big_tests/tests/graphql_server_SUITE.erl"},
                {line,213}]},
     {test_server,ts_tc,3,[{file,"test_server.erl"},{line,1782}]},
     {test_server,run_test_case_eval1,6,
            [{file,"test_server.erl"},{line,1291}]},
     {test_server,run_test_case_eval,9,
            [{file,"test_server.erl"},{line,1223}]}]}}

Report log

service_mongoose_system_metrics_SUITE:log_transparency:just_removed_from_config_logs_question

{error,
  {{badrpc,nodedown},
   [{distributed_helper,rpc,
      [#{node => mongooseim3@localhost},
       mongoose_service,ensure_stopped,
       [service_mongoose_system_metrics]],
      [{file,
         "/home/circleci/project/big_tests/tests/distributed_helper.erl"},
       {line,121}]},
    {service_mongoose_system_metrics_SUITE,disable_system_metrics,1,
      [{file,
         "/home/circleci/project/big_tests/tests/service_mongoose_system_metrics_SUITE.erl"},
       {line,434}]},
    {service_mongoose_system_metrics_SUITE,
      just_removed_from_config_logs_question,1,
      [{file,
         "/home/circleci/project/big_tests/tests/service_mongoose_system_metrics_SUITE.erl"},
       {line,281}]},
    {test_server,ts_tc,3,[{file,"test_server.erl"},{line,1782}]},
    {test_server,run_test_case_eval1,6,
      [{file,"test_server.erl"},{line,1291}]},
    {test_server,run_test_case_eval,9,
      [{file,"test_server.erl"},{line,1223}]}]}}

Report log

service_domain_db_SUITE:init_per_suite

{'EXIT',
  {{badrpc,nodedown},
   [{distributed_helper,rpc,
      [#{node => mongooseim3@localhost},
       mongoose_service,loaded_services_with_opts,[]],
      [{file,
         "/home/circleci/project/big_tests/tests/distributed_helper.erl"},
       {line,121}]},
    {dynamic_services,save_services,2,
      [{file,
         "/home/circleci/project/big_tests/tests/dynamic_services.erl"},
       {line,15}]},
    {lists,foldl_1,3,[{file,"lists.erl"},{line,1355}]},
    {service_domain_db_SUITE,init_per_suite,1,
      [{file,
         "/home/circleci/project/big_tests/tests/service_domain_db_SUITE.erl"},
       {line,176}]},
    {test_server,ts_tc,3,[{file,"test_server.erl"},{line,1782}]},
    {test_server,run_test_case_eval1,6,
      [{file,"test_server.erl"},{line,1379}]},
    {test_server,run_test_case_eval,9,
      [{file,"test_server.erl"},{line,1223}]}]}}

Report log

mysql_redis_25 / mysql_redis / 57a29b5
Reports root/ big
OK: 4557 / Failed: 0 / User-skipped: 113 / Auto-skipped: 0

pgsql_cets_25 / pgsql_cets / 57a29b5
Reports root/ big
OK: 4547 / Failed: 0 / User-skipped: 123 / Auto-skipped: 0

mssql_mnesia_25 / odbc_mssql_mnesia / 57a29b5
Reports root/ big
OK: 4568 / Failed: 0 / User-skipped: 102 / Auto-skipped: 0

elasticsearch_and_cassandra_25 / elasticsearch_and_cassandra_mnesia / 57a29b5
Reports root/ big
OK: 2730 / Failed: 0 / User-skipped: 671 / Auto-skipped: 0

codecov · 2023-06-30T18:34:44Z

Codecov Report

Patch coverage: 92.30% and project coverage change: -0.01 ⚠️

Comparison is base (b83dc98) 82.21% compared to head (72975d1) 82.21%.

Additional details and impacted files

@@               Coverage Diff                @@
##           feature/cets    #4050      +/-   ##
================================================
- Coverage         82.21%   82.21%   -0.01%     
================================================
  Files               541      542       +1     
  Lines             34012    34023      +11     
================================================
+ Hits              27962    27971       +9     
- Misses             6050     6052       +2

Impacted Files	Coverage Δ
src/mod_bosh.erl	`95.74% <50.00%> (ø)`
src/mod_bosh_cets.erl	`100.00% <100.00%> (ø)`

... and 9 files with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Do you have feedback about the report comment? Let us know in this issue.

chrzaszcz

Looks good, I added one minor question.

src/mod_bosh_cets.erl

mongoose-im · 2023-07-10T11:16:35Z

small_tests_25_arm64 / small_tests / 72975d1
Reports root / small

small_tests_24 / small_tests / 72975d1
Reports root / small

small_tests_25 / small_tests / 72975d1
Reports root / small

ldap_mnesia_24 / ldap_mnesia / 72975d1
Reports root/ big
OK: 2224 / Failed: 0 / User-skipped: 838 / Auto-skipped: 0

dynamic_domains_pgsql_mnesia_24 / pgsql_mnesia / 72975d1
Reports root/ big
OK: 4198 / Failed: 0 / User-skipped: 90 / Auto-skipped: 0

dynamic_domains_pgsql_mnesia_25 / pgsql_mnesia / 72975d1
Reports root/ big
OK: 4192 / Failed: 1 / User-skipped: 90 / Auto-skipped: 5

bosh_SUITE:essential:accept_higher_hold_value

{error,
  {{assertEqual,
     [{module,bosh_SUITE},
      {line,260},
      {expression,"get_bosh_sessions ( )"},
      {expected,[]},
      {value,
        [{bosh_session,<<"0fbc94b8ffe2d9615fc98beb413afb3293e36517">>,
           <9316.10104.0>}]}]},
   [{bosh_SUITE,accept_higher_hold_value,1,
      [{file,"/home/circleci/project/big_tests/tests/bosh_SUITE.erl"},
       {line,260}]},
    {test_server,ts_tc,3,[{file,"test_server.erl"},{line,1782}]},
    {test_server,run_test_case_eval1,6,
      [{file,"test_server.erl"},{line,1291}]},
    {test_server,run_test_case_eval,9,
      [{file,"test_server.erl"},{line,1223}]}]}}

Report log

dynamic_domains_mysql_redis_25 / mysql_redis / 72975d1
Reports root/ big
OK: 4172 / Failed: 0 / User-skipped: 116 / Auto-skipped: 0

ldap_mnesia_25 / ldap_mnesia / 72975d1
Reports root/ big
OK: 2224 / Failed: 0 / User-skipped: 838 / Auto-skipped: 0

pgsql_mnesia_24 / pgsql_mnesia / 72975d1
Reports root/ big
OK: 4571 / Failed: 0 / User-skipped: 99 / Auto-skipped: 0

elasticsearch_and_cassandra_25 / elasticsearch_and_cassandra_mnesia / 72975d1
Reports root/ big
OK: 2730 / Failed: 0 / User-skipped: 671 / Auto-skipped: 0

dynamic_domains_mssql_mnesia_25 / odbc_mssql_mnesia / 72975d1
Reports root/ big
OK: 4195 / Failed: 0 / User-skipped: 93 / Auto-skipped: 0

pgsql_cets_25 / pgsql_cets / 72975d1
Reports root/ big
OK: 4547 / Failed: 0 / User-skipped: 123 / Auto-skipped: 0

internal_mnesia_25 / internal_mnesia / 72975d1
Reports root/ big
OK: 2369 / Failed: 1 / User-skipped: 692 / Auto-skipped: 0

mod_global_distrib_SUITE:hosts_refresher:test_host_refreshing

{error,
  {{trees_for_connections_present,true,[{times,50,false}],ok},
   [{mongoose_helper,do_wait_until,2,
      [{file,"/home/circleci/project/big_tests/tests/mongoose_helper.erl"},
       {line,371}]},
    {mod_global_distrib_SUITE,test_host_refreshing,1,
      [{file,
         "/home/circleci/project/big_tests/tests/mod_global_distrib_SUITE.erl"},
       {line,384}]},
    {test_server,ts_tc,3,[{file,"test_server.erl"},{line,1782}]},
    {test_server,run_test_case_eval1,6,
      [{file,"test_server.erl"},{line,1291}]},
    {test_server,run_test_case_eval,9,
      [{file,"test_server.erl"},{line,1223}]}]}}

Report log

pgsql_mnesia_25 / pgsql_mnesia / 72975d1
Reports root/ big
OK: 4571 / Failed: 0 / User-skipped: 99 / Auto-skipped: 0

mysql_redis_25 / mysql_redis / 72975d1
Reports root/ big
OK: 4557 / Failed: 0 / User-skipped: 113 / Auto-skipped: 0

mssql_mnesia_25 / odbc_mssql_mnesia / 72975d1
Reports root/ big
OK: 4568 / Failed: 0 / User-skipped: 102 / Auto-skipped: 0

dynamic_domains_pgsql_mnesia_25 / pgsql_mnesia / 72975d1
Reports root/ big
OK: 4198 / Failed: 0 / User-skipped: 90 / Auto-skipped: 0

internal_mnesia_25 / internal_mnesia / 72975d1
Reports root/ big
OK: 2370 / Failed: 0 / User-skipped: 692 / Auto-skipped: 0

chrzaszcz

👍

Add mod_bosh_cets

57a29b5

arcusfelis added waiting-for-review and removed waiting-for-review labels Jun 30, 2023

chrzaszcz approved these changes Jul 7, 2023

View reviewed changes

src/mod_bosh_cets.erl Outdated Show resolved Hide resolved

arcusfelis added 2 commits July 10, 2023 12:52

Update docs

dee22c4

Use records in mod_bosh_cets

72975d1

chrzaszcz approved these changes Jul 10, 2023

View reviewed changes

chrzaszcz merged commit 1cc81f6 into feature/cets Jul 10, 2023
4 checks passed

chrzaszcz deleted the mu-cets-bosh branch July 10, 2023 12:12

chrzaszcz added this to the 6.2.0 milestone Dec 11, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

CETS backend for mod_bosh #4050

CETS backend for mod_bosh #4050

arcusfelis commented Jun 30, 2023 •

edited

mongoose-im commented Jun 30, 2023 •

edited

codecov bot commented Jun 30, 2023 •

edited

chrzaszcz left a comment

mongoose-im commented Jul 10, 2023 •

edited

chrzaszcz left a comment

CETS backend for mod_bosh #4050

CETS backend for mod_bosh #4050

Conversation

arcusfelis commented Jun 30, 2023 • edited

mongoose-im commented Jun 30, 2023 • edited

codecov bot commented Jun 30, 2023 • edited

Codecov Report

chrzaszcz left a comment

Choose a reason for hiding this comment

mongoose-im commented Jul 10, 2023 • edited

chrzaszcz left a comment

Choose a reason for hiding this comment

arcusfelis commented Jun 30, 2023 •

edited

mongoose-im commented Jun 30, 2023 •

edited

codecov bot commented Jun 30, 2023 •

edited

mongoose-im commented Jul 10, 2023 •

edited