Change domain validation logic #4184

jacekwegr · 2023-12-04T12:08:43Z

This PR changes subdomain validation process. While the validation of subdomain patterns continues to use 'example.com' for creating FQDNs, this update removes the step of attempting to resolve these FQDNs. Additionally, the domains are now validated using inet_parse:domain, followed by a resolution attempt.

This PR also moves certain test helpers, relocating them to the test/common directory. This change was made to enable the use of logger_ct_backend in small tests.

codecov · 2023-12-04T12:16:38Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (c5ff950) 84.28% compared to head (e0d1fd1) 84.29%.

Additional details and impacted files

@@           Coverage Diff           @@
##           master    #4184   +/-   ##
=======================================
  Coverage   84.28%   84.29%           
=======================================
  Files         551      551           
  Lines       33467    33471    +4     
=======================================
+ Hits        28208    28213    +5     
+ Misses       5259     5258    -1

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

chrzaszcz

Looks good 👍

test/config_parser_SUITE.erl

chrzaszcz

Looks good, I have one tiny comment.

test/config_parser_SUITE.erl

- Changed domain validation logic - Moved some helpers to `test/common` to use ct:logger_backend in small tests

mongoose-im · 2023-12-06T08:01:39Z

elasticsearch_and_cassandra_26 / elasticsearch_and_cassandra_mnesia / e0d1fd1
Reports root/ big
OK: 368 / Failed: 0 / User-skipped: 38 / Auto-skipped: 0

small_tests_26 / small_tests / e0d1fd1
Reports root / small

small_tests_25 / small_tests / e0d1fd1
Reports root / small

small_tests_26_arm64 / small_tests / e0d1fd1
Reports root / small

dynamic_domains_mysql_redis_26 / mysql_redis / e0d1fd1
Reports root/ big
OK: 4191 / Failed: 1 / User-skipped: 135 / Auto-skipped: 0

sm_SUITE:parallel_unacknowledged_message_hook:unacknowledged_message_hook_filter

{error,{{badmatch,false},
    [{escalus_session,stream_resumption,2,
              [{file,"/home/circleci/project/big_tests/_build/default/lib/escalus/src/escalus_session.erl"},
               {line,259}]},
     {escalus_connection,connection_step,2,
               [{file,"/home/circleci/project/big_tests/_build/default/lib/escalus/src/escalus_connection.erl"},
                {line,161}]},
     {lists,foldl_1,3,[{file,"lists.erl"},{line,1599}]},
     {escalus_connection,start,2,
               [{file,"/home/circleci/project/big_tests/_build/default/lib/escalus/src/escalus_connection.erl"},
                {line,145}]},
     {sm_helper,connect_spec,3,
          [{file,"/home/circleci/project/big_tests/tests/sm_helper.erl"},
           {line,153}]},
     {sm_SUITE,unacknowledged_message_hook_filter,1,
           [{file,"/home/circleci/project/big_tests/tests/sm_SUITE.erl"},
          {line,802}]},
     {test_server,ts_tc,3,[{file,"test_server.erl"},{line,1793}]},
     {test_server,run_test_case_eval1,6,
            [{file,"test_server.erl"},{line,1302}]}]}}

Report log

dynamic_domains_pgsql_mnesia_25 / pgsql_mnesia / e0d1fd1
Reports root/ big
OK: 4225 / Failed: 0 / User-skipped: 102 / Auto-skipped: 0

ldap_mnesia_25 / ldap_mnesia / e0d1fd1
Reports root/ big
OK: 2268 / Failed: 0 / User-skipped: 847 / Auto-skipped: 0

ldap_mnesia_26 / ldap_mnesia / e0d1fd1
Reports root/ big
OK: 2268 / Failed: 0 / User-skipped: 847 / Auto-skipped: 0

dynamic_domains_pgsql_mnesia_26 / pgsql_mnesia / e0d1fd1
Reports root/ big
OK: 4224 / Failed: 1 / User-skipped: 102 / Auto-skipped: 0

sm_SUITE:ping_timeout

{error,{noproc,{gen_server,call,[<0.81131.0>,get_sm_h]}}}

Report log

dynamic_domains_mssql_mnesia_26 / odbc_mssql_mnesia / e0d1fd1
Reports root/ big
OK: 4222 / Failed: 0 / User-skipped: 105 / Auto-skipped: 0

mysql_redis_26 / mysql_redis / e0d1fd1
Reports root/ big
OK: 4593 / Failed: 0 / User-skipped: 130 / Auto-skipped: 0

pgsql_cets_26 / pgsql_cets / e0d1fd1
Reports root/ big
OK: 4273 / Failed: 0 / User-skipped: 110 / Auto-skipped: 0

internal_mnesia_26 / internal_mnesia / e0d1fd1
Reports root/ big
OK: 2408 / Failed: 0 / User-skipped: 707 / Auto-skipped: 0

pgsql_mnesia_25 / pgsql_mnesia / e0d1fd1
Reports root/ big
OK: 4614 / Failed: 0 / User-skipped: 109 / Auto-skipped: 0

mssql_mnesia_26 / odbc_mssql_mnesia / e0d1fd1
Reports root/ big
OK: 4600 / Failed: 11 / User-skipped: 112 / Auto-skipped: 0

amp_big_SUITE:basic:error_deliver_to_offline_user_test

{error,{{badrpc,timeout},
    [{escalus_rpc,call_with_cookie_match,
            [mongooseim@localhost,ejabberd_admin,register,
             [<<"bOb_error_deliver_to_offline_user_test_72_error_deliver_to_offline_user_test_128">>,
            <<"localhost">>,<<"makrolika">>],
             3000,mongooseim],
            [{file,"/home/circleci/project/big_tests/_build/default/lib/escalus/src/escalus_rpc.erl"},
             {line,34}]},
     {lists,foreach_1,2,[{file,"lists.erl"},{line,1686}]},
     {escalus_ejabberd,create_users,2,
               [{file,"/home/circleci/project/big_tests/_build/default/lib/escalus/src/escalus_ejabberd.erl"},
              {line,211}]},
     {escalus_fresh,create_users,2,
            [{file,"/home/circleci/project/big_tests/_build/default/lib/escalus/src/escalus_fresh.erl"},
             {line,62}]},
     {escalus_fresh,story,3,
            [{file,"/home/circleci/project/big_tests/_build/default/lib/escalus/src/escalus_fresh.erl"},
             {line,27}]},
     {test_server,ts_tc,3,[{file,"test_server.erl"},{line,1793}]},
     {test_server,run_test_case_eval1,6,
            [{file,"test_server.erl"},{line,1302}]},
     {test_server,run_test_case_eval,9,
            [{file,"test_server.erl"},{line,1234}]}]}}

Report log

amp_big_SUITE:basic:notify_match_resource_exact_test

{error,{{badmatch,0},
    [{escalus_story,drop_presences,2,
            [{file,"/home/circleci/project/big_tests/_build/default/lib/escalus/src/escalus_story.erl"},
             {line,191}]},
     {escalus_story,'-start_ready_clients/2-fun-0-',3,
            [{file,"/home/circleci/project/big_tests/_build/default/lib/escalus/src/escalus_story.erl"},
             {line,135}]},
     {lists,foldl_1,3,[{file,"lists.erl"},{line,1599}]},
     {escalus_story,start_ready_clients,2,
            [{file,"/home/circleci/project/big_tests/_build/default/lib/escalus/src/escalus_story.erl"},
             {line,128}]},
     {lists,flatmap_1,2,[{file,"lists.erl"},{line,1579}]},
     {lists,flatmap_1,2,[{file,"lists.erl"},{line,1579}]},
     {escalus_story,story,4,
            [{file,"/home/circleci/project/big_tests/_build/default/lib/escalus/src/escalus_story.erl"},
             {line,69}]},
     {test_server,ts_tc,3,[{file,"test_server.erl"},{line,1793}]}]}}

Report log

amp_big_SUITE:basic:notify_deliver_to_offline_user_recipient_privacy_test

{error,
  {timeout_when_waiting_for_stanza,
    [{escalus_client,wait_for_stanza,
       [{client,
          <<"bob_notify_deliver_to_offline_user_recipient_privacy_test_77@localhost/res1">>,
          escalus_tcp,<0.3154.0>,
          [{event_manager,<0.2874.0>},
           {server,<<"localhost">>},
           {username,
             <<"bOb_notify_deliver_to_offline_user_recipient_privacy_test_77">>},
           {resource,<<"res1">>}],
          [{event_client,
             [{event_manager,<0.2874.0>},
            {server,<<"localhost">>},
            {username,
              <<"bOb_notify_deliver_to_offline_user_recipient_privacy_test_77">>},
            {resource,<<"res1">>}]},
           {resource,<<"res1">>},
           {username,
             <<"bob_notify_deliver_to_offline_user_recipient_privacy_test_77">>},
           {server,<<"localhost">>},
           {host,<<"localhost">>},
           {port,5222},
           {auth,{escalus_auth,auth_plain}},
           {wspath,undefined},
           {username,
             <<"bOb_notify_deliver_to_offline_user_recipient_privacy_test_77">>},
           {server,<<"localhost">>},
           {password,<<"makrolika">>},
           {stream_id,<<"e45844d41511c8e4">>}]},
        5000],
       [{file,
          "/home/circleci/project/big_tests/_build/default/lib/escalus/src/escalus_client.erl"},
        {line,136}]},
     {privacy_helper,set_default_list,2,
       [{file,
          "/home/circleci/project/big_tests/tests/privacy_helper.e...

Report log

amp_big_SUITE:basic:notify_deliver_to_offline_user_recipient_privacy_test

{error,
  {timeout_when_waiting_for_stanza,
    [{escalus_client,wait_for_stanza,
       [{client,
          <<"bob_notify_deliver_to_offline_user_recipient_privacy_test_75@localhost/res1">>,
          escalus_tcp,<0.3161.0>,
          [{event_manager,<0.2826.0>},
           {server,<<"localhost">>},
           {username,
             <<"bOb_notify_deliver_to_offline_user_recipient_privacy_test_75">>},
           {resource,<<"res1">>}],
          [{event_client,
             [{event_manager,<0.2826.0>},
            {server,<<"localhost">>},
            {username,
              <<"bOb_notify_deliver_to_offline_user_recipient_privacy_test_75">>},
            {resource,<<"res1">>}]},
           {resource,<<"res1">>},
           {username,
             <<"bob_notify_deliver_to_offline_user_recipient_privacy_test_75">>},
           {server,<<"localhost">>},
           {host,<<"localhost">>},
           {port,5222},
           {auth,{escalus_auth,auth_plain}},
           {wspath,undefined},
           {username,
             <<"bOb_notify_deliver_to_offline_user_recipient_privacy_test_75">>},
           {server,<<"localhost">>},
           {password,<<"makrolika">>},
           {stream_id,<<"ae0c2d9f3db290f9">>}]},
        5000],
       [{file,
          "/home/circleci/project/big_tests/_build/default/lib/escalus/src/escalus_client.erl"},
        {line,136}]},
     {privacy_helper,set_default_list,2,
       [{file,
          "/home/circleci/project/big_tests/tests/privacy_helper.e...

Report log

graphql_account_SUITE:admin_account_http:admin_register_user_limit_error

{error,
  {{badmap,null},
   [{erlang,map_get,
      [<<"message">>,null],
      [{error_info,#{module => erl_erts_errors}}]},
    {graphql_helper,get_value,2,
      [{file,"/home/circleci/project/big_tests/tests/graphql_helper.erl"},
       {line,292}]},
    {graphql_account_SUITE,admin_register_user_limit_error,1,
      [{file,
         "/home/circleci/project/big_tests/tests/graphql_account_SUITE.erl"},
       {line,387}]},
    {test_server,ts_tc,3,[{file,"test_server.erl"},{line,1793}]},
    {test_server,run_test_case_eval1,6,
      [{file,"test_server.erl"},{line,1302}]},
    {test_server,run_test_case_eval,9,
      [{file,"test_server.erl"},{line,1234}]}]}}

Report log

graphql_account_SUITE:admin_account_cli:admin_register_user_limit_error

{error,
  {#{what => invalid_response_code,expected_type => ok,
     response_code => {exit_status,1}},
   [{graphql_helper,assert_response_code,2,
      [{file,"/home/circleci/project/big_tests/tests/graphql_helper.erl"},
       {line,256}]},
    {graphql_helper,get_ok_value,2,
      [{file,"/home/circleci/project/big_tests/tests/graphql_helper.erl"},
       {line,239}]},
    {graphql_account_SUITE,admin_register_user_limit_error,1,
      [{file,
         "/home/circleci/project/big_tests/tests/graphql_account_SUITE.erl"},
       {line,387}]},
    {test_server,ts_tc,3,[{file,"test_server.erl"},{line,1793}]},
    {test_server,run_test_case_eval1,6,
      [{file,"test_server.erl"},{line,1302}]},
    {test_server,run_test_case_eval,9,
      [{file,"test_server.erl"},{line,1234}]}]}}

Report log

graphql_account_SUITE:domain_admin_account:admin_register_user_limit_error

{error,
  {{badmap,null},
   [{erlang,map_get,
      [<<"message">>,null],
      [{error_info,#{module => erl_erts_errors}}]},
    {graphql_helper,get_value,2,
      [{file,"/home/circleci/project/big_tests/tests/graphql_helper.erl"},
       {line,292}]},
    {graphql_account_SUITE,admin_register_user_limit_error,1,
      [{file,
         "/home/circleci/project/big_tests/tests/graphql_account_SUITE.erl"},
       {line,387}]},
    {test_server,ts_tc,3,[{file,"test_server.erl"},{line,1793}]},
    {test_server,run_test_case_eval1,6,
      [{file,"test_server.erl"},{line,1302}]},
    {test_server,run_test_case_eval,9,
      [{file,"test_server.erl"},{line,1234}]}]}}

Report log

graphql_last_SUITE:admin_http:admin_last_configured:admin_old_users:admin_list_old_users_domain

{error,{{badmatch,[#{<<"jid">> => <<"bob@localhost">>,
           <<"timestamp">> => <<"2023-12-06T08:03:29.000000Z">>},
           #{<<"jid">> =>
             <<"bob_error_deliver_to_offline_user_test_72_error_deliver_to_offline_user_test_128@localhost">>,
           <<"timestamp">> => null}]},
    [{graphql_last_SUITE,admin_list_old_users_domain_story,3,
               [{file,"/home/circleci/project/big_tests/tests/graphql_last_SUITE.erl"},
                {line,388}]},
     {test_server,ts_tc,3,[{file,"test_server.erl"},{line,1793}]},
     {test_server,run_test_case_eval1,6,
            [{file,"test_server.erl"},{line,1302}]},
     {test_server,run_test_case_eval,9,
            [{file,"test_server.erl"},{line,1234}]}]}}

Report log

graphql_last_SUITE:admin_http:admin_last_configured:admin_old_users:admin_list_old_users_global

{error,{{badmatch,[#{<<"jid">> => <<"alice@localhost.bis">>,
           <<"timestamp">> => <<"2023-12-06T08:03:29.000000Z">>},
           #{<<"jid">> => <<"bob@localhost">>,
           <<"timestamp">> => <<"2023-12-06T08:03:29.000000Z">>},
           #{<<"jid">> =>
             <<"bob_error_deliver_to_offline_user_test_72_error_deliver_to_offline_user_test_128@localhost">>,
           <<"timestamp">> => null}]},
    [{graphql_last_SUITE,admin_list_old_users_global_story,4,
               [{file,"/home/circleci/project/big_tests/tests/graphql_last_SUITE.erl"},
                {line,412}]},
     {test_server,ts_tc,3,[{file,"test_server.erl"},{line,1793}]},
     {test_server,run_test_case_eval1,6,
            [{file,"test_server.erl"},{line,1302}]},
     {test_server,run_test_case_eval,9,
            [{file,"test_server.erl"},{line,1234}]}]}}

Report log

graphql_last_SUITE:admin_http:admin_last_configured:admin_old_users:admin_remove_old_users_domain

{error,{{badmatch,[#{<<"jid">> => <<"bob@localhost">>,
           <<"timestamp">> => <<"2023-12-06T08:03:29.000000Z">>},
           #{<<"jid">> =>
             <<"bob_error_deliver_to_offline_user_test_72_error_deliver_to_offline_user_test_128@localhost">>,
           <<"timestamp">> => null}]},
    [{graphql_last_SUITE,admin_remove_old_users_domain_story,4,
               [{file,"/home/circleci/project/big_tests/tests/graphql_last_SUITE.erl"},
                {line,340}]},
     {test_server,ts_tc,3,[{file,"test_server.erl"},{line,1793}]},
     {test_server,run_test_case_eval1,6,
            [{file,"test_server.erl"},{line,1302}]},
     {test_server,run_test_case_eval,9,
            [{file,"test_server.erl"},{line,1234}]}]}}

Report log

graphql_muc_light_SUITE:admin_cli:admin_muc_light:end_per_group

{error,
 {{unregistering_failed,
   {amount,3},
   {unregistered_items,
  [{{<<"_admin_change_room_config_non_existent_domain_1243">>,
     [{escalus_event_mgr,<0.24437.0>},
    {tc_name,admin_change_room_config_non_existent_domain},
    {escalus_cleaner,<0.24436.0>},
    {watchdog,<0.24435.0>},
    {muc_light_host,<<"muclight.localhost">>},
    {secondary_muc_light_host,<<"muclight.localhost.bis">>},
    {protocol,cli},
    {schema_endpoint,admin},
    {{ejabberd_cwd,mongooseim@localhost},
     "/home/circleci/project/_build/mim1/rel/mongooseim"},
    {preset,"odbc_mssql_mnesia"},
    {mim_data_dir,
     "/home/circleci/project/big_tests/tests/graphql_muc_light_SUITE_data"},
    {tc_logfile,
     "https://circleci-mim-results.s3.eu-central-1.amazonaws.com/PR/4184/201960/odbc_mssql_mnesia.26.1.2-amd64/big/ct_run.test%40af21f0492493.2023-12-06_07.58.04/big_tests.tests.graphql_muc_light_SUITE.logs/run.2023-12-06_08.02.36/graphql_muc_light_suite.admin_change_room_config_non_existent_domain.59650.html"},
    {tc_group_properties,[{name,admin_muc_light}]},
    {tc_group_path,[[{name,admin_cli}]]},
    {data_dir,
     "/home/circleci/project/big_tests/_build/default/lib/mongoose_tests/ebin/graphql_muc_light_SUITE_data/"},
    {priv_dir,
     "https://circleci-mim-results.s3.eu-central-1.amazonaws.com/PR/4184/201960/odbc_mssql_mnesia.26.1.2-amd64/big/ct_run.test%40af21f0492493.2023-12-06_07.58.04/big_tests.tests.graphql_muc_light_SUITE.logs/run.2023-12-06_08.02.36/log_private/"},
    {{saved_modules,mongooseim@localhost,<<"localhost">>},
     #{mod_vcard =>
      #{matches => 30,
        host => {prefix,<<"vjud.">>}...

Report log

graphql_muc_light_SUITE:domain_admin:domain_admin_muc_light:admin_create_room_with_unprepped_id

{error,
  {{badmatch,null},
   [{graphql_muc_light_SUITE,admin_create_room_with_unprepped_id,1,
      [{file,
         "/home/circleci/project/big_tests/tests/graphql_muc_light_SUITE.erl"},
       {line,1155}]},
    {test_server,ts_tc,3,[{file,"test_server.erl"},{line,1793}]},
    {test_server,run_test_case_eval1,6,
      [{file,"test_server.erl"},{line,1302}]},
    {test_server,run_test_case_eval,9,
      [{file,"test_server.erl"},{line,1234}]}]}}

Report log

pgsql_mnesia_26 / pgsql_mnesia / e0d1fd1
Reports root/ big
OK: 4614 / Failed: 0 / User-skipped: 109 / Auto-skipped: 0

dynamic_domains_mysql_redis_26 / mysql_redis / e0d1fd1
Reports root/ big
OK: 4192 / Failed: 0 / User-skipped: 135 / Auto-skipped: 0

dynamic_domains_pgsql_mnesia_26 / pgsql_mnesia / e0d1fd1
Reports root/ big
OK: 4225 / Failed: 0 / User-skipped: 102 / Auto-skipped: 0

mssql_mnesia_26 / odbc_mssql_mnesia / e0d1fd1
Reports root/ big
OK: 4611 / Failed: 0 / User-skipped: 112 / Auto-skipped: 0

chrzaszcz

👍

This comment was marked as outdated.

Sign in to view

jacekwegr force-pushed the change-domain-validation branch from f4ee7a4 to be6206a Compare December 4, 2023 13:53

This comment was marked as outdated.

Sign in to view

jacekwegr marked this pull request as ready for review December 4, 2023 14:48

chrzaszcz reviewed Dec 4, 2023

View reviewed changes

test/config_parser_SUITE.erl Outdated Show resolved Hide resolved

test/config_parser_SUITE.erl Outdated Show resolved Hide resolved

This comment was marked as outdated.

Sign in to view

chrzaszcz reviewed Dec 5, 2023

View reviewed changes

test/config_parser_SUITE.erl Outdated Show resolved Hide resolved

jacekwegr added 3 commits December 6, 2023 08:51

Change domain validation logic

6b9fb65

- Changed domain validation logic - Moved some helpers to `test/common` to use ct:logger_backend in small tests

Additional tests for domain validation

04a0f5d

Refactor invalid host testing

e0d1fd1

This comment was marked as outdated.

Sign in to view

jacekwegr force-pushed the change-domain-validation branch from f86a839 to e0d1fd1 Compare December 6, 2023 07:54

chrzaszcz approved these changes Dec 6, 2023

View reviewed changes

chrzaszcz merged commit f44b5b0 into master Dec 6, 2023
4 checks passed

chrzaszcz deleted the change-domain-validation branch December 6, 2023 09:11

chrzaszcz added this to the 6.2.0 milestone Dec 11, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Change domain validation logic #4184

Change domain validation logic #4184

jacekwegr commented Dec 4, 2023 •

edited

This comment was marked as outdated.

codecov bot commented Dec 4, 2023 •

edited

This comment was marked as outdated.

chrzaszcz left a comment

This comment was marked as outdated.

chrzaszcz left a comment

This comment was marked as outdated.

mongoose-im commented Dec 6, 2023 •

edited

chrzaszcz left a comment

Change domain validation logic #4184

Change domain validation logic #4184

Conversation

jacekwegr commented Dec 4, 2023 • edited

This comment was marked as outdated.

codecov bot commented Dec 4, 2023 • edited

Codecov Report

This comment was marked as outdated.

chrzaszcz left a comment

Choose a reason for hiding this comment

This comment was marked as outdated.

chrzaszcz left a comment

Choose a reason for hiding this comment

This comment was marked as outdated.

mongoose-im commented Dec 6, 2023 • edited

chrzaszcz left a comment

Choose a reason for hiding this comment

jacekwegr commented Dec 4, 2023 •

edited

codecov bot commented Dec 4, 2023 •

edited

mongoose-im commented Dec 6, 2023 •

edited