Add an option to format a section/list as a map #3420
Conversation
chrzaszcz
force-pushed
the
config-with-maps
branch
from
f7210e7
to
65f3806
Compare
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
Codecov Report
@@ Coverage Diff @@
## master #3420 +/- ##
==========================================
+ Coverage 80.76% 80.79% +0.02%
==========================================
Files 414 414
Lines 32335 32312 -23
==========================================
- Hits 26116 26106 -10
+ Misses 6219 6206 -13
Continue to review full report at Codecov.
|
This comment has been minimized.
This comment has been minimized.
chrzaszcz
force-pushed
the
config-with-maps
branch
from
ebc729d
to
bcf2c88
Compare
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
chrzaszcz
force-pushed
the
config-with-maps
branch
from
e8c36e9
to
6fc0eb6
Compare
This comment has been minimized.
This comment has been minimized.
chrzaszcz
force-pushed
the
config-with-maps
branch
from
6fc0eb6
to
3a013a2
Compare
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
chrzaszcz
force-pushed
the
config-with-maps
branch
from
b23d1f9
to
3fafbd6
Compare
This comment has been minimized.
This comment has been minimized.
chrzaszcz
force-pushed
the
config-with-maps
branch
from
3fafbd6
to
6b74e47
Compare
This comment has been minimized.
This comment has been minimized.
chrzaszcz
force-pushed
the
config-with-maps
branch
from
6b74e47
to
b97ef9a
Compare
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
chrzaszcz
force-pushed
the
config-with-maps
branch
from
9340919
to
649f396
Compare
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
chrzaszcz
force-pushed
the
config-with-maps
branch
from
5963044
to
c200f31
Compare
This comment has been minimized.
This comment has been minimized.
This option will allow to format a KV list as a map.
Put it: - after 'validate' to validate the lists normally (without two branches: for lists and maps) - before 'process' to process maps instead of KV lists (easier).
If the supplied key is a list, it denotes a path in the config tree
The whole 'auth' section becomes a map as it was complicated, having both nested and top-level options. Regarding s2s, only the options with tagged keys are changed to maps. The whole s2s section could also be a map, but it is not necessary.
- The 'foreach' format (now unused) is removed (it was temporary) - The types are updated with maps and without the complex s2s keys
Use mongoose_config API directly instead of ejabberd_auth - it seems more straightforward and the former convention was used only partially.
The changes in other modules were minimised for now. Full conversion needs changing mod_vcard and shared roster code.
Option overriding is now simplified.
chrzaszcz
force-pushed
the
config-with-maps
branch
2 times, most recently
from
bc5c5c6
to
3f2fe04
Compare
|
small_tests_24 / small_tests / bc5c5c6 small_tests_23 / small_tests / bc5c5c6 dynamic_domains_pgsql_mnesia_24 / pgsql_mnesia / bc5c5c6 dynamic_domains_pgsql_mnesia_23 / pgsql_mnesia / bc5c5c6 dynamic_domains_mysql_redis_24 / mysql_redis / bc5c5c6 dynamic_domains_mssql_mnesia_24 / odbc_mssql_mnesia / bc5c5c6 ldap_mnesia_23 / ldap_mnesia / bc5c5c6 ldap_mnesia_24 / ldap_mnesia / bc5c5c6 internal_mnesia_24 / internal_mnesia / bc5c5c6 elasticsearch_and_cassandra_24 / elasticsearch_and_cassandra_mnesia / bc5c5c6 pgsql_mnesia_24 / pgsql_mnesia / bc5c5c6 mysql_redis_24 / mysql_redis / bc5c5c6 pgsql_mnesia_23 / pgsql_mnesia / bc5c5c6 sm_SUITE:parallel_manual_ack_freq_1:resume_session_state_stop_c2s{error,{thrown,{timeout,msg}}}mssql_mnesia_24 / odbc_mssql_mnesia / bc5c5c6 riak_mnesia_24 / riak_mnesia / bc5c5c6 |
chrzaszcz
force-pushed
the
config-with-maps
branch
from
3f2fe04
to
6d03575
Compare
|
small_tests_24 / small_tests / 3f2fe04 small_tests_23 / small_tests / 3f2fe04 dynamic_domains_pgsql_mnesia_23 / pgsql_mnesia / 3f2fe04 dynamic_domains_mysql_redis_24 / mysql_redis / 3f2fe04 dynamic_domains_pgsql_mnesia_24 / pgsql_mnesia / 3f2fe04 sm_SUITE:parallel:messages_are_properly_flushed_during_resumption_p1_fsm_old{error,
{{badmatch,
{error,
{connection_step_failed,
{#Fun<sm_SUITE.11.68776247>,
{client,
<<"alicE_messages_are_properly_flushed_during_resumption_p1_fsm_old_1964@domain.example.com">>,
escalus_tcp,<0.31571.1>,undefined,
[{username,
<<"alicE_messages_are_properly_flushed_during_resumption_p1_fsm_old_1964">>},
{server,<<"domain.example.com">>},
{host,<<"localhost">>},
{password,<<"matygrysa">>},
{stream_management,true},
{stream_id,<<"f6b65b8ca739088c">>}]},
[{compression,[<<"zlib">>]},
{starttls,true},
{stream_management,true},
{advanced_message_processing,true},
{client_state_indication,false},
{sasl_mechanisms,[<<"SCRAM-SHA-256">>,<<"PLAIN">>]},
{caps,undefined}]},
{timeout,get_resumed}}}},
[{sm_SUITE,
'-messages_are_properly_flushed_during_resumption_p1_fsm_old/1-fun-1-',
3,
[{file,"/home/circleci/project/big_tests/tests/sm_SUITE.erl"},
{line,1270}]},
{escalus_story,story,4,
[{file,
"/home/circleci/project/big_tests/_build/default/lib/escalus/src/escalus_story.erl"},
{line,72}]},
{test_server,ts_tc,3,[{file,"test_server.erl"},{line,1783}]},
{test_server,run_test_case_eval1,6,
[{file,"test_server.erl"},{line,1292}]},
{test_server,run_test_case_eval,9,
[{file,"test_se...ldap_mnesia_23 / ldap_mnesia / 3f2fe04 ldap_mnesia_24 / ldap_mnesia / 3f2fe04 internal_mnesia_24 / internal_mnesia / 3f2fe04 pgsql_mnesia_23 / pgsql_mnesia / 3f2fe04 mysql_redis_24 / mysql_redis / 3f2fe04 elasticsearch_and_cassandra_24 / elasticsearch_and_cassandra_mnesia / 3f2fe04 mssql_mnesia_24 / odbc_mssql_mnesia / 3f2fe04 pgsql_mnesia_24 / pgsql_mnesia / 3f2fe04 riak_mnesia_24 / riak_mnesia / 3f2fe04 |
Motivation: after removal of 'foreach' this step no longer modifies the value and always wraps it in a list (optionally with a key).
chrzaszcz
force-pushed
the
config-with-maps
branch
from
6d03575
to
1af9d2d
Compare
|
small_tests_24 / small_tests / 6d03575 small_tests_23 / small_tests / 6d03575 dynamic_domains_mysql_redis_24 / mysql_redis / 6d03575 dynamic_domains_pgsql_mnesia_24 / pgsql_mnesia / 6d03575 dynamic_domains_pgsql_mnesia_23 / pgsql_mnesia / 6d03575 dynamic_domains_mssql_mnesia_24 / odbc_mssql_mnesia / 6d03575 ldap_mnesia_23 / ldap_mnesia / 6d03575 ldap_mnesia_24 / ldap_mnesia / 6d03575 internal_mnesia_24 / internal_mnesia / 6d03575 pgsql_mnesia_24 / pgsql_mnesia / 6d03575 elasticsearch_and_cassandra_24 / elasticsearch_and_cassandra_mnesia / 6d03575 pgsql_mnesia_23 / pgsql_mnesia / 6d03575 sm_SUITE:parallel:messages_are_properly_flushed_during_resumption{error,
{{badmatch,
{error,
{connection_step_failed,
{#Fun<sm_SUITE.11.2813143>,
{client,
<<"alicE_messages_are_properly_flushed_during_resumption_2046@localhost">>,
escalus_tcp,<0.13699.2>,undefined,
[{username,
<<"alicE_messages_are_properly_flushed_during_resumption_2046">>},
{server,<<"localhost">>},
{password,<<"matygrysa">>},
{stream_management,true},
{host,<<"localhost">>},
{stream_id,<<"c723ab1bf95ca86b">>}]},
[{compression,[<<"zlib">>]},
{starttls,true},
{stream_management,true},
{advanced_message_processing,true},
{client_state_indication,false},
{sasl_mechanisms,[<<"SCRAM-SHA-256">>,<<"PLAIN">>]},
{caps,undefined}]},
{timeout,get_resumed}}}},
[{sm_SUITE,'-messages_are_properly_flushed_during_resumption/1-fun-1-',
3,
[{file,"/home/circleci/project/big_tests/tests/sm_SUITE.erl"},
{line,1226}]},
{escalus_story,story,4,
[{file,
"/home/circleci/project/big_tests/_build/default/lib/escalus/src/escalus_story.erl"},
{line,72}]},
{test_server,ts_tc,3,[{file,"test_server.erl"},{line,1754}]},
{test_server,run_test_case_eval1,6,
[{file,"test_server.erl"},{line,1263}]},
{test_server,run_test_case_eval,9,
[{file,"test_server.erl"},{line,1195}]}]}}sm_SUITE:parallel_manual_ack_freq_1:resume_session_state_stop_c2s{error,{thrown,{timeout,msg}}}mssql_mnesia_24 / odbc_mssql_mnesia / 6d03575 rest_client_SUITE:roster:add_and_remove_some_contacts_properly{error,
{timeout_when_waiting_for_stanza,
[{escalus_client,wait_for_stanza,
[{client,
<<"bOb_add_and_remove_some_contacts_properly_2172@localhost/res1">>,
escalus_tcp,<0.7935.2>,
[{event_manager,<0.7878.2>},
{server,<<"localhost">>},
{username,
<<"bOb_add_and_remove_some_contacts_properly_2172">>},
{resource,<<"res1">>}],
[{event_client,
[{event_manager,<0.7878.2>},
{server,<<"localhost">>},
{username,
<<"bOb_add_and_remove_some_contacts_properly_2172">>},
{resource,<<"res1">>}]},
{resource,<<"res1">>},
{username,
<<"bOb_add_and_remove_some_contacts_properly_2172">>},
{server,<<"localhost">>},
{host,<<"localhost">>},
{port,5222},
{auth,{escalus_auth,auth_plain}},
{wspath,undefined},
{username,
<<"bOb_add_and_remove_some_contacts_properly_2172">>},
{server,<<"localhost">>},
{password,<<"makrolika">>},
{stream_id,<<"6f73a5a7caab91d6">>}]},
5000],
[{file,
"/home/circleci/project/big_tests/_build/default/lib/escalus/src/escalus_client.erl"},
{line,136}]},
{rest_client_SUITE,add_contact_check_roster_push,2,
[{file,
"/home/circleci/project/big_tests/tests/rest_client_SUITE.erl"},
{line,1310}]},
{lists,foreach,2,...mysql_redis_24 / mysql_redis / 6d03575 riak_mnesia_24 / riak_mnesia / 6d03575 |
|
small_tests_24 / small_tests / 1af9d2d small_tests_23 / small_tests / 1af9d2d dynamic_domains_pgsql_mnesia_24 / pgsql_mnesia / 1af9d2d dynamic_domains_mysql_redis_24 / mysql_redis / 1af9d2d dynamic_domains_pgsql_mnesia_23 / pgsql_mnesia / 1af9d2d dynamic_domains_mssql_mnesia_24 / odbc_mssql_mnesia / 1af9d2d ldap_mnesia_24 / ldap_mnesia / 1af9d2d internal_mnesia_24 / internal_mnesia / 1af9d2d ldap_mnesia_23 / ldap_mnesia / 1af9d2d pgsql_mnesia_24 / pgsql_mnesia / 1af9d2d elasticsearch_and_cassandra_24 / elasticsearch_and_cassandra_mnesia / 1af9d2d sm_SUITE:parallel:messages_are_properly_flushed_during_resumption_p1_fsm_old{error,
{{badmatch,
{error,
{connection_step_failed,
{#Fun<sm_SUITE.11.68776247>,
{client,
<<"alicE_messages_are_properly_flushed_during_resumption_p1_fsm_old_1612@localhost">>,
escalus_tcp,<0.25546.1>,undefined,
[{username,
<<"alicE_messages_are_properly_flushed_during_resumption_p1_fsm_old_1612">>},
{server,<<"localhost">>},
{password,<<"matygrysa">>},
{stream_management,true},
{host,<<"localhost">>},
{stream_id,<<"f6de8750d4604e67">>}]},
[{compression,[<<"zlib">>]},
{starttls,true},
{stream_management,true},
{advanced_message_processing,true},
{client_state_indication,false},
{sasl_mechanisms,[<<"SCRAM-SHA-256">>,<<"PLAIN">>]},
{caps,undefined}]},
{timeout,get_resumed}}}},
[{sm_SUITE,
'-messages_are_properly_flushed_during_resumption_p1_fsm_old/1-fun-1-',
3,
[{file,"/home/circleci/project/big_tests/tests/sm_SUITE.erl"},
{line,1270}]},
{escalus_story,story,4,
[{file,
"/home/circleci/project/big_tests/_build/default/lib/escalus/src/escalus_story.erl"},
{line,72}]},
{test_server,ts_tc,3,[{file,"test_server.erl"},{line,1783}]},
{test_server,run_test_case_eval1,6,
[{file,"test_server.erl"},{line,1292}]},
{test_server,run_test_case_eval,9,
[{file,"test_server.erl"},{line,1...mysql_redis_24 / mysql_redis / 1af9d2d mssql_mnesia_24 / odbc_mssql_mnesia / 1af9d2d pgsql_mnesia_23 / pgsql_mnesia / 1af9d2d service_domain_db_SUITE:db:db_reinserted_from_one_node_while_service_disabled_on_another{error,
{{badmatch,{ok,<<"dbgroup2">>}},
[{service_domain_db_SUITE,
db_reinserted_from_one_node_while_service_disabled_on_another,1,
[{file,
"/home/circleci/project/big_tests/tests/service_domain_db_SUITE.erl"},
{line,586}]},
{test_server,ts_tc,3,[{file,"test_server.erl"},{line,1754}]},
{test_server,run_test_case_eval1,6,
[{file,"test_server.erl"},{line,1263}]},
{test_server,run_test_case_eval,9,
[{file,"test_server.erl"},{line,1195}]}]}}riak_mnesia_24 / riak_mnesia / 1af9d2d pgsql_mnesia_23 / pgsql_mnesia / 1af9d2d |
Introduce the new
format_itemsfield to allow conversion of selected config sections to maps. In the future, when we convert enough sections, we could makeformat_items = mapdefault for sections. For now it would mean too many changes.Another important change is the ability to call
get_optandlookup_optinmongoose_configwith the complete path for a nested option, e.g.[{auth, HostType}, riak, bucket_type]. The path might look cleaner withHostTypepassed as a separate argument, but this would increase the complexity ofmongoose_config, which is quite clean and simple right now.The conversion to maps is done for the
authsection and for selecteds2soptions. The main motivation for choosing these particular options was that their processing used theforeachformat, which was a temporary hack.Other changes and notes:
authfor now. This should be finished when converting modules using LDAP.format = nonejust like before.format_itemsstep beforeprocessallows simpler processing functions, taking a map as an argument.format_itemsstep aftervalidateallows validation of lists just like before, improving steps separation.Some further polishing could be done later, e.g. regarding the dafaults and password format.