Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Handle node-specific options correctly in reload_cluster #1948

Merged
merged 67 commits into from Jul 4, 2018
Merged

Handle node-specific options correctly in reload_cluster #1948

Show file tree
Hide file tree
Changes from 66 commits
Commits
Show all changes
67 commits
Select commit Hold shift + click to select a range
531b509
Move pure parts of ejabberd_config into mongoose_config
arcusfelis Jun 18, 2018
c9f26c7
Move include_config_file into mongoose_config
arcusfelis Jun 18, 2018
29d563a
Move is_file_readable into mongoose_config_utils
arcusfelis Jun 18, 2018
215bca4
Move exit_or_halt/1, get_absolute_path/1, get_config_lines/4 into mon…
arcusfelis Jun 18, 2018
fc21ae5
Split ejabberd_config:set_opts/1
arcusfelis Jun 18, 2018
2e2f4ba
Replace old style catch with try...catch in ejabberd_config
arcusfelis Jun 18, 2018
2c23f38
Replace compare_result record with a map
arcusfelis Jun 18, 2018
5165aad
Introduce mongoose_config:config_diff type
arcusfelis Jun 18, 2018
85b1dc9
Add compute_current_config_version/0 into ejabberd_config
arcusfelis Jun 18, 2018
cbee13a
Rename do_set_global_opts to maybe_clean_global_opts
arcusfelis Jun 18, 2018
8de40a5
Add ejabberd_config:handle_table_does_not_exist_error/1 function
arcusfelis Jun 18, 2018
3041db7
Add mongoose_config:flatten_opts/2
arcusfelis Jun 19, 2018
a91191d
Add mongoose_config:does_pattern_match/2
arcusfelis Jun 21, 2018
ddb97bb
Allow underscore in configs
arcusfelis Jun 21, 2018
a151825
Add node_specific_options key to config
arcusfelis Jun 21, 2018
ee589a0
Introduce mongoose_config:categorized_options/0 type
arcusfelis Jun 21, 2018
78832d3
Add get_categorized_options() function
arcusfelis Jun 21, 2018
c4efc24
Add ejabberd_config:config_state/0, config_states/0
arcusfelis Jun 21, 2018
956705b
Add failed_checks in mongoose_config
arcusfelis Jun 22, 2018
d2f558b
Add checks into strategy and change calculation
arcusfelis Jun 22, 2018
86f9d1e
Rewrite reload_cluster and reload_local
arcusfelis Jun 22, 2018
f65d62f
Correctly flatten simple module options
arcusfelis Jun 22, 2018
2750c5b
Run dump_reload_state
arcusfelis Jun 25, 2018
d82b003
Make reloading handle host-specific options correctly
arcusfelis Jun 25, 2018
d6f07fb
Handle no_update_required
arcusfelis Jun 25, 2018
eeeeb65
Fix compilations warnings for mongoose_config
arcusfelis Jun 25, 2018
5c309a6
Add ejabberd_config_SUITE:cluster_smoke test
arcusfelis Jun 25, 2018
2764b2c
Add change_module_option_with_node_param_opts test case
arcusfelis Jun 25, 2018
233a839
Fix dialyzer warnings
arcusfelis Jun 25, 2018
a223918
Add print_flatten_config command
arcusfelis Jun 26, 2018
b880fbc
Document node_specific_options
arcusfelis Jun 26, 2018
6a15956
Replace flatten with FLATTEN
arcusfelis Jun 26, 2018
1bb547b
Do not flatten empty lists of suboptions
arcusfelis Jun 26, 2018
6f15391
Add flatten_options docs
arcusfelis Jun 26, 2018
0aa5d23
Check that nested node specific optinos are handled correctly
arcusfelis Jun 26, 2018
1a2ea42
Increase slave node boot timeout in ejabberd_config_SUITE
arcusfelis Jun 27, 2018
0c9367e
Fix wrong name for inconsistent_ondisc_local_versions bug
arcusfelis Jun 27, 2018
79abaf0
Properly calculate changes for deps modules
arcusfelis Jun 27, 2018
274b63a
Add ejabberd.no_listeners.gd.node1_v1.cfg
arcusfelis Jun 27, 2018
6db477d
Print dots in small tests
arcusfelis Jun 28, 2018
7bc4142
Extend node-specific-options example
arcusfelis Jun 28, 2018
1a71044
Replace flatten with flat where is needed
arcusfelis Jun 29, 2018
8c46afa
Extract code into mongoose_config_flat
arcusfelis Jun 29, 2018
60474cf
Rename raise_error_on_module_start_failure to is_mim_or_ct_running
arcusfelis Jun 29, 2018
198340c
Fix config_version calculation in config_info
arcusfelis Jun 29, 2018
12f18ee
Extend node-specific modules docs
arcusfelis Jun 29, 2018
7182238
Typos in mongoose_config_utils and mongoose_config
arcusfelis Jun 29, 2018
1d30318
Refactor search_hosts_and_pools/2
arcusfelis Jun 29, 2018
f1c4088
Refactor config parser
arcusfelis Jun 29, 2018
be1e93c
Check that required files exist outside of pure code
arcusfelis Jun 29, 2018
4c82556
Remove "duplicate Running small tests" print
arcusfelis Jun 29, 2018
26b420e
Refactor categorize_options function
arcusfelis Jun 29, 2018
b7ce3b1
Unify reload commands
arcusfelis Jun 29, 2018
35f7e90
Remove print_reload_strategy
arcusfelis Jun 29, 2018
7ad5e7e
Move config stuff into a subdirectory
arcusfelis Jul 2, 2018
e447776
Rename mongoose_config into mongoose_config_parser
arcusfelis Jul 2, 2018
8830f3a
Extract reloading code into mongoose_config_reload
arcusfelis Jul 2, 2018
d0b5ad8
Remove unused compare_modules function
arcusfelis Jul 2, 2018
b5c073f
Move key() and value() types into mongoose_config_parser
arcusfelis Jul 2, 2018
b791362
Rename reloading strategy into reloading context
arcusfelis Jul 2, 2018
9c207cf
Fix compacting of values of required_files list
arcusfelis Jul 3, 2018
f04606d
Use correct slave timeout options
arcusfelis Jul 3, 2018
f3b0690
Fix verify_reload_output in reload_helper
arcusfelis Jul 3, 2018
e7527cc
Fail travis build if small_tests fail
arcusfelis Jul 3, 2018
c3c379f
Fix slave_node boot_timeout in ejabberd_config_SUITE
arcusfelis Jul 3, 2018
acefbc9
Fix undefined fun call to expand_opts [dialyzer]
arcusfelis Jul 3, 2018
6554ab9
Language check [skip ci]
arcusfelis Jul 4, 2018
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
2 changes: 1 addition & 1 deletion big_tests/tests/reload_helper.erl
View file
Open in desktop
Expand Up @@ -27,7 +27,7 @@
-import(distributed_helper, [rpc/4, rpc/5]).

-define(CTL_RELOAD_OUTPUT_PREFIX,
"# Reloaded: " ++ atom_to_list(ct:get_config({hosts, mim, node}))).
"done").

backup_ejabberd_config_file(Node, Config) ->
{ok, _} = rpc(Node, file, copy, [node_cfg(Node, current, Config),
Expand Down
215 changes: 215 additions & 0 deletions doc/developers-guide/flat_options.md
View file
Open in desktop
@@ -0,0 +1,215 @@
# Flat options format

`'FLAT'` means that the value was separated into more options. For example,
module options are separated into a list of `module_opt`-s.

Each flat option key starts with an option type:

- `l` - local options
- `h` - local host options
- `g` - global options

# Flat options example

```erlang
{[l,listen],'FLAT'}.
{[l,listener,{5280,{0,0,0,0},tcp},ejabberd_cowboy],'FLAT'}.
{[l,listener_opt,{5280,{0,0,0,0},tcp},ejabberd_cowboy,num_acceptors],10}.
{[l,listener_opt,{5280,{0,0,0,0},tcp},ejabberd_cowboy,transport_options],
[{max_connections,1024}]}.
{[l,listener_opt,{5280,{0,0,0,0},tcp},ejabberd_cowboy,modules],
[{"_","/http-bind",mod_bosh},
{"_","/ws-xmpp",mod_websockets,
[{ejabberd_service,[{access,all},
{shaper_rule,fast},
{ip,{127,0,0,1}},
{password,"secret"}]}]}]}.
{[l,listener,{5285,{0,0,0,0},tcp},ejabberd_cowboy],'FLAT'}.
{[l,listener_opt,{5285,{0,0,0,0},tcp},ejabberd_cowboy,num_acceptors],10}.
{[l,listener_opt,{5285,{0,0,0,0},tcp},ejabberd_cowboy,transport_options],
[{max_connections,1024}]}.
{[l,listener_opt,{5285,{0,0,0,0},tcp},ejabberd_cowboy,ssl],
[{certfile,"priv/ssl/fake_cert.pem"},
{keyfile,"priv/ssl/fake_key.pem"},
{password,[]}]}.
{[l,listener_opt,{5285,{0,0,0,0},tcp},ejabberd_cowboy,modules],
[{"_","/http-bind",mod_bosh},{"_","/ws-xmpp",mod_websockets,[]}]}.
{[l,listener,{8088,{127,0,0,1},tcp},ejabberd_cowboy],'FLAT'}.
{[l,listener_opt,{8088,{127,0,0,1},tcp},ejabberd_cowboy,num_acceptors],10}.
{[l,listener_opt,{8088,{127,0,0,1},tcp},ejabberd_cowboy,transport_options],
[{max_connections,1024}]}.
{[l,listener_opt,{8088,{127,0,0,1},tcp},ejabberd_cowboy,modules],
[{"localhost","/api",mongoose_api_admin,[]}]}.
{[l,listener,{8089,{0,0,0,0},tcp},ejabberd_cowboy],'FLAT'}.
{[l,listener_opt,{8089,{0,0,0,0},tcp},ejabberd_cowboy,num_acceptors],10}.
{[l,listener_opt,{8089,{0,0,0,0},tcp},ejabberd_cowboy,transport_options],
[{max_connections,1024}]}.
{[l,listener_opt,{8089,{0,0,0,0},tcp},ejabberd_cowboy,protocol_options],
[{compress,true}]}.
{[l,listener_opt,{8089,{0,0,0,0},tcp},ejabberd_cowboy,ssl],
[{certfile,"priv/ssl/fake_cert.pem"},
{keyfile,"priv/ssl/fake_key.pem"},
{password,[]}]}.
{[l,listener_opt,{8089,{0,0,0,0},tcp},ejabberd_cowboy,modules],
[{"_","/api/sse",lasse_handler,[mongoose_client_api_sse]},
{"_","/api/messages/[:with]",mongoose_client_api_messages,[]},
{"_","/api/contacts/[:jid]",mongoose_client_api_contacts,[]},
{"_","/api/rooms/[:id]",mongoose_client_api_rooms,[]},
{"_","/api/rooms/:id/users/[:user]",mongoose_client_api_rooms_users,[]},
{"_","/api/rooms/[:id]/messages",mongoose_client_api_rooms_messages,[]}]}.
{[l,listener,{5288,{127,0,0,1},tcp},ejabberd_cowboy],'FLAT'}.
{[l,listener_opt,{5288,{127,0,0,1},tcp},ejabberd_cowboy,num_acceptors],10}.
{[l,listener_opt,{5288,{127,0,0,1},tcp},ejabberd_cowboy,transport_options],
[{max_connections,1024}]}.
{[l,listener_opt,{5288,{127,0,0,1},tcp},ejabberd_cowboy,modules],
[{"localhost","/api",mongoose_api,
[{handlers,[mongoose_api_metrics,mongoose_api_users]}]}]}.
{[l,listener,{5222,{0,0,0,0},tcp},ejabberd_c2s],'FLAT'}.
{[l,listener_opt,{5222,{0,0,0,0},tcp},ejabberd_c2s,certfile],
"priv/ssl/fake_server.pem"}.
{[l,listener_simple_opt,{5222,{0,0,0,0},tcp},ejabberd_c2s,starttls],simple}.
{[l,listener_opt,{5222,{0,0,0,0},tcp},ejabberd_c2s,zlib],10000}.
{[l,listener_opt,{5222,{0,0,0,0},tcp},ejabberd_c2s,access],c2s}.
{[l,listener_opt,{5222,{0,0,0,0},tcp},ejabberd_c2s,shaper],c2s_shaper}.
{[l,listener_opt,{5222,{0,0,0,0},tcp},ejabberd_c2s,max_stanza_size],65536}.
{[l,listener_opt,{5222,{0,0,0,0},tcp},ejabberd_c2s,protocol_options],
["no_sslv3"]}.
{[l,listener_opt,{5222,{0,0,0,0},tcp},ejabberd_c2s,dhfile],
"priv/ssl/fake_dh_server.pem"}.
{[l,listener,{5223,{0,0,0,0},tcp},ejabberd_c2s],'FLAT'}.
{[l,listener_opt,{5223,{0,0,0,0},tcp},ejabberd_c2s,zlib],4096}.
{[l,listener_opt,{5223,{0,0,0,0},tcp},ejabberd_c2s,access],c2s}.
{[l,listener_opt,{5223,{0,0,0,0},tcp},ejabberd_c2s,shaper],c2s_shaper}.
{[l,listener_opt,{5223,{0,0,0,0},tcp},ejabberd_c2s,max_stanza_size],65536}.
{[l,listener,{5269,{0,0,0,0},tcp},ejabberd_s2s_in],'FLAT'}.
{[l,listener_opt,{5269,{0,0,0,0},tcp},ejabberd_s2s_in,shaper],s2s_shaper}.
{[l,listener_opt,{5269,{0,0,0,0},tcp},ejabberd_s2s_in,max_stanza_size],131072}.
{[l,listener_opt,{5269,{0,0,0,0},tcp},ejabberd_s2s_in,protocol_options],
["no_sslv3"]}.
{[l,listener_opt,{5269,{0,0,0,0},tcp},ejabberd_s2s_in,dhfile],
"priv/ssl/fake_dh_server.pem"}.
{[l,listener,{8888,{127,0,0,1},tcp},ejabberd_service],'FLAT'}.
{[l,listener_opt,{8888,{127,0,0,1},tcp},ejabberd_service,access],all}.
{[l,listener_opt,{8888,{127,0,0,1},tcp},ejabberd_service,shaper_rule],fast}.
{[l,listener_opt,{8888,{127,0,0,1},tcp},ejabberd_service,password],"secret"}.
{[l,listener,{8189,{127,0,0,1},tcp},ejabberd_service],'FLAT'}.
{[l,listener_opt,{8189,{127,0,0,1},tcp},ejabberd_service,access],all}.
{[l,listener_opt,{8189,{127,0,0,1},tcp},ejabberd_service,hidden_components],
true}.
{[l,listener_opt,{8189,{127,0,0,1},tcp},ejabberd_service,shaper_rule],fast}.
{[l,listener_opt,{8189,{127,0,0,1},tcp},ejabberd_service,password],"secret"}.
{[l,all_metrics_are_global],false}.
{[l,s2s_certfile],"priv/ssl/fake_server.pem"}.
{[l,node_start],{1530,15976,143119}}.
{[l,odbc_pools],[]}.
{[l,registration_timeout],infinity}.
{[l,outgoing_s2s_port],5299}.
{[l,services],
[{service_admin_extra,[{submods,[node,accounts,sessions,vcard,roster,last,
private,stanza,stats]}]}]}.
{[l,max_fsm_queue],1000}.
{[l,s2s_use_starttls],optional}.
{[h,<<"anonymous.localhost">>,auth_method],anonymous}.
{[h,<<"anonymous.localhost">>,s2s_default_policy],allow}.
{[h,<<"localhost.bis">>,modules],'FLAT'}.
{[h,<<"localhost.bis">>,module,mod_carboncopy],'FLAT'}.
{[h,<<"localhost.bis">>,module,mod_stream_management],'FLAT'}.
{[h,<<"localhost.bis">>,module,mod_muc_commands],'FLAT'}.
{[h,<<"localhost.bis">>,module,mod_amp],'FLAT'}.
{[h,<<"localhost.bis">>,module,mod_offline],'FLAT'}.
{[h,<<"localhost.bis">>,module_opt,mod_offline,access_max_user_messages],
max_user_offline_messages}.
{[h,<<"localhost.bis">>,module,mod_last],'FLAT'}.
{[h,<<"localhost.bis">>,module,mod_roster],'FLAT'}.
{[h,<<"localhost.bis">>,module,mod_bosh],'FLAT'}.
{[h,<<"localhost.bis">>,module,mod_blocking],'FLAT'}.
{[h,<<"localhost.bis">>,module,mod_vcard],'FLAT'}.
{[h,<<"localhost.bis">>,module_opt,mod_vcard,host],"vjud.@HOST@"}.
{[h,<<"localhost.bis">>,module,mod_muc_light_commands],'FLAT'}.
{[h,<<"localhost.bis">>,module,mod_commands],'FLAT'}.
{[h,<<"localhost.bis">>,module,mod_disco],'FLAT'}.
{[h,<<"localhost.bis">>,module_opt,mod_disco,users_can_see_hidden_services],
false}.
{[h,<<"localhost.bis">>,module,mod_privacy],'FLAT'}.
{[h,<<"localhost.bis">>,module,mod_private],'FLAT'}.
{[h,<<"localhost.bis">>,module,mod_sic],'FLAT'}.
{[h,<<"localhost.bis">>,module,mod_adhoc],'FLAT'}.
{[h,<<"localhost.bis">>,module,mod_register],'FLAT'}.
{[h,<<"localhost.bis">>,module_opt,mod_register,welcome_message],{[]}}.
{[h,<<"localhost.bis">>,module_opt,mod_register,ip_access],'FLAT'}.
{[h,<<"localhost.bis">>,module_subopt,mod_register,ip_access,allow],
"127.0.0.0/8"}.
{[h,<<"localhost.bis">>,module_subopt,mod_register,ip_access,deny],
"0.0.0.0/0"}.
{[h,<<"localhost.bis">>,module_opt,mod_register,access],register}.
{[h,<<"localhost">>,modules],'FLAT'}.
{[h,<<"localhost">>,module,mod_carboncopy],'FLAT'}.
{[h,<<"localhost">>,module,mod_stream_management],'FLAT'}.
{[h,<<"localhost">>,module,mod_muc_commands],'FLAT'}.
{[h,<<"localhost">>,module,mod_amp],'FLAT'}.
{[h,<<"localhost">>,module,mod_offline],'FLAT'}.
{[h,<<"localhost">>,module_opt,mod_offline,access_max_user_messages],
max_user_offline_messages}.
{[h,<<"localhost">>,module,mod_last],'FLAT'}.
{[h,<<"localhost">>,module,mod_roster],'FLAT'}.
{[h,<<"localhost">>,module,mod_bosh],'FLAT'}.
{[h,<<"localhost">>,module,mod_blocking],'FLAT'}.
{[h,<<"localhost">>,module,mod_vcard],'FLAT'}.
{[h,<<"localhost">>,module_opt,mod_vcard,host],"vjud.@HOST@"}.
{[h,<<"localhost">>,module,mod_muc_light_commands],'FLAT'}.
{[h,<<"localhost">>,module,mod_commands],'FLAT'}.
{[h,<<"localhost">>,module,mod_disco],'FLAT'}.
{[h,<<"localhost">>,module_opt,mod_disco,users_can_see_hidden_services],false}.
{[h,<<"localhost">>,module,mod_privacy],'FLAT'}.
{[h,<<"localhost">>,module,mod_private],'FLAT'}.
{[h,<<"localhost">>,module,mod_sic],'FLAT'}.
{[h,<<"localhost">>,module,mod_adhoc],'FLAT'}.
{[h,<<"localhost">>,module,mod_register],'FLAT'}.
{[h,<<"localhost">>,module_opt,mod_register,welcome_message],{[]}}.
{[h,<<"localhost">>,module_opt,mod_register,ip_access],'FLAT'}.
{[h,<<"localhost">>,module_subopt,mod_register,ip_access,allow],"127.0.0.0/8"}.
{[h,<<"localhost">>,module_subopt,mod_register,ip_access,deny],"0.0.0.0/0"}.
{[h,<<"localhost">>,module_opt,mod_register,access],register}.
{[h,<<"localhost">>,auth_method],internal}.
{[h,<<"anonymous.localhost">>,auth_opts],[]}.
{[h,<<"fed1">>,s2s_addr],{127,0,0,1}}.
{[h,<<"localhost.bis">>,auth_opts],[]}.
{[h,<<"localhost">>,auth_opts],[]}.
{[h,<<"anonymous.localhost">>,modules],'FLAT'}.
{[h,<<"anonymous.localhost">>,module,mod_carboncopy],'FLAT'}.
{[h,<<"anonymous.localhost">>,module,mod_stream_management],'FLAT'}.
{[h,<<"anonymous.localhost">>,module,mod_muc_commands],'FLAT'}.
{[h,<<"anonymous.localhost">>,module,mod_amp],'FLAT'}.
{[h,<<"anonymous.localhost">>,module,mod_offline],'FLAT'}.
{[h,<<"anonymous.localhost">>,module_opt,mod_offline,access_max_user_messages],
max_user_offline_messages}.
{[h,<<"anonymous.localhost">>,module,mod_last],'FLAT'}.
{[h,<<"anonymous.localhost">>,module,mod_roster],'FLAT'}.
{[h,<<"anonymous.localhost">>,module,mod_bosh],'FLAT'}.
{[h,<<"anonymous.localhost">>,module,mod_blocking],'FLAT'}.
{[h,<<"anonymous.localhost">>,module,mod_vcard],'FLAT'}.
{[h,<<"anonymous.localhost">>,module_opt,mod_vcard,host],"vjud.@HOST@"}.
{[h,<<"anonymous.localhost">>,module,mod_muc_light_commands],'FLAT'}.
{[h,<<"anonymous.localhost">>,module,mod_commands],'FLAT'}.
{[h,<<"anonymous.localhost">>,module,mod_disco],'FLAT'}.
{[h,<<"anonymous.localhost">>,module_opt,mod_disco,
users_can_see_hidden_services],
false}.
{[h,<<"anonymous.localhost">>,module,mod_privacy],'FLAT'}.
{[h,<<"anonymous.localhost">>,module,mod_private],'FLAT'}.
{[h,<<"anonymous.localhost">>,module,mod_sic],'FLAT'}.
{[h,<<"anonymous.localhost">>,module,mod_adhoc],'FLAT'}.
{[h,<<"anonymous.localhost">>,module,mod_register],'FLAT'}.
{[h,<<"anonymous.localhost">>,module_opt,mod_register,welcome_message],{[]}}.
{[h,<<"anonymous.localhost">>,module_opt,mod_register,ip_access],'FLAT'}.
{[h,<<"anonymous.localhost">>,module_subopt,mod_register,ip_access,allow],
"127.0.0.0/8"}.
{[h,<<"anonymous.localhost">>,module_subopt,mod_register,ip_access,deny],
"0.0.0.0/0"}.
{[h,<<"anonymous.localhost">>,module_opt,mod_register,access],register}.
{[h,<<"anonymous.localhost">>,allow_multiple_connections],true}.
{[h,<<"localhost.bis">>,auth_method],internal}.
{[h,<<"localhost.bis">>,s2s_default_policy],allow}.
{[h,<<"localhost">>,s2s_default_policy],allow}.
{[h,<<"anonymous.localhost">>,anonymous_protocol],both}.
```