Custom client-cert verification during TLS handshake

[vaibhavmurkute]

Hello,

I see emqx's listeners are started before loading plugins/modules: link
Once started, is it possible to update a listener's configuration (e.g. SslOptions) afterwards?

The use-case I have in mind is to expose a hookpoint on TLS handshake (custom-certificate-verify), with a rough flow like this:

1. Plugin adds custom-verify function with a hook like this:

emqx:hook('tls.handshake.custom_verify',  {?MODULE, custom_verify_fun, [Env]})

2. Once plugins are loaded, EMQX looks for 'tls.handshake.custom_verify' hooks, gets the callback_action for the hook:

Callbacks = emqx_hooks:lookup('tls.handshake.custom_verify'),
FirstCallback = lists:nth(1, Callbacks),   % consider only first callback for now
custom_verify_fun = emqx_hooks:callback_action(FirstCallback)

3. Update/restart mqtt:ssl listener with verify_fun option added in the ssl_options.

{
	"SslOptions" : {
	
		{"verify_fun", custom_verify_fun, []}
	}
}

Does this approach look good? Is it possible to update the mqtt:ssl listener's configuration (e.g. SslOptions) after we load plugins?

EDIT:
I wonder if a plugin can call emqx_listeners:update_listeners_env/2 function to have ssl_options updated for mqtt:ssl listener.

Thanks!

[vaibhavmurkute]

One way to do this could be: restart the listeners with updated config by calling emqx_listener API from a plugin. No hooks.

[vaibhavmurkute]

Here is the plugin code I used to verify this concept. This plugin adds a custom cert-verify to the TLS listener (to be called during TLS handshake) and restarts the listener:

%% Called when the plugin application start
load(Env) ->
    NewSslListener = update_ssl_listener(),
    emqx_listeners:update_listeners_env('update', NewSslListener),
    io:format("~n#### Stopping SSL listener: ~p~n", [emqx_listeners:stop_listener(NewSslListener)]),
    io:format("~n#### Starting SSL listener: ~p~n", [emqx_listeners:start_listener(NewSslListener)]).

update_ssl_listener() ->
	Listeners = emqx:get_env(listeners, []),
	SslListener = find_ssl_listener(Listeners),
	ListenerOpts = maps:get('opts', SslListener),
	NewListenerOpts = add_custom_verify_to_ssl_options(ListenerOpts),
	maps:put('opts', NewListenerOpts, SslListener).

find_ssl_listener([]) -> false;
find_ssl_listener([#{name := "external", proto := 'ssl'} = L | _]) -> L;
find_ssl_listener([_ | Rest]) -> find_ssl_listener(Rest).

add_custom_verify_to_ssl_options(Options) ->
	case proplists:get_value(ssl_options, Options) of
		undefined -> Options;
		SslOpts ->
			NewSslOpts = lists:append(SslOpts,
						  [{verify_fun,
						    {fun(OtpCert, {bad_cert, _} = Reason, UserState) ->
								     io:format("~n ### bad_cert during TLS handshake: ~p~n", [Reason]),
								     io:format("~n ### Validate Certificate: ~p~n", [OtpCert]),
								     {valid, UserState};
							(_,{extension, _}, UserState) ->
								     {unknown, UserState};
							(_, valid, UserState) ->
								     {valid, UserState};
							(_, valid_peer, UserState) ->
								     {valid, UserState}
						     end, []}}]),
			replace(Options, ssl_options, NewSslOpts)
	end.

replace(Opts, Key, Value) -> [{Key, Value} | proplists:delete(Key, Opts)].

[zmstone]

Maybe better to have a different plugin mechanism for verify functions.

In emqx code, we always inject function fun emqx_tls_lib:verify_fun/3 to the common ssl options. And in this function, it looks up a ets table or persistent_term to which plugins can inject/override/splice other callbacks.

this will allow callbacks to be plugable without having to restart the listeners

[MikeDombo]

Any update on this functionality? Would be really great to expose this to plugins for custom authentication.

[id]

@qzhuyan it sounds similar to what you are currently working on, or?

[MikeDombo]

@id if you're thinking about this PR (#10553) then, no. That PR is for EMQX 4.4, not EMQX 5 and it doesn't let plugins add custom logic as a verify_fun.

[qzhuyan]

there are indeed different requirements on verify_fun looks like we need some design on a chain of verify fun.
For customized verify fun, I guess having a callback module as the starting point should be sufficient for most of cases, not necessarily a plugin?

[MikeDombo]

I'm sure there is a distinction that I don't fully understand between a plugin and a callback module; but in any case, some way to register a custom verify_fun which would receive the client cert and allow the implementation to choose to allow, deny, or ignore (go to next in chain).

[keynslug]

@MikeDombo, would you mind sharing your use case, what you want to achieve with custom verify_fun?

Sidenote. If only the peer certificate is needed, one could for example:

1. Configure listener with verify_none.
2. Hook into client.connect and retrieve peercert from ConnInfo (it should be available here again with this fix, planned for inclusion in the next release).
3. Alternatively, hook into client.authenticate and retrieve subject / common name from ClientInfo, or even the whole peercert depending on the value of peer_cert_as_username / peer_cert_as_username configuration options. Both of those hooks may give feedback to the core logic.

[MikeDombo]

Things may have changed with EMQX 5, though I am skeptical. But, we tried those with EMQX 4 and none of it worked.

Using verify_none meant that we didn't receive the peer cert at all, if I recall correctly. So none of those options that you listed would work because the peer cert was never provided. And if we kept verify_peer, then the peer cert would be rejected before we could make a decision because the peer cert wasn't trusted by the CA list configured in EMQX.

[keynslug]

You're right. Sadly, that currently won't work because TLS stack won't ask for a client certificate with verify_none.