ejabberd 18.03 sends server cerfile instead of the domin_cerfile as specified in the jid domain_part

[cmeng-git]

What version of ejabberd are you using?
ejabberd 18,03

What operating system (version) are you using?
ubuntu 16.04-4

How did you install ejabberd (source, package, distribution)?
Source build

What did not work as expected? Are there error messages in the log? What
Server/Host atalk.sytes.net provides the following three xmpp domains' services, each with its own certfiles defined as below in ejabberd.yml i.e.

hosts:

• "atalk.org"
• "atalk.sytes.net"
• "icrypto.com"

certfiles:

• "/usr/local/etc/ejabberd/atalk.org.pem"
• "/usr/local/etc/ejabberd/atalk.sytes.net.pem"
• "/usr/local/etc/ejabberd/icrypto.pem"

With the above configurations, ejabberd always send users e.g. aaa123@atalk.org and xyz123@icrypto.com the atalk.sytes.net.pem instead of the specific correct domain_certfile.

was the unexpected behavior? What was the expected result?
ejabberd should send the domain_certfile according to the user domain_part instead of the actual server certfile.

[zinid]

ejabberd should send the domain_certfile according to the user domain_part instead of the actual server certfile

I don't think so. The option domain_certfile is deprecated and will be probably removed in the future. The reason the server picks incorrect certificate file most likely means ejabberd was unable to read some of your certificate files.

[cmeng-git]

I have made changes to some incorrect observation in my last comments. The problem only happen on atalk.org domain.

If I commented out atalk.sytes.net.pem in certfiles section i.e.
## - "/usr/local/etc/ejabberd/atalk.sytes.net.pem"

Then ejabberd 18.03 uses the correct atalk.org.pem cerfile when establishing the session with aaa123@atalk.org.

When user on aTalk client first login, it will display the exact content of the ssl cert for user approval. The above observation is reflected exactly in the aTalk certification display content for the two test cases.

The atalk.org.pem content is shown below:
cmeng@engsvr:/usr/local/etc/ejabberd$ openssl x509 -text -noout -in atalk.org.pem

Certificate:
    Data:
        Version: 3 (0x2)
        Serial Number: 18070839554214568913 (0xfac884d4c609a3d1)
    Signature Algorithm: sha256WithRSAEncryption
        Issuer: CN=atalk.org
        Validity
            Not Before: Dec 26 01:07:16 2015 GMT
            Not After : Dec 23 01:07:16 2025 GMT
        Subject: CN=atalk.org
        Subject Public Key Info:
            Public Key Algorithm: rsaEncryption
                Public-Key: (2048 bit)
                Modulus:
                    00:e1:ed:fc:01:59:35:a0:0e:59:d1:7a:4c:f7:77:
                    66:af:0d:fb:b3:9f:2d:49:75:7f:b7:02:3b:dc:26:
                    2b:98:21:48:f1:dc:8a:9c:e5:f7:86:59:aa:ac:23:
                    a3:ee:ff:d4:95:28:20:63:24:32:df:2f:2f:14:22:
                    92:e7:dc:5d:d1:23:8b:b5:b9:29:a9:94:20:b5:8b:
                    5c:81:91:42:17:b8:bf:e5:9e:97:ca:95:60:0e:2a:
                    aa:67:b9:cc:ed:84:7a:3c:d1:2c:60:30:02:e0:c2:
                    94:76:75:d7:f2:26:10:87:2e:a8:9a:98:ff:3a:da:
                    c3:65:08:b2:ab:35:22:9c:8c:ff:83:0c:ba:3d:ec:
                    d4:21:6c:b1:b4:5b:94:3a:28:d8:a0:e2:9b:16:66:
                    fc:01:00:85:92:db:77:2e:0e:96:85:27:4d:b8:39:
                    e3:8e:18:4f:ef:cb:0e:88:09:b1:38:6a:3e:27:d6:
                    68:ec:98:82:2d:4b:93:cc:9f:50:78:47:c1:1d:7b:
                    dd:68:ad:83:15:33:1c:e3:f7:34:99:fb:fe:09:2e:
                    72:9d:59:c7:32:81:c4:f6:1f:44:db:e8:57:ad:75:
                    5a:26:7f:34:a6:a1:45:92:1c:b9:33:d2:9d:50:7f:
                    41:45:86:6c:10:61:c4:d6:7c:fb:8f:a6:31:37:71:
                    5d:ef
                Exponent: 65537 (0x10001)
        X509v3 extensions:
            X509v3 Basic Constraints: 
                CA:FALSE
    Signature Algorithm: sha256WithRSAEncryption
         4b:17:76:7f:b8:1d:53:48:6c:05:e8:d3:fe:14:0b:c9:d3:36:
         99:ae:f9:c9:fb:6e:5a:af:87:de:c6:e4:8b:21:84:bb:7a:13:
         69:2c:2b:81:3a:33:b1:c1:fd:c8:dd:50:44:66:a9:25:85:ae:
         e5:13:a0:74:bf:99:0a:12:8c:ec:a2:68:1e:1d:33:f5:b6:77:
         71:2d:c1:fe:94:04:8b:17:2d:50:a3:ee:17:10:fb:75:13:93:
         46:82:bb:ea:42:75:ea:50:57:52:80:f4:71:6e:86:b9:f2:d0:
         61:4d:d6:45:d3:f1:69:d5:51:ba:4a:5d:77:42:3e:a9:b9:1d:
         f9:3e:73:31:84:d0:b8:2a:53:7b:dc:c3:6b:25:d6:43:0a:52:
         bf:3c:90:f1:7b:d9:5d:cc:6e:eb:62:c1:40:f1:81:e9:ab:9f:
         7c:af:16:a1:29:e5:7e:53:8d:9d:07:f2:d7:8d:84:08:9c:1c:
         34:56:16:11:5b:de:6d:e8:74:2b:98:5b:77:1f:63:76:3b:d4:
         bc:95:d5:e3:1b:d8:93:28:36:cd:c1:cf:a7:6e:76:b7:c0:b9:
         c5:55:83:79:59:97:ec:a6:e4:ba:0b:2c:fc:e8:dc:94:d1:13:
         31:cb:3c:c5:0e:b8:32:0b:c1:af:9b:e6:06:83:4a:c5:7f:e4:
         9a:c1:7e:bc

[zinid]

So what? Why don't you think that's a problem of your client setting either SNI or doing STARTTLS incorrectly? Could you please check with some existing proven to be working clients such as Conversations?

[cmeng-git]

aTalk was fully working with ejabberd v17.08 without the reported problem.
aTalk uses Smack and the verification of the ssl cert is done within XMPPTCPConnection as below. It checks the ssl cert only with the Domain name.

To allow me to continue with the aTalk testing with ejabberd v18.03, I temporally comment out the atalk.sytes.net.pem cert.

public interface HostnameVerifier {
    /**
     * Verify that the host name is an acceptable match with
     * the server's authentication scheme.
     *
     * @param hostname the host name
     * @param session SSLSession used on the connection to host
     * @return true if the host name is acceptable
     */
    public boolean verify(String hostname, SSLSession session);
}

There is no problem with Conversations, because it checks the ssl cert against both the Domain and Host name. atalk.sytes.net is the FQDN of the ejabberd v18.03 server

public interface DomainHostnameVerifier extends HostnameVerifier {
    boolean verify(String domain, String hostname, SSLSession sslSession);
}

[zinid]

aTalk uses Smack and the verification of the ssl cert is done within XMPPTCPConnection as below. It checks the ssl cert only with the Domain name.

In that case there are hundreds of servers with much more sofisticated certificates configuration and they don't have such problems.
You could actually try to connect to ejabberd using openssl to check what certificate it provides.
Also, please stop attaching Java code or client screenshots, nobody reads them. Perform openssl command with starttls or server-name and show the output.

[cmeng-git]

The following is the output of openssl, but not sure if I use the cli correctly. The received cert seems correct for atalk.org. Look like I need to get back to Smack team for some advice.

My apology if the problem is indeed within aTalk client.

cmeng@engsvr:~$ openssl s_client -servername atalk.org -connect atalk.sytes.net:5222 -starttls xmpp
CONNECTED(00000003)
depth=0 CN = atalk.org
verify error:num=18:self signed certificate
verify return:1
depth=0 CN = atalk.org
verify return:1
---
Certificate chain
 0 s:/CN=atalk.org
   i:/CN=atalk.org
---
Server certificate
-----BEGIN CERTIFICATE-----
MIICuDCCAaCgAwIBAgIJAPrIhNTGCaPRMA0GCSqGSIb3DQEBCwUAMBQxEjAQBgNV
BAMTCWF0YWxrLm9yZzAeFw0xNTEyMjYwMTA3MTZaFw0yNTEyMjMwMTA3MTZaMBQx
EjAQBgNVBAMTCWF0YWxrLm9yZzCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoC
ggEBAOHt/AFZNaAOWdF6TPd3Zq8N+7OfLUl1f7cCO9wmK5ghSPHcipzl94ZZqqwj
o+7/1JUoIGMkMt8vLxQikufcXdEji7W5KamUILWLXIGRQhe4v+Wel8qVYA4qqme5
zO2EejzRLGAwAuDClHZ11/ImEIcuqJqY/zraw2UIsqs1IpyM/4MMuj3s1CFssbRb
lDoo2KDimxZm/AEAhZLbdy4OloUnTbg5444YT+/LDogJsThqPifWaOyYgi1Lk8yf
UHhHwR173WitgxUzHOP3NJn7/gkucp1ZxzKBxPYfRNvoV611WiZ/NKahRZIcuTPS
nVB/QUWGbBBhxNZ8+4+mMTdxXe8CAwEAAaMNMAswCQYDVR0TBAIwADANBgkqhkiG
9w0BAQsFAAOCAQEASxd2f7gdU0hsBejT/hQLydM2ma75yftuWq+H3sbkiyGEu3oT
aSwrgTozscH9yN1QRGapJYWu5ROgdL+ZChKM7KJoHh0z9bZ3cS3B/pQEixctUKPu
FxD7dROTRoK76kJ16lBXUoD0cW6GufLQYU3WRdPxadVRukpdd0I+qbkd+T5zMYTQ
uCpTe9zDayXWQwpSvzyQ8XvZXcxu62LBQPGB6auffK8WoSnlflONnQfy142ECJwc
NFYWEVvebeh0K5hbdx9jdjvUvJXV4xvYkyg2zcHPp252t8C5xVWDeVmX7Kbkugss
/OjclNETMcs8xQ64MgvBr5vmBoNKxX/kmsF+vA==
-----END CERTIFICATE-----
subject=/CN=atalk.org
issuer=/CN=atalk.org
---
No client certificate CA names sent
Client Certificate Types: RSA sign, DSA sign, ECDSA sign
Requested Signature Algorithms: RSA+SHA512:DSA+SHA512:ECDSA+SHA512:RSA+SHA384:DSA+SHA384:ECDSA+SHA384:RSA+SHA256:DSA+SHA256:ECDSA+SHA256:RSA+SHA224:DSA+SHA224:ECDSA+SHA224:RSA+SHA1:DSA+SHA1:ECDSA+SHA1
Shared Requested Signature Algorithms: RSA+SHA512:DSA+SHA512:ECDSA+SHA512:RSA+SHA384:DSA+SHA384:ECDSA+SHA384:RSA+SHA256:DSA+SHA256:ECDSA+SHA256:RSA+SHA224:DSA+SHA224:ECDSA+SHA224:RSA+SHA1:DSA+SHA1:ECDSA+SHA1
Peer signing digest: SHA512
Server Temp Key: ECDH, P-256, 256 bits
---
SSL handshake has read 1701 bytes and written 632 bytes
---
New, TLSv1/SSLv3, Cipher is ECDHE-RSA-AES256-GCM-SHA384
Server public key is 2048 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
SSL-Session:
    Protocol  : TLSv1.2
    Cipher    : ECDHE-RSA-AES256-GCM-SHA384
    Session-ID: 
    Session-ID-ctx: 
    Master-Key: 2AD005959D7AD5C87D77DC5BE00D9702CB0F6D019286503B3B6EA58E9911150392FE7121222DE2E05F85F44EED878CB1
    Key-Arg   : None
    PSK identity: None
    PSK identity hint: None
    SRP username: None
    Start Time: 1523198048
    Timeout   : 300 (sec)
    Verify return code: 18 (self signed certificate)
---
read:errno=0

[cmeng-git]

Below is the captured ejabberd.log showing the initial login TLS handshake between atalk.org client and ejabberd v18.03 server. I also captured the TLSv1.2 protocol exchanges using wireshark between the two entities and check for the certificate content sent from ejabberd v18.03.

The actual ssl cert sent by ejabberd v18.03 during the xmpp TLS handshake is different from when using openssl cli.

====== ejabberd.log =======
00:52:29.234 [debug] (tcp|<0.662.0>) Received XML on stream = <<"<stream:stream xmlns='jabber:client' to='atalk.org' xmlns:stream='http://etherx.jabber.org/streams' version='1.0' xml:lang='en'>">>
00:52:29.234 [debug] (tcp|<0.662.0>) Send XML on stream = <<"<?xml version='1.0'?><stream:stream id='9586934371911915521' version='1.0' xml:lang='en' xmlns:stream='http://etherx.jabber.org/streams' from='atalk.org' xmlns='jabber:client'>">>
00:52:29.234 [debug] (tcp|<0.662.0>) Send XML on stream = <<"<stream:features><starttls xmlns='urn:ietf:params:xml:ns:xmpp-tls'><required/></starttls></stream:features>">>
00:52:29.242 [debug] (tcp|<0.662.0>) Received XML on stream = <<"<starttls xmlns='urn:ietf:params:xml:ns:xmpp-tls'></starttls>">>
00:52:29.242 [debug] State: {maxrate,1000,0.0,1523206349234484}, Size=61
M=30.5, I=8.229
00:52:29.242 [debug] (tcp|<0.662.0>) Send XML on stream = <<"<proceed xmlns='urn:ietf:params:xml:ns:xmpp-tls'/>">>

====== wireshark TLSv1.2 cer Info ========
issuer: rdnSequence (0)
rdnSequence: 1 item (id-at-commonName=atalk.sytes.net)
RDNSequence item: 1 item (id-at-commonName=atalk.sytes.net)
RelativeDistinguishedName item (id-at-commonName=atalk.sytes.net)
Id: 2.5.4.3 (id-at-commonName)
DirectoryString: printableString (1)
printableString: atalk.sytes.net

[zinid]

This is strange anyway, because for <starttls/> ejabberd tries to pick the certificate for the domain defined in the stream header.
Could you please check in wireshark what SNI the client sets? It's part of "extensions" of the very first packet sent by a client (aka "TLS Client Hello"), called "servername" or something like that.
Also, please provide your config file:

$ grep -v '^$\|^\s*\#' /path/to/ejabberd.yml

(with sensitive data removed).

[zinid]

It seems like your client sets atalk.sytes.net in SNI (instead of the domain). In terms of openssl command this is what happens:

$ openssl s_client -xmpphost atalk.org -connect atalk.sytes.net:5222 -starttls xmpp -servername atalk.sytes.net 
CONNECTED(00000003)
depth=0 CN = atalk.sytes.net
verify error:num=18:self signed certificate
verify return:1
depth=0 CN = atalk.sytes.net
verify return:1
Server did acknowledge servername extension.
---
Certificate chain
 0 s:/CN=atalk.sytes.net
   i:/CN=atalk.sytes.net

[cmeng-git]

You are correct. The client actually sets the serverName to atalk.sytes.net. I need to review the aTalk source and refer the issue to smack team for advice.

Extension: server_name
Server Name: atalk.sytes.net

Below is the ejabberd configurations

loglevel: 5
log_rotate_size: 10485760
log_rotate_date: ""
log_rotate_count: 1
log_rate_limit: 100
hosts:
  - "atalk.org"
  - "atalk.sytes.net"
  - "icrypto.com"
certfiles:
 - "/usr/local/etc/ejabberd/atalk.org.pem"
 - "/usr/local/etc/ejabberd/icrypto.pem"
 - "/usr/local/etc/ejabberd/atalk.sytes.net.pem"
listen:
  -
    port: 5222
    ip: "::"
    module: ejabberd_c2s
    starttls_required: true
    max_stanza_size: 65536
    shaper: c2s_shaper
    access: c2s
  -
    port: 5269
    ip: "::"
    module: ejabberd_s2s_in
  -
    port: 5280
    ip: "::"
    module: ejabberd_http
    request_handlers:
      "/ws": ejabberd_http_ws
      "/bosh": mod_bosh
      "/api": mod_http_api
    web_admin: true
    register: true
    captcha: true
  - 
    port: 8888
    ip: "::"
    module: ejabberd_service
    access: all
    shaper_rule: fast
    ip: "127.0.0.1"
    hosts:
      "jn.atalk.org":
        password: "#secret_#1"
  - 
    port: 5275
    module: ejabberd_service
    access: all
    shaper_rule: fast
    hosts:
      "jitsi-videobridge.atalk.org":
        password: "#secret_#2"
  - 
    port: 5347
    module: ejabberd_service
    access: all
    shaper_rule: fast
    hosts:
      "focus.atalk.org":
        password: "#secret_#3"
  - 
    port: 3478
    transport: udp
    module: ejabberd_stun
    use_turn: true
    turn_ip: "115.66.220.38"
    auth_type: user
    auth_realm: "atalk.sytes.net"
  -
    port: 5349
    module: ejabberd_stun
fqdn: "atalk.sytes.net"
host_config:
  "atalk.org":
    auth_method: sql
    auth_password_format: scram
  "atalk.sytes.net":
    auth_method: anonymous
    anonymous_protocol: both
    allow_multiple_connections: false
  "icrypto.com":
    auth_method: sql
    auth_password_format: scram
host_config:
   "atalk.org":
      sql_type: mysql
      sql_server: "localhost"
      sql_database: "ejb_atalk"
      sql_username: "root"
      sql_password: "#secret_#4"
host_config:
   "icrypto.com":
      sql_type: mysql
      sql_server: "localhost"
      sql_database: "ejb_icrypto"
      sql_username: "root"
      sql_password: "#secret_#5"
host_config:
   "atalk.sytes.net":
      sql_type: mysql
      sql_server: "localhost"
      sql_database: "ejabberd"
      sql_username: "root"
      sql_password: "#secret_#6"
shaper:
  normal: 1000
  fast: 50000
  proxyrate: 204800
max_fsm_queue: 10000
acl:
  admin:
    user:
      - "admin@atalk.org"
      - "focus@atalk.org"
      - "admin@icrypto.com"
  muc_admin:
    server:
      - "atalk.org"
      - "icrypto.com"
  muc_customer:
    server:
      - "atalk.org"
      - "icrypto.com"
  proxy_users:
    server:
      - "atalk.org"
      - "atalk.sytes.net"
      - "icrypto.com"
  local:
    user_regexp: ""
  loopback:
    ip:
      - "127.0.0.0/8"
      - "::1/128"
      - "::FFFF:127.0.0.1/128"
shaper_rules:
  max_user_sessions: 10
  max_user_offline_messages:
    - 5000: admin
    - 100
  c2s_shaper:
    - none: admin
    - normal
  s2s_shaper: fast
  proxy65_shaper:
    - none: admin
    - proxyrate: proxy_users
access_rules:
  local:
    - allow: local
  c2s:
    - deny: blocked
    - allow
  announce:
    - allow: admin
  configure:
    - allow: admin
  muc_admin:
    - allow: muc_admin
  muc_create:
    - allow: muc_customer
  muc_access: 
    - allow: muc_customer
  multicast:
    - allow
  pubsub_createnode: 
    - allow
  proxy65_access:
    - allow: proxy_users
  register:
    - allow
  trusted_network:
    - allow: loopback
api_permissions:
  "console commands":
    from:
      - ejabberd_ctl
    who: all
    what: "*"
  "admin access":
    who:
      - access:
          - allow:
            - acl: loopback
            - acl: admin
      - oauth:
        - scope: "ejabberd:admin"
        - access:
          - allow:
            - acl: loopback
            - acl: admin
    what:
      - "*"
      - "!stop"
      - "!start"
  "public commands":
    who:
      - ip: "127.0.0.1/8"
    what:
      - "status"
      - "connected_users_number"
  
language: "en"
captcha_cmd: "/usr/local/lib/ejabberd/priv/bin/captcha.sh"
captcha_host: "atalk.sytes.net:5280"
acme:
   contact: "mailto:example-admin@example.com"
   ca_url: "https://acme-v01.api.letsencrypt.org"
modules:
  mod_adhoc: {}
  mod_admin_extra: {}
  mod_announce: # recommends mod_adhoc
    db_type: sql
    access: announce
  mod_blocking: {} # requires mod_privacy
  mod_caps: {}
  mod_carboncopy: {}
  mod_client_state: {}
  mod_configure: {} # requires mod_adhoc
  mod_disco: {}
  mod_irc: ## {}
    db_type: sql
  mod_bosh: {}
  mod_last: # {}
    db_type: sql
  mod_mam: # {}
    db_type: sql
    default: never
  mod_muc:
    db_type: sql
    host: "conference.@HOST@"
    access:
      - allow
    access_admin:
      - allow: admin
    access_create: muc_create
    access_persistent: muc_create
    history_size: 20
    max_users: 200
    min_message_interval: 0.4
    min_presence_interval: 4
    default_room_options:
      allow_private_messages: true
      allow_user_invites: true
      anonymous: false
      captcha_protected: true
      mam: true
      persistent: false
      public: false
      public_list: false
  mod_muc_admin: {}
  mod_muc_log: ## {}
    access_log: muc_admin
    timezone: universal ## local/universal  
  mod_offline:
    db_type: sql 
    access_max_user_messages: max_user_offline_messages
  mod_ping: # {}
    send_pings: true
    ping_interval: 240
    timeout_action: none
  mod_privacy: ## {}
    db_type: sql
  mod_private: ## {}
    db_type: sql
  mod_proxy65: ## {}
    name: "File Transfer Proxy"
    ip: "0.0.0.0"
    hostname: "115.66.84.204"
    port: 7777
    max_connections: 5
    access: proxy65_access
    shaper: proxy65_shaper
  mod_pubsub:
    db_type: sql
    access_createnode: pubsub_createnode
    ignore_pep_from_offline: true
    last_item_cache: false
    plugins:
      - "flat"
      - "hometree"
      - "pep" # pep requires mod_caps
  mod_push: {}
  mod_push_keepalive: {}
  mod_register: 
    captcha_protected: true
    password_strength: 32
    welcome_message: 
      subject: "Welcome!"
      body: |-
        Welcome to aTalk XMPP server. 
        For information visit: http://atalk.sytes.net
    registration_watchers:
      - "admin@atalk.org"
    ip_access: all
    access: register
  mod_roster: ## {}
    db_type: sql
    versioning: true
  mod_shared_roster: ## {}
    db_type: sql
  mod_sic: {}
  mod_stats: {}
  mod_time: {}
  mod_vcard: ## {}
    db_type: sql
    search: false
  mod_vcard_xupdate: ## {}
    use_cache: true
    cache_size: 1000  # size of entries
    cache_life_time: 604800 # maximum life time of cached data e.g. 7 day=7*24*3600  
  mod_version: {}
  mod_stream_mgmt: ## {}
    resend_on_timeout: true
  mod_s2s_dialback: {}
  mod_http_api: {}
  mod_fail2ban: {}
host_config:
  "atalk.sytes.net":
    modules:
      mod_disco: {}
      mod_bosh: {}
      mod_muc:
        db_type: sql
        host: "conference.@HOST@"
        access: all
        access_create: all
        access_persistent: all
        access_admin: focus@atalk.org
        history_size: 0
        max_users: 200
        default_room_options:
          allow_private_messages: true
          anonymous: false
          public: true
          public_list: false
          persistent: false
allow_contrib_modules: true

[cmeng-git]

Hi Zinid,
Thanks for your guidance and support.

Smack team has proposed a working solution. See link below:

https://discourse.igniterealtime.org/t/smack-4-2-3-always-uses-hostname-instead-of-domainname-when-requesting-tls-certificate-from-ejabberd-server/81308

[lock]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.