Cannot load usrloc table at startup using DMQ_USRLOC sync feature

[beneditomarques]

I have an scenario with two kamailios, and no databases. At the first, I authenticate one subscriber successfully and save in memory usrloc:

/etc/kamailio # kamailio --version
version: kamailio 5.4.4 (x86_64/linux)
flags: USE_TCP, USE_TLS, USE_SCTP, TLS_HOOKS, USE_RAW_SOCKS, DISABLE_NAGLE, USE_MCAST, DNS_IP_HACK, SHM_MMAP, PKG_MALLOC, Q_MALLOC, F_MALLOC, TLSF_MALLOC, DBG_SR_MEMORY, USE_FUTEX, FAST_LOCK-ADAPTIVE_WAIT, USE_DNS_CACHE, USE_DNS_FAILOVER, USE_NAPTR, USE_DST_BLACKLIST, HAVE_RESOLV_RES, TLS_PTHREAD_MUTEX_SHARED
ADAPTIVE_WAIT_LOOPS 1024, MAX_RECV_BUFFER_SIZE 262144, MAX_URI_SIZE 1024, BUF_SIZE 65535, DEFAULT PKG_SIZE 8MB
poll method support: poll, epoll_lt, epoll_et, sigio_rt, select.
id: unknown
compiled on 16:13:18 Feb 15 2021 with gcc 9.3.0
/etc/kamailio #
/etc/kamailio #

/etc/kamailio # cat /etc/os-release
NAME="Alpine Linux"
ID=alpine
VERSION_ID=3.12.3
PRETTY_NAME="Alpine Linux v3.12"
HOME_URL="https://alpinelinux.org/"
BUG_REPORT_URL="https://bugs.alpinelinux.org/"

 # Handle SIP registrations
route[REGISTRAR] {
	if (!is_method("REGISTER")) return;

	if (!save("location","0x05")) {
		sl_reply_error();
	}
	exit;
}

--

/etc/kamailio # kamcmd ul.dump
{
        Domains: {
                Domain: {
                        Domain: location
                        Size: 1024
                        AoRs: {
                                Info: {
                                        AoR: 1014@my.domain
                                        HashID: -1989102610
                                        Contacts: {
                                                Contact: {
                                                        Address: sip:1014@10.0.0.2:7071;ob
                                                        Expires: 299
                                                        Q: -1.000000
                                                        Call-ID: d5cf6d5f293647a59ba64ba996a793b0
                                                        CSeq: 43689
                                                        User-Agent: MicroSIP/3.20.7
                                                        Received: sip:10.0.0.2:7071
                                                        Path: [not set]
                                                        State: CS_NEW
                                                        Flags: 1
                                                        CFlags: 0
                                                        Socket: [not set]
                                                        Methods: 8159
                                                        Ruid: uloc-62d95ca2-3c-1
                                                        Instance: [not set]
                                                        Reg-Id: 0
                                                        Server-Id: 0
                                                        Tcpconn-Id: -1
                                                        Keepalive: 0
                                                        Last-Keepalive: 1658412252
                                                        KA-Roundtrip: 0
                                                        Last-Modified: 1658412252
                                                }
                                        }
                                }
                        }
                        Stats: {
                                Records: 1
                                Max-Slots: 1
                        }
                }
        }
}
/etc/kamailio # 

The problem is: when I start the second kamailio server, the current usrloc users are not replicated to it, so in the second server the usrloc is empty:

/etc/kamailio # kamcmd ul.dump
{
        Domains: {
                Domain: {
                        Domain: location
                        Size: 1024
                        AoRs: {
                        }
                        Stats: {
                                Records: 0
                                Max-Slots: 0
                        }
                }
        }
}

The modules loaded:

• dmq.so
• dmq_usrloc.so

Params (sync enabled):

modparam("dmq_usrloc", "enable", 1)
modparam("dmq_usrloc", "sync", 1)
modparam("dmq_usrloc", "usrloc_domain", "location")
modparam("dmq_usrloc", "usrloc_delete", 1)
modparam("dmq_usrloc", "replicate_socket_info", 0)

Obs: The DMQ replication is working perfectly. So, if I register another user in server 1, server 2 receives the KDMQ message and the new user appears on usrloc. The real problem is to load pre registered users at startup.

Log messages from second server at startup:

0(51) DEBUG: <core> [core/sr_module.c:873]: init_mod(): dmq_usrloc
0(51) INFO: dmq_usrloc [dmq_usrloc.c:81]: mod_init(): dmq usrloc replication mode = 1
0(51) DEBUG: <core> [core/sr_module.c:652]: find_mod_export_record(): found export of <ul_bind_usrloc> in module usrloc [/usr/lib/kamailio/modules/usrloc.so]
0(51) DEBUG: <core> [core/sr_module.c:652]: find_mod_export_record(): found export of <bind_dmq> in module dmq [/usr/lib/kamailio/modules/dmq.so]
0(51) DEBUG: dmq_usrloc [usrloc_sync.c:286]: usrloc_dmq_initialize(): loaded dmq api
0(51) DEBUG: dmq_usrloc [usrloc_sync.c:299]: usrloc_dmq_initialize(): dmq peer registered
0(51) DEBUG: dmq_usrloc [dmq_usrloc.c:111]: mod_init(): dmq_usrloc initialized

0(51) DEBUG: <core> [core/sr_module.c:804]: init_mod_child(): idx 0 rank -127: dmq_usrloc [main]
0(51) DEBUG: dmq_usrloc [dmq_usrloc.c:127]: child_init(): child_init PROC_INIT

7(62) DEBUG: dmq_usrloc [usrloc_sync.c:552]: usrloc_dmq_request_sync(): requesting sync from dmq peers
0(51) DEBUG: <core> [core/sr_module.c:804]: init_mod_child(): idx 0 rank 0: dialog [main]
7(62) DEBUG: dmq_usrloc [usrloc_sync.c:568]: usrloc_dmq_request_sync(): sending serialized data {"action":3}
7(62) DEBUG: dmq_usrloc [usrloc_sync.c:317]: usrloc_dmq_send(): sending dmq broadcast...

7(62) DEBUG: dmq [dmq_funcs.c:166]: bcast_dmq_message1(): skipping node sip:10.0.2.38:5090
7(62) DEBUG: dmq [dmq_funcs.c:166]: bcast_dmq_message1(): skipping node sip:10.0.2.30:5090
7(62) DEBUG: dmq [dmq_funcs.c:166]: bcast_dmq_message1(): skipping node sip:0.0.0.0:5090

List nodes at SERVER 1:

/etc/kamailio # kamcmd dmq.list_nodes
{
        host: 10.0.1.36
        port: 5090
        resolved_ip: 10.0.1.36
        status: active
        last_notification: 0
        local: 0
}
{
        host: 10.0.1.35
        port: 5090
        resolved_ip: 10.0.1.35
        status: active
        last_notification: 0
        local: 0
}
{
        host: 0.0.0.0
        port: 5090
        resolved_ip: 0.0.0.0
        status: active
        last_notification: 0
        local: 1
}

List nodes at SERVER 2:

/etc/kamailio # kamcmd dmq.list_nodes
{
        host: 10.0.1.35
        port: 5090
        resolved_ip: 10.0.1.35
        status: active
        last_notification: 0
        local: 0
}
{
        host: 10.0.1.36
        port: 5090
        resolved_ip: 10.0.1.36
        status: active
        last_notification: 0
        local: 0
}
{
        host: 0.0.0.0
        port: 5090
        resolved_ip: 0.0.0.0
        status: active
        last_notification: 0
        local: 1
}

[miconda]

Try with latest Kamailio version and see if it is the same behaviour. It seems to be a matter of your configuration options and for that you have to email to sr-users@lists.kamailio.org mailing list, because you get there feedback from users that have DMQ on their systems. The tracker here is for reporting bugs in the C code and mostly are the developers taking care of it, many not using various modules.

If it proves to be an issue in the code, this can be reopen.