segfault due to inifinite loop in hmac branch

[Tomo59]

we are using babeld on hmac branch with the following options -C key type blake2s id sign value XXXXXXXXXXXXXXXXX -C default hmac sign no_hmac_verify true. From time to time, we see babeld crashing with a segfault.

We managed to have a coredump file, and the problem is flushbuf (called from main function) calls send_crypto_seqno which calls start_message which in turn calls flushbuf in a infinite loop.

Here are the information from gdb:

Core was generated by `babeld -h 15 -H 15 -L /var/log/re6stnet/babeld.log -S /var/lib/re6stnet/babeld.'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0  0x0000556e5c0f8b57 in send_crypto_seqno (buf=buf@entry=0x556e5d406e10, ifp=ifp@entry=0x556e5d3ff680) at message.c:1147
(gdb) bt
#0  0x0000556e5c0f8b57 in send_crypto_seqno (buf=buf@entry=0x556e5d406e10, ifp=ifp@entry=0x556e5d3ff680) at message.c:1147
#1  0x0000556e5c0f8c2b in flushbuf (buf=buf@entry=0x556e5d406e10, ifp=ifp@entry=0x556e5d3ff680) at message.c:1038
#2  0x0000556e5c0f8d95 in start_message (buf=buf@entry=0x556e5d406e10, ifp=ifp@entry=0x556e5d3ff680, type=type@entry=17, len=len@entry=12) at message.c:1103
#3  0x0000556e5c0f8b84 in send_crypto_seqno (buf=buf@entry=0x556e5d406e10, ifp=ifp@entry=0x556e5d3ff680) at message.c:1154
#4  0x0000556e5c0f8c2b in flushbuf (buf=buf@entry=0x556e5d406e10, ifp=ifp@entry=0x556e5d3ff680) at message.c:1038
#5  0x0000556e5c0f8d95 in start_message (buf=buf@entry=0x556e5d406e10, ifp=ifp@entry=0x556e5d3ff680, type=type@entry=17, len=len@entry=12) at message.c:1103
#6  0x0000556e5c0f8b84 in send_crypto_seqno (buf=buf@entry=0x556e5d406e10, ifp=ifp@entry=0x556e5d3ff680) at message.c:1154
#7  0x0000556e5c0f8c2b in flushbuf (buf=buf@entry=0x556e5d406e10, ifp=ifp@entry=0x556e5d3ff680) at message.c:1038
#8  0x0000556e5c0f8d95 in start_message (buf=buf@entry=0x556e5d406e10, ifp=ifp@entry=0x556e5d3ff680, type=type@entry=17, len=len@entry=12) at message.c:1103
#9  0x0000556e5c0f8b84 in send_crypto_seqno (buf=buf@entry=0x556e5d406e10, ifp=ifp@entry=0x556e5d3ff680) at message.c:1154
#10 0x0000556e5c0f8c2b in flushbuf (buf=buf@entry=0x556e5d406e10, ifp=ifp@entry=0x556e5d3ff680) at message.c:1038
#11 0x0000556e5c0f8d95 in start_message (buf=buf@entry=0x556e5d406e10, ifp=ifp@entry=0x556e5d3ff680, type=type@entry=17, len=len@entry=12) at message.c:1103
[....]
(gdb) up 100000000
#261944 0x0000556e5c0f0645 in main (argc=<optimized out>, argv=<optimized out>) at babeld.c:826
826                         flushbuf(&neigh->buf, neigh->ifp);
(gdb) down
#261943 0x0000556e5c0f8c2b in flushbuf (buf=0x556e5d406e10, ifp=0x556e5d3ff680) at message.c:1038
1038                send_crypto_seqno(buf, ifp);
(gdb) p *buf
$12 = {
  sin6 = {
    sin6_family = 10, 
    sin6_port = 10266, 
    sin6_flowinfo = 0, 
    sin6_addr = {
      __in6_u = {
        __u6_addr8 = "\376\200\000\000\000\000\000\000Lf\214\377\376u\366\030", 
        __u6_addr16 = {33022, 0, 0, 0, 26188, 65420, 30206, 6390}, 
        __u6_addr32 = {33022, 0, 4287391308, 418805246}
      }
    }, 
    sin6_scope_id = 56
  }, 
  buf = 0x556e5d403740 "\n\026\002@i\330\177", 
  len = 1368, 
  size = 1436, 
  flush_interval = 7500, 
  timeout = {
    tv_sec = 10498367, 
    tv_usec = 127533
  }, 
  have_id = 0 '\000', 
  have_nh = 0 '\000', 
  have_prefix = 0 '\000', 
  id = "\000\000\000\000\000\000\000", 
  nh = "\000\000\000", 
  prefix = '\000' <repeats 15 times>, 
  hello = -1
}
(gdb) p *ifp
$13 = {
  next = 0x0, 
  conf = 0x556e5d3fe910, 
  ifindex = 56, 
  flags = 321, 
  cost = 96, 
  channel = -2, 
  hello_timeout = {
    tv_sec = 10498368, 
    tv_usec = 206016
  }, 
  update_timeout = {
    tv_sec = 10498411, 
    tv_usec = 673029
  }, 
  update_flush_timeout = {
    tv_sec = 0, 
    tv_usec = 0
  }, 
  name = "re6stnet10\000\000\000\000\000", 
  ipv4 = 0x0, 
  numll = 1, 
  ll = 0x556e5d401fa0, 
  buf = {
    sin6 = {
      sin6_family = 10, 
      sin6_port = 10266, 
      sin6_flowinfo = 0, 
      sin6_addr = {
        __in6_u = {
          __u6_addr8 = "\377\002", '\000' <repeats 11 times>, "\001\000\006", 
          __u6_addr16 = {767, 0, 0, 0, 0, 0, 256, 1536}, 
          __u6_addr32 = {767, 0, 0, 100663552}
        }
      }, 
      sin6_scope_id = 56
    }, 
    buf = 0x556e5d4019f0 "\006\n", 
    len = 0, 
    size = 1436, 
    flush_interval = 7500, 
    timeout = {
      tv_sec = 0, 
      tv_usec = 0
    }, 
    have_id = 0 '\000', 
    have_nh = 0 '\000', 
    have_prefix = 0 '\000', 
    id = "\362\336\361\377\376\300\001\070", 
    nh = "\000\000\000", 
    prefix = "$\001Q\200\000\000\000\275\000\000\000\000\000\000\000", 
    hello = -1
  }, 
  buffered_updates = 0x0, 
  num_buffered_updates = 0, 
  update_bufsize = 0, 
  last_update_time = 10498341, 
  hello_seqno = 47223, 
  hello_interval = 15000, 
  update_interval = 60000, 
  rtt_decay = 125, 
  rtt_min = 10000, 
  rtt_max = 500000, 
  max_rtt_penalty = 5000, 
  key = 0x556e5d3fe1c0, 
  pc = 33480, 
  index = "-\214K\221\001\037\202'"
}
(gdb) down
#261939 0x0000556e5c0f8b84 in send_crypto_seqno (buf=buf@entry=0x556e5d406e10, ifp=ifp@entry=0x556e5d3ff680) at message.c:1154
1154        start_message(buf, ifp, MESSAGE_CRYPTO_SEQNO, 4 + INDEX_LEN);
(gdb) p (buf->size - buf->len)
$16 = 68

The exact source code used include patches from Nexedi and can be found here : https://lab.nexedi.com/nexedi/babeld/tree/hmac-nxd2

[Tomo59]

ping @MisterDA as I think the problem is not fixed in #52

[MisterDA]

Yep. I’ll look into that :)

[jech]

Patch looks good to me. Please add an explicit check in the function that there is enough buffer space, and I'll cherry pick it.

[MisterDA]

Patch modified and simplified here f32bf63.

[Tomo59]

Hello what is the status regarding this issue ? We are waiting for the fix to deploy HMAC in our network.

[MisterDA]

Hello Thomas,

The patch is now eb11023 (because of rebases).

Regarding the MAC status, you can find updated code in my MAC authentication for the Babel routing protocol PR. It (currently) rebases the hmac branch against 1.9.2, cleans up the code, and add various fixes. The configuration language is also changed (s/hmac/mac/).
It lacks support for key rotation, multiple keys, and default keys configuration, but I have prototypes for that, which still need a bit of work before they’re ready.

[jech]

This is now commit af02039, and has been merged into master.