F-stack rack and BBR both causes PCB memory leak

[MrSonicMaster]

My application suddenly start spamming ff_socket: "No buffer space available." so I went to debug why. I figured out that setting tcp stack back to default freebsd stack solved the issue, it only happens on tcp stack to rack/bbr. Also does not happen unless socket is bound and connected (leak is in pcb), so test case must cover that.

After debugging for a couple hours, i think it is stemming from tcphpts.
I spewed logs in certain places to get insight. Hopefully the info can be helpful for you to get to the bottom of it.

Here is logs when using freebsd stack.
First log shows that a 4th pcb has been allocated, for total of 4 currently allocated. Middle lines show the socket creation and instant destruction, basically ff_socket, ff_bind, ff_connect, ff_close combo. Expected refcnt increases/decreases. Very last line shows that the pcb is getting freed, and down to 3 currently handed out. Looks good!

ALLOC 4
tcp_newtopcb do thing 0
IN_PCBREF INCR REF: 1 -> 2
tcp_usr_detach
DISCARD CB!
DRAINCNT IS 0
DO PCB RELE
IN_PCBRELE_WLOCKED DECR REF: 2 -> 1
RELEASE RET ZERO! 1
in_pcbfree CALLED! SCHEDULE DEFERRED.
in_pcbfree_deferred!
IN_PCBRELE_WLOCKED DECR REF: 1 -> 0
PCB RELE WLOCKED 3

Here's logs with rack/bbr stack enabled. Now we see tcp hpts is adding a refcnt, which didn't happen before, but does not seem to be decreasing it. I have twiddled with tcp timewait/keepalive settings and brought them down to very low values, and let it sit there for hours in hopes to see it eventually release, but nope. Socket was not given nearly enough time to actually connect so i don't think that would be the issue anyways.

so, in_pcbfree sets INP_FREED flag, which is picked up during in_pcbfree_deferred's call to in_pcbrele_wlocked, however during that call the refcnt is still 2, so somebody else is holding ref (appears to be tcphpts) meaning it can't be freed yet.

ALLOC 4
tcp_newtopcb do thing 0
IN_PCBREF INCR REF: 1 -> 2
hpts do thing 0
IN_PCBREF INCR REF: 2 -> 3
tcp_usr_detach
DISCARD CB!
DRAINCNT IS 0
DO PCB RELE
IN_PCBRELE_WLOCKED DECR REF: 3 -> 2
RELEASE RET ZERO! 2
in_pcbfree CALLED! SCHEDULE DEFERRED.
in_pcbfree_deferred!
IN_PCBRELE_WLOCKED DECR REF: 2 -> 1
RELEASE RET ZERO! 1
FLAG IS FREED!

Test case can be similar to #679 except also binds/connects to some host, then instant ff_close.

Will run forever with freebsd stack without issue, will exit with these logs on bbr or rack stack:

ff_socket: No buffer space available
ff_socket failed, fd:-1, errno:105, No buffer space available
Done 262154

[jfb8856606]

F-Stack hasn't run test case as client with rack and BBR, and I will debug it.

[jfb8856606]

Now bbr and rack can call hpts_timeout_dir() with timer, and free to uma zone pcbinfo->ipi_zone.

But the timer of HPTS has some other issue, and will adjust later, see also #701.

And now uma zone not freed to OS, will adjust later also.