[Issue] mixnode is leaking approx 5k file handles per day #1985
Description
Describe the issue
The nym-mixnode process has an increasing consumption of file handles. The increase is very linear at a rate of roughly 5100 handles per day (see screenshot). It does not matter whether the node is actively mixing or not. The rate always stays the same. From the start of the process until it eventually hits an "open file" limit.
Hitting that limit renders the process non-functional. It will not crash and still keeps its TCP port listening. Of course it cannot really accept connections anymore. Instead, the process will produce huge amounts of logs with this line:
Sep 1 06:08:22 he3 nym-mixnode[2395218]: 2022-09-01T04:08:22.168Z WARN mixnode_common::verloc::listener > Failed to accept incoming connection - Os { code: 24, kind: Uncategorized, message: "Too many open files" }
Sep 1 06:08:22 he3 nym-mixnode[2395218]: 2022-09-01T04:08:22.168Z WARN mixnode_common::verloc::listener > Failed to accept incoming connection - Os { code: 24, kind: Uncategorized, message: "Too many open files" }
Sep 1 06:08:22 he3 nym-mixnode[2395218]: 2022-09-01T04:08:22.168Z WARN mixnode_common::verloc::listener > Failed to accept incoming connection - Os { code: 24, kind: Uncategorized, message: "Too many open files" }
From an operational perspective this is sub-par. A half-life process is worse than a crashed one because it prevents automatic recovery (e.g. through automatic systemd restart).
I did not verify whether this also happens on an unbonded mixnode. However I'd argue this is also quite pointless.
Expected behaviour
- The amount of open file handles should be constant within boundaries.
- If an open-file-limit is reached, the process should crash.
Steps to Reproduce
- Run a bonded mixnode
- Observe amount of open file handles
Screenshots
Which area of Nym were you using?
- Application: nym-mixnode
- OS: Debian bullseye
- Version: 1.0.2