Increase NOFILE limit for tox-bootstrapd

[nurupo]

tox-bootstrapd can use around 600 TCP sockets during TCP server's normal
functioning. Many systems default to having a soft limit of 1024 open file
descriptors, which we are close to reaching, so it was suggested we bump that
limit to a higher number. iphy suggested increasing it to 32768.

Related to the discussion at #1227.

---

This change is 

[CLAassistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

[codecov]

Codecov Report

Merging #1234 into master will increase coverage by <.1%.
The diff coverage is n/a.

@@           Coverage Diff            @@
##           master   #1234     +/-   ##
========================================
+ Coverage      83%   83.1%   +<.1%     
========================================
  Files          82      82             
  Lines       14685   14684      -1     
========================================
+ Hits        12192   12203     +11     
+ Misses       2493    2481     -12

Impacted Files	Coverage Δ
toxcore/LAN_discovery.c	83% <0%> (-2.9%)	⬇️
toxcore/friend_connection.c	93.4% <0%> (-0.9%)	⬇️
toxcore/Messenger.c	86.8% <0%> (-0.1%)	⬇️
toxcore/DHT.c	76.8% <0%> (ø)	⬆️
toxav/toxav.c	67.7% <0%> (+0.6%)	⬆️
auto_tests/toxav_many_test.c	93.2% <0%> (+0.6%)	⬆️
toxav/msi.c	65.6% <0%> (+1.1%)	⬆️
toxav/audio.c	70.3% <0%> (+4.6%)	⬆️

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update b56166f...d89f83f. Read the comment docs.

[iphydf]

Reviewed 7 of 7 files at r1.
Reviewable status: 0 of 1 approvals obtained (waiting on @nurupo)

---

.travis/cmake-freebsd-stage1, line 0 at r1 (raw file):
Any idea why reviewable thinks this is a binary file? Are there any weird control characters in it accidentally?

---

other/bootstrap_daemon/tox-bootstrapd.sh, line 41 at r1 (raw file):

		cat << EOF

Warning: Your system has very few filedescriptors available in total.

"file descriptors"

---

other/bootstrap_daemon/tox-bootstrapd.sh, line 44 at r1 (raw file):

Maybe you should try raising that by adding 'fs.file-max=100000' to your
/etc/sysctl.conf file.  Feel free to pick any number that you deem appropriate.

Typically, users may not have an idea what value is appropriate. We should recommend 32768, which is an arbitrary but somewhat widely spread number.

---

other/bootstrap_daemon/src/tox-bootstrapd.c, line 452 at r1 (raw file):

                          "Consider raising the limit to at least %ju or the recommended %ju. "
                          "Continuing using the current limit (%ju).\n",
                          (uintmax_t)limit.rlim_cur, (uintmax_t)rlim_min, (uintmax_t)rlim_suggested, (uintmax_t)limit.rlim_cur);

uint32_t or even "unsigned" should be enough, so we can use %u instead of %ju (I'm not sure how portable that is - mingw32 seems to have some issues with it according to robinli).

[nurupo]

Reviewable status: complete! 1 of 1 approvals obtained (waiting on @nurupo)

---

other/bootstrap_daemon/tox-bootstrapd.sh, line 44 at r1 (raw file):

Previously, iphydf wrote…

Typically, users may not have an idea what value is appropriate. We should recommend 32768, which is an arbitrary but somewhat widely spread number.

That's a bit different. We recommend 32768 limit for the tox-bootstrapd process, however what we are talking in here about, the fs.file-max of /etc/sysctl.conf, is the limit of the entire Linux system, e.g. how many file descriptors can the entire system have open at the same time, all of the processes combined (summed). That's the same value that /proc/sys/fs/file-max reports, which we check a few lines above to make sure we don't exhaust the system of all file descriptors. Well, the system can still run out of fds if there are a lot of other programs/daemons running, but at least we make an attempt to pick a reasonable number.

---

other/bootstrap_daemon/src/tox-bootstrapd.c, line 452 at r1 (raw file):

Previously, iphydf wrote…

uint32_t or even "unsigned" should be enough, so we can use %u instead of %ju (I'm not sure how portable that is - mingw32 seems to have some issues with it according to robinli).

rlim_t is defined as a some unspecified unsigned integer type. On some systems it's unsigned long, on other it's unsigned long long. I initially used %llu but got a format warning saying that I use format for unsigned long long for a value of unsigned long type on either Linux or FreeBSD build on Travis-CI. If you make it %lu, you'd get a format warning on some systems sayingthat you are using unsigned long format for a value of the type unsigned long long. It appears like casting it to the biggest unsigned integer type and printing it out as the biggest u signed integer type -- %ju format string, I think it's defined in C99 -- is the best option. If %ju is not portable enough (MSVC has spotty C99 support), we should come up with something else. I don't see your suggestion of %u working though, it would cause the same format warnings.

[iphydf]

Reviewable status: complete! 1 of 1 approvals obtained (waiting on @nurupo and @iphydf)

---

other/bootstrap_daemon/tox-bootstrapd.sh, line 44 at r1 (raw file):

Previously, nurupo wrote…

That's a bit different. We recommend 32768 limit for the tox-bootstrapd process, however what we are talking in here about, the fs.file-max of /etc/sysctl.conf, is the limit of the entire Linux system, e.g. how many file descriptors can the entire system have open at the same time, all of the processes combined (summed). That's the same value that /proc/sys/fs/file-max reports, which we check a few lines above to make sure we don't exhaust the system of all file descriptors. Well, the system can still run out of fds if there are a lot of other programs/daemons running, but at least we make an attempt to pick a reasonable number.

Ok, sounds reasonable.

---

other/bootstrap_daemon/src/tox-bootstrapd.c, line 452 at r1 (raw file):

Previously, nurupo wrote…

rlim_t is defined as a some unspecified unsigned integer type. On some systems it's unsigned long, on other it's unsigned long long. I initially used %llu but got a format warning saying that I use format for unsigned long long for a value of unsigned long type on either Linux or FreeBSD build on Travis-CI. If you make it %lu, you'd get a format warning on some systems sayingthat you are using unsigned long format for a value of the type unsigned long long. It appears like casting it to the biggest unsigned integer type and printing it out as the biggest u signed integer type -- %ju format string, I think it's defined in C99 -- is the best option. If %ju is not portable enough (MSVC has spotty C99 support), we should come up with something else. I don't see your suggestion of %u working though, it would cause the same format warnings.

I'll be more specific. I thought I was, by putting this comment on the line containing "uintmax_t". More specifically I meant: replace uintmax_t with either uint32_t or unsigned, so that %u can be used.

[nurupo]

Reviewable status: complete! 1 of 1 approvals obtained (waiting on @nurupo)

---

other/bootstrap_daemon/src/tox-bootstrapd.c, line 452 at r1 (raw file):

Previously, iphydf wrote…

I'll be more specific. I thought I was, by putting this comment on the line containing "uintmax_t". More specifically I meant: replace uintmax_t with either uint32_t or unsigned, so that %u can be used.

You realize that you are suggesting potentially downcasting, for example, 64-bit unsigned integer to 32-bit unsigned integer? For example, rlim_t can be defined as a 64-bit unsigned long longon a system, but we would downcast it to unsigned which is 32-bit on that particular system. The issue with downcasting is that you might truncate/overflow the value in that log message. While I wouldn't expect rlim_t to be a such large number that it wouldn't fit in either uint32_t or unsigned, for some reason its exact type/width are not defined in Linux, FreeBSD and OpenBSD manpages, and some systems do use 64-bit unsigned integers for it, in fact one (or more) of our systems on Travis-CI does, so the fd number could technically be set to a 64-bit unsigned integer (or even 128 bit if some weird ABI with 128 bit integers comes out, as the manpages don't specify exact type/width of rlimit_t), so downcasting, while it would generally work, is not safe, it might result in us printing the wrong number.

[nurupo]

Looks like MSVC supports uintmax_t since at least 2013 version (there is no info on any earlier version, perhaps because they are discontinued).

The j in format string is not supported by MSVC though.

The hh, j, z, and t length prefixes are not supported.

[nurupo]

Oh, and for everyone reading this who is not iphy and thus probably doesn't understand the context, we kind of want to make tox-bootstrapd run on Windows in the future, possibly, maybe, so that's why we a bit concerned about MSVC despite it being linux/unix (posix?) daemon.

[nurupo]

Huh, this documentation for MSVC 2017 (and 2015?) C Runtime says that you can use j, so it's supported? But the other documentation for MSVC 2015 I linked in an earlier comment said that it isn't. Perhaps it's supported only starting MSVC 2017?

[iphydf]

Reviewable status: complete! 1 of 1 approvals obtained (waiting on @nurupo)

---

other/bootstrap_daemon/src/tox-bootstrapd.c, line 452 at r1 (raw file):

Previously, nurupo wrote…

You realize that you are suggesting potentially downcasting, for example, 64-bit unsigned integer to 32-bit unsigned integer? For example, rlim_t can be defined as a 64-bit unsigned long longon a system, but we would downcast it to unsigned which is 32-bit on that particular system. The issue with downcasting is that you might truncate/overflow the value in that log message. While I wouldn't expect rlim_t to be a such large number that it wouldn't fit in either uint32_t or unsigned, for some reason its exact type/width are not defined in Linux, FreeBSD and OpenBSD manpages, and some systems do use 64-bit unsigned integers for it, in fact one (or more) of our systems on Travis-CI does, so the fd number could technically be set to a 64-bit unsigned integer (or even 128 bit if some weird ABI with 128 bit integers comes out, as the manpages don't specify exact type/width of rlimit_t), so downcasting, while it would generally work, is not safe, it might result in us printing the wrong number.

Yes. Yet I don't expect systems to have set a specific number greater than 4 billion that is not numeric_limits<rlim_t>::max(). Regardless, rlim is not going to work on Windows, so it doesn't really matter.

[iphydf]

Can you rebase on master?

[nurupo]

Regardless, rlim is not going to work on Windows, so it doesn't really matter.

Huh. Somehow I went along with

I'm not sure how portable that is - mingw32 seems to have some issues with it according to robinli

without realizing that this won't be an issue as we would compile out that portion of the code on Windows.

[nurupo]

Reviewable status: complete! 1 of 1 approvals obtained (waiting on @nurupo)

---

.travis/cmake-freebsd-stage1, line at r1 (raw file):

Previously, iphydf wrote…

Any idea why reviewable thinks this is a binary file? Are there any weird control characters in it accidentally?

There are control characters on lines 60 and 66, they are there intentionally.

---

other/bootstrap_daemon/tox-bootstrapd.sh, line 41 at r1 (raw file):

Previously, iphydf wrote…

"file descriptors"

Fixed.

[nurupo]

Removing the .travis.yml commit. Here is the last build with everything enabled https://travis-ci.org/TokTok/c-toxcore/builds/444894774.

[nurupo]

Oh, and rebased on master.

[iphydf]

Can you rebase again, and then merge?

[nurupo]

Rebased.

I can merge it, I guess.

git checkout master
git merge --ff-only bootstrapd-increase-nofile

right? Seems to produce the right result locally.