vsftpd syscall error on Monterey

[valeriansaliou]

brew config

HOMEBREW_VERSION: 3.3.2
ORIGIN: https://github.com/Homebrew/brew
HEAD: 9daf5e2d28396c7a4379605e89e029d9fccb41e6
Last commit: 2 days ago
Core tap ORIGIN: https://github.com/Homebrew/homebrew-core
Core tap HEAD: 6975114acaf6e9853cb901c5b83c69fc0e284f17
Core tap last commit: 27 minutes ago
Core tap branch: master
HOMEBREW_PREFIX: /usr/local
HOMEBREW_CASK_OPTS: []
HOMEBREW_CORE_GIT_REMOTE: https://github.com/Homebrew/homebrew-core
HOMEBREW_MAKE_JOBS: 16
Homebrew Ruby: 2.6.8 => /System/Library/Frameworks/Ruby.framework/Versions/2.6/usr/bin/ruby
CPU: 16-core 64-bit kabylake
Clang: 13.0.0 build 1300
Git: 2.33.1 => /usr/local/bin/git
Curl: 7.77.0 => /usr/bin/curl
macOS: 12.0.1-x86_64
CLT: N/A
Xcode: 13.1

brew doctor

Please note that these warnings are just used to help the Homebrew maintainers
with debugging if you file an issue. If everything you use Homebrew for is
working fine: please don't worry or file an issue; just ignore this. Thanks!

Warning: Some installed formulae are deprecated or disabled.
You should find replacements for the following formulae:
  go@1.13
  node@10

Warning: "config" scripts exist outside your system or Homebrew directories.
`./configure` scripts often look for *-config scripts to determine if
software packages are installed, and which additional flags to use when
compiling and linking.

Having additional scripts in your path can confuse software installed via
Homebrew if the config script overrides a system or Homebrew-provided
script of the same name. We found the following "config" scripts:
  /Applications/Postgres.app/Contents/Versions/latest/bin/gdal-config

Warning: You have unlinked kegs in your Cellar.
Leaving kegs unlinked can lead to build-trouble and cause formulae that depend on
those kegs to fail to run properly once built. Run `brew link` on these:
  wireshark

• I ran brew update and am still able to reproduce my issue.
• I have resolved all warnings from brew doctor and that did not fix my problem.

What were you trying to do (and why)?

Installed latest vsftpd formula on Monterey, vsftpd starts just fine as before, but is unable to fork when creating a new FTP connection.

What happened (include all command output)?

vsftpd fails on a syscall error defined in: https://github.com/djarosz/vsftpd/blob/master/sysutil.c#L2798

The error returned by vsftpd upon opening the connection is the following:

500 OOPS: setrlimit

Note that setrlimit is a syscal, which seems to have been removed (?) in Monterey. Googling setrlimit Monterey pops similar issues with other software, eg. dovecot.

What did you expect to happen?

Expected that the FTP connection would be opened w/o any issue.

Step-by-step reproduction instructions (by running brew commands)

- Run vsftpd 3.0.5 with a working configuration (any configuration, really)
- vsftpd starts as normal
- Try to open a FTP connection to vsftpd
- vsftpd fails and reject the connection with: `OOPS: setrlimit`

[carlocab]

This sounds like an upstream issue rather than a packaging issue, no? If so, please report this upstream. Thanks!

[valeriansaliou]

You're right, doing so right now. Thought some patch could be brought in the formula, or build flag. Thanks!

[valeriansaliou]

Ref: djarosz/vsftpd#3

[valeriansaliou]

Okay, given the fact that there's already a macOS-related patch within the current vsftpd formula at: https://github.com/Homebrew/homebrew-core/blob/HEAD/Formula/vsftpd.rb#L28

I've made a patch, that works for me locally. The idea is to avoid tuning any rlimit/ulimit if the platform is Apple. The way I patched this is very similar to the other existing formula patch.

Tested & working, FTP connections now work again, and file uploads are also OK on my end.

Patch diff:

diff --git a/sysutil.c b/sysutil.c
index ff8885b..c465b99 100644
--- a/sysutil.c
+++ b/sysutil.c
@@ -2792,6 +2792,7 @@ vsf_sysutil_getuid(void)
 void
 vsf_sysutil_set_address_space_limit(unsigned long bytes)
 {
+#if! defined(__APPLE__)
   /* Unfortunately, OpenBSD is missing RLIMIT_AS. */
 #ifdef RLIMIT_AS
   int ret;
@@ -2807,12 +2808,14 @@ vsf_sysutil_set_address_space_limit(unsigned long bytes)
     die("setrlimit");
   }
 #endif /* RLIMIT_AS */
+#endif
   (void) bytes;
 }
 
 void
 vsf_sysutil_set_no_fds()
 {
+#if! defined(__APPLE__)
   int ret;
   struct rlimit rlim;
   rlim.rlim_cur = 0;
@@ -2822,11 +2825,13 @@ vsf_sysutil_set_no_fds()
   {
     die("setrlimit NOFILE");
   }
+#endif
 }
 
 void
 vsf_sysutil_set_no_procs()
 {
+#if! defined(__APPLE__)
 #ifdef RLIMIT_NPROC
   int ret;
   struct rlimit rlim;
@@ -2838,6 +2843,7 @@ vsf_sysutil_set_no_procs()
     die("setrlimit NPROC");
   }
 #endif
+#endif
 }
 
 void

[valeriansaliou]

Made a patch on the homebrew formula itself. Tested the formula locally and working.

Opened 2 PRs:

• vsftpd 3.0.5 patch for Monterey (fixes non-existing setrlimit syscall) #88573
• vsftpd 3.0.5 patch for Monterey (fixes non-existing setrlimit syscall) formula-patches#403

[SMillerDev]

Please also submit any patches upstream.

[valeriansaliou]

Yes, doing this right now.

[carlocab]

Fixed in #88573.