Fix regression with "too many open files" on macOS

[mitchchn]

rlimit tweaking was removed in #3512. But without it, the rlimit is only 256, and I can't use hugo serve to preview even a relatively modest site without seeing this error:

Error: listen tcp 127.0.0.1:1313: socket: too many open files

The ulimit command still works fine for me in Sierra (10.12.6 beta), so I don't understand why it was removed from Hugo:

ulimit -n 10000; hugo serve

Any clarification on this, @bep?

[bep]

Not sure why you run Sierra beta, but it started to fail for me at some point, and Googling it pointed me to some posts saying that "now you have to set the ulimit in some plist file" etc. So we're not maintaining a hack/magic that only work for some -- when the real solution is to set a more sensible default value yourself.

I will keep this open as a documentation issue, but I'm not reverting any ulimit hack.

[mitchchn]

1. Have you actually tested the plist approach? It doesn't appear to work for me, but I'll keep testing.

2. Once the correct solution is known, would you consider allowing a script to be included in the repo, outside of the hugo binary, so that users don't have to set up a launch daemon by themselves?

[budparr]

@mitchchn This has helped me: https://superuser.com/a/828010

[bep]

Have you actually tested the plist approach?

Yes and it worked for me. But my main point is that I cannot have a hack in Hugo that fails, i.e. creates ERROR in the log for me; that makes Hugo near hopeless to work with and if it fails on my macOS (I don't run Sierra beta), then others will soon start to complain as well.

[bep]

This article seem to describe it well:

https://blog.dekstroza.io/ulimit-shenanigans-on-osx-el-capitan/

[mitchchn]

OK, it worked after following the steps in that post. I'd still like to have a shell script which sets this up, even if that script is only linked to from the documentation and not included in Hugo. It's a lot to ask writers (not developers!) to set up a launch daemon manually just so they can use the preview server.

[bep]

Just test-reverted the commit in question:

ERROR 2017/06/12 20:00:00 Unable to increase number of open files limit operation not permitted

I could start my hugo server with sudo, but that doesn't have a nice ring to it. I'm on macOS 10.12.5. I'm open to suggestions for working workarounds.

[mitchchn]

I also reverted the commit and got the same error, but then I changed the rlimit to a smaller value (still much larger than the default of 256):

		rLimit.Max = 10000
		rLimit.Cur = 10000

Rebuilt and had no issues. Can you confirm that, @bep?

[bep]

Rebuilt and had no issues. Can you confirm that, @bep?

Yes. I have at least one other issue that I should get out there and will do a patch release in a day or so, hopefully before people start shouting. Thanks for the heads up.

[mitchchn]

Thanks. For what it's worth, the max allowable value seems to be 64000.

[binarymist]

ulimit -n
65536

hugo server

                   | EN   
+------------------+-----+
  Pages            | 513  
  Paginator pages  | 139  
  Non-page files   |   1  
  Static files     | 139  
  Processed images |   0  
  Aliases          | 331  
  Sitemaps         |   1  
  Cleaned          |   0  

On Linux Mint, and hugo server still won't run. Anyone know why hugo server won't run, or have any other tips?
hugo version
v0.37.1 linux/amd64 BuildDate: 2018-03-07T17:56:25Z

lsof reports about half a million open files, but only a handfull are within my static site root directory.

I get the feeling hugo is reporting on open files outside of the static website root directory?

[mohin7]

I found this solution from here,
https://www.docsy.dev/docs/getting-started/#known-issues

[github-actions]

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.