System/Kernel Parameter For Better Security/Performance

[MiteshShah]

For 8 core CPU and 32GB RAM

fs.inotify.max_user_watches = 10240000
fs.file-max = 2097152
kernel.pid_max = 2097152
kernel.shmmax = 4294967296

net.core.rmem_max = 16777216
net.core.wmem_max = 16777216
net.core.rmem_default = 1048576
net.core.wmem_default = 1048576
net.core.netdev_max_backlog = 65536
net.core.somaxconn = 65536
net.core.optmem_max = 25165824

net.ipv4.tcp_rmem = 4096 1048576 16777216
net.ipv4.tcp_wmem = 4096 1048576 16777216
net.ipv4.tcp_max_syn_backlog = 65536

vm.max_map_count = 262144

[MiteshShah]

# Avoid a smurf attack
net.ipv4.icmp_echo_ignore_broadcasts = 1

# Turn on protection for bad icmp error messages
net.ipv4.icmp_ignore_bogus_error_responses = 1

# Turn on syncookies for SYN flood attack protection
net.ipv4.tcp_syncookies = 1

# Turn on and log spoofed, source routed, and redirect packets
net.ipv4.conf.all.log_martians = 1
net.ipv4.conf.default.log_martians = 1

# No source routed packets here
net.ipv4.conf.all.accept_source_route = 0
net.ipv4.conf.default.accept_source_route = 0

# Turn on reverse path filtering
net.ipv4.conf.all.rp_filter = 1
net.ipv4.conf.default.rp_filter = 1

# Make sure no one can alter the routing tables
net.ipv4.conf.all.accept_redirects = 0
net.ipv4.conf.default.accept_redirects = 0
net.ipv4.conf.all.secure_redirects = 0
net.ipv4.conf.default.secure_redirects = 0

# Don't act as a router
net.ipv4.ip_forward = 0
net.ipv4.conf.all.send_redirects = 0
net.ipv4.conf.default.send_redirects = 0


# Turn on execshild
kernel.exec-shield = 1
kernel.randomize_va_space = 1

# Tuen IPv6
net.ipv6.conf.default.router_solicitations = 0
net.ipv6.conf.default.accept_ra_rtr_pref = 0
net.ipv6.conf.default.accept_ra_pinfo = 0
net.ipv6.conf.default.accept_ra_defrtr = 0
net.ipv6.conf.default.autoconf = 0
net.ipv6.conf.default.dad_transmits = 0
net.ipv6.conf.default.max_addresses = 1

# Optimization for port usefor LBs
# Increase system file descriptor limit
fs.file-max = 65535

# Allow for more PIDs (to reduce rollover problems); may break some programs 32768
kernel.pid_max = 65536

# Increase system IP port limits
net.ipv4.ip_local_port_range = 2000 65000

# Increase TCP max buffer size setable using setsockopt()
net.ipv4.tcp_rmem = 4096 87380 8388608
net.ipv4.tcp_wmem = 4096 87380 8388608

# Increase Linux auto tuning TCP buffer limits
# min, default, and max number of bytes to use
# set max to at least 4MB, or higher if you use very high BDP paths
# Tcp Windows etc
net.core.rmem_max = 8388608
net.core.wmem_max = 8388608
net.core.netdev_max_backlog = 5000
net.ipv4.tcp_window_scaling = 1

Found some other useful parameter at http://www.cyberciti.biz/tips/linux-unix-bsd-nginx-webserver-security.html

[rahul286]

We have some more at http://rtcamp.com/tutorials/linux/sysctl-conf/ (used on rtCamp.com)

We need to find a logical way to tweak these values. As of now we are doing it randomly :|

[rahul286]

I observed fs.file-max controls max listen queue in php-fpm and also few things in mysql.

Also related - https://rtcamp.com/tutorials/linux/increase-open-files-limit/

[rahul286]

@MiteshShah lets reduce swap usage

vm.swappiness=10

[rahul286]

About - https://rtcamp.com/tutorials/linux/increase-open-files-limit/ pam limit may not be needed for nginx/php/mysql

[rahul286]

Some more related links:

• http://www.cyberciti.biz/faq/linux-unix-nginx-too-many-open-files/
• http://stackoverflow.com/questions/16919673/nginx-file-descriptor-limit
• http://serverfault.com/questions/516802/too-many-open-files-with-nginx-cant-seem-to-raise-limit

[ghost]

I see a lot of ideas coming up for all sorts of "tuning".
This could easily get out of control...
I have two general ideas:

1. If those options are made available, there could/should be different "profiles" to load (e.g. 512 MB 1 vCPU / 1GB vCPU / 1GB 2 vCPU, etc.) otherwise people will run their boxes into the ground (or leave resources idle). But because of that, I suggest the following:
2. For all such features, only the most proven ones should be enabled. There should be a "--notuning" or somesuch switch for those who don't want the risk or have already tuned manually.
   I belong to the first group (don't want the risk) because I am skeptical that these values can be correct because it'd take an enormous amount of QA to do it right (unless it's done only for 1 hosting company's offer which makes it a bit more manageable).

[rahul286]

@Rippler your concerns are valid. I am also thinking we should avoid over-optimizing at all costs.

May be we should touch values like:

• fs.file-max (i think this controls number of connection/file descriptor and can affect scalability)
• fs.inotify.max_user_watches (i think this control how many file we can watch. backup miletsone may need this)
• net.core.somaxconn (I think this controls spikes i.e. if too many connection are requested, queue them up rather than dropping right away)

Yesterday only I came across this answer - http://askubuntu.com/a/184221/47565 which made me believe it's not quite right to set vm.swappiness=10

Any further suggestions are welcome. :-)

[ghost]

@rahul286 I support your thinking here.

Another thing that should perhaps should be considered here is that people don't end up with lower maximum values than before (e.g. if some default is 4096, someone increased it to 100000 and it ends up getting tuned to 65535).
It'd be nice to be able to save the "before" values to a text file for backup and troubleshooting purpose.

[michacassola]

Can these things be handled differently with docker? Are they still relevant?