Kernel settings for a lot of open connections

Measure first

ss -s
cat /proc/sys/fs/file-nr
nstat -az TcpExtListenOverflows TcpExtListenDrops

ListenOverflows climbing means the accept queue is full and the kernel is discarding completed handshakes. That is a specific problem with a specific fix. Most other symptoms are not solved by sysctl at all, and tuning a machine that is not under pressure only moves the eventual failure somewhere harder to find.

The settings that earn their place

# /etc/sysctl.d/90-connections.conf
fs.file-max = 2097152
net.core.somaxconn = 65535
net.core.netdev_max_backlog = 32768
net.ipv4.tcp_max_syn_backlog = 65535
net.ipv4.ip_local_port_range = 10240 65535
net.ipv4.tcp_fin_timeout = 15
net.ipv4.tcp_tw_reuse = 1
net.ipv4.tcp_slow_start_after_idle = 0
net.ipv4.tcp_mtu_probing = 1
net.core.default_qdisc = fq
net.ipv4.tcp_congestion_control = bbr

sysctl --system
sysctl net.ipv4.tcp_congestion_control

What each one is doing:

• somaxconn caps the listen backlog, and your application still has to ask for it. In nginx that is listen ... backlog=65535; in most languages it is the second argument to listen().
• ip_local_port_range matters on the client side. A proxy opening outbound connections to one destination exhausts ephemeral ports at around twenty-eight thousand with the default range.
• tcp_tw_reuse lets the kernel reuse TIMEWAIT sockets for new outbound connections. Safe. Its cousin `tcptw_recycle` was removed from Linux in 4.12, and any guide still recommending it was written before 2017.
• fq with bbr is a real improvement on long or lossy paths and a wash on short clean ones. Both are in the stock kernels.

Connection tracking

A stateful firewall gives every connection a conntrack entry, and the table has a ceiling. Reaching it drops packets and writes nf_conntrack: table full to the kernel log.

sysctl net.netfilter.nf_conntrack_max net.netfilter.nf_conntrack_count

Raise it if you have the memory — each entry costs a few hundred bytes:

net.netfilter.nf_conntrack_max = 1048576

Where the workload is a stateless public service, the better answer is not to track it:

table inet raw {
  chain prerouting {
    type filter hook prerouting priority raw;
    tcp dport 443 notrack
  }
}

The limits systemd ignores

/etc/security/limits.conf applies to logins through PAM. It has no effect whatsoever on a service started by systemd, which is why raising nofile there and restarting nginx changes precisely nothing. Set it on the unit:

mkdir -p /etc/systemd/system/nginx.service.d
cat > /etc/systemd/system/nginx.service.d/limits.conf <<EOF
[Service]
LimitNOFILE=1048576
EOF
systemctl daemon-reload
systemctl restart nginx

Verify against the running service rather than the file you just wrote:

systemctl show nginx -p LimitNOFILE

Three things not to do

• Do not paste a fifty-line sysctl file from a forum. Half of it targets a kernel that has not shipped in a decade, and one line will quietly break path MTU discovery.
• Do not raise tcp_rmem and tcp_wmem without a long fat path to justify it. On a one-millisecond path, larger buffers buy latency and memory pressure.
• Do not disable tcp_timestamps. You lose round-trip estimation and PAWS, and gain a rumour about performance.

Measure again after each change, one change at a time, and keep the file in version control so the next person can see what you did and why.