NetworkingMarch 4, 2025

Stop guessing: a practical look at TCP buffer tuning

If you have ever searched for "slow TCP throughput Linux," you have seen the block. A wall of sysctl lines pasted from a forum thread that was old when you found it. People copy it onto production servers without reading a single value. Sometimes it helps. Often it does nothing. Occasionally it makes things worse. Let's actually understand what is going on.

The core idea: bandwidth-delay product

A single TCP connection can only have so much data "in flight" — sent but not yet acknowledged. That limit is the size of the send and receive buffers. To keep a fast, long link fully utilized, you need enough buffer to cover one full round trip of data.

The magic number is the bandwidth-delay product (BDP):

BDP (bytes) = bandwidth (bytes/sec) x round-trip-time (sec)

Example: a 1 Gbit/s link with 80 ms RTT
  = (1,000,000,000 / 8) bytes/sec x 0.080 sec
  = 125,000,000 x 0.080
  = 10,000,000 bytes  (about 10 MB)

If your receive buffer caps out at 6 MB on that link, you physically cannot fill the pipe. The sender stalls waiting for ACKs, and you blame the network when the bottleneck is a kernel default. This is why the same settings that transform a transcontinental transfer do nothing on your LAN: across a 0.3 ms link, the BDP is tiny and the default buffers already cover it.

The knobs that matter

On modern Linux, autotuning handles most cases. The values below are the ceilings autotuning is allowed to grow into — they are maximums, not fixed allocations:

# Maximum read/write socket buffer (bytes)
net.core.rmem_max = 16777216
net.core.wmem_max = 16777216

# TCP autotuning: min, default, max (bytes)
net.ipv4.tcp_rmem = 4096 131072 16777216
net.ipv4.tcp_wmem = 4096 16384 16777216

The three-value form is the one people misread. It is not "low, normal, high traffic." It is the minimum the kernel guarantees, the starting default, and the maximum autotuning may reach under memory pressure. For our 10 MB BDP example, a 16 MB ceiling gives comfortable headroom.

Measure before you touch anything

Tuning blind is how you end up with the cursed forum block. Find your real RTT and your real throughput first:

# Round-trip time to the far end
ping -c 20 remote-host

# Actual achievable throughput (run iperf3 -s on the other side)
iperf3 -c remote-host -t 30

# What is the socket actually negotiating right now?
ss -tim dst remote-host

That last command is underused. The -i flag prints per-socket internals: current congestion window, RTT estimate, retransmits, and the negotiated buffer sizes. If cwnd is small and stable while retrans climbs, your problem is loss and congestion control, not buffer size. Tuning buffers there is rearranging deck chairs.

Congestion control is the other half

Buffers decide how much can be in flight. The congestion control algorithm decides how much actually is. On a lossy or long link, switching algorithms often matters more than buffer size:

# See what is loaded and what is active
sysctl net.ipv4.tcp_available_congestion_control
sysctl net.ipv4.tcp_congestion_control

# BBR handles loss-prone links far better than CUBIC
sysctl -w net.ipv4.tcp_congestion_control=bbr

CUBIC, the long-standing default, treats packet loss as a signal to back off hard. On a link with low-level loss that is not from congestion — a flaky wireless hop, a slightly overloaded transit provider — CUBIC collapses throughput needlessly. BBR models bandwidth and latency directly and tends to keep the pipe fuller. It is not a universal win, but on real-world internet paths it usually helps.

Make it stick, but make it deliberate

Once you have tested values that actually move the needle, persist them properly rather than editing the live kernel and forgetting:

# /etc/sysctl.d/90-tcp-tuning.conf
net.core.rmem_max = 16777216
net.core.wmem_max = 16777216
net.ipv4.tcp_rmem = 4096 131072 16777216
net.ipv4.tcp_wmem = 4096 16384 16777216
net.ipv4.tcp_congestion_control = bbr

# Apply without a reboot
sysctl --system

The short version

• Compute your BDP. If your buffer ceiling is below it, you cannot fill the link.
• Bigger buffers do nothing on low-latency links — the default already covers the BDP.
• Measure RTT, throughput, and ss -ti before changing a single value.
• If you see retransmits, the answer is congestion control, not buffers.
• Never paste a config you have not read. Including this one.

Filed under Networking. Found a mistake? I would genuinely like to know — see the about page for how to reach me.