Skip to content
All Posts

Notes on Socket Programming

Systems-level socket notes on EAGAIN, MTU and MSS, send and receive buffers, and the effects of Nagle's algorithm.

325 wordsView all posts with this tag networking

EAGAIN

Socket reads and writes on Linux and Unix can return EAGAIN. With a nonblocking socket, for example, an event may tell the server to read even though no data is available:

EAGAIN: Resource temporarily unavailable

The asynchronous operation returned immediately without reading data, so it asks the caller to retry later. File operations and a failed fork can produce the same error. Windows calls it EWOULDBLOCK. Nonblocking applications should handle these cases through errno.

Transfer-size limits

The MTU is the link layer’s maximum transmission unit, commonly 1,500 bytes. An IP header uses 16 bits for packet length, giving a maximum of 65,535 bytes.

An IP packet larger than the MTU must be fragmented and reassembled by the receiver. Networks can have different MTUs, causing further fragmentation. If fragmentation is disabled because it hurts performance and a network’s MTU is smaller, the packet is dropped and an ICMP response asks the sender to fragment. The smallest value along the route is the path MTU.

The MSS is the maximum TCP payload sent to the peer:

MSS = MTU - IP header - TCP header, usually 1460 = 1500 - 20 - 20.

Both endpoints advertise an MSS during the TCP handshake and use the smaller value.

Socket buffers

The default size is commonly 1024 * 8 = 8192 bytes. Data is first copied into a system send buffer; the system then transmits it. If the buffer frequently fills, increase it with setsockopt.

Buffers reduce network I/O. Data copied into the send buffer is scheduled by the system rather than necessarily sent at once, and it remains there until the peer acknowledges it.

Nagle’s algorithm

By default, Nagle’s algorithm permits at most one unacknowledged small segment at a time. Sending is allowed when:

  1. The packet reaches the MSS.
  2. It contains FIN.
  3. TCP_NODELAY is enabled.
  4. TCP_CORK is disabled and all previously sent small packets are acknowledged.
  5. A timeout, commonly 200 ms, expires.

This avoids flooding the network with small packets and improves throughput at the cost of latency. Set TCP_NODELAY to disable Nagle’s algorithm.

Originally published on SegmentFault.