Bufferbloat

Bufferbloat is a phenomenon in a packet-switched computer network whereby excess buffering of packets inside the network causes high latency and jitter, as well as reducing the overall network throughput. The phenomenon was detailed at least as far back as 1985,^[1] but gained more widespread attention starting in 2009.^[2][3][4]

Bufferbloat occurs when a network link becomes congested, causing packets to become queued in the buffer. In a first-in first-out queuing system, larger buffers result in longer queues and higher latency, but do not improve network throughput and may reduce goodput to zero.

Buffering

This problem is caused mainly by router and switch manufacturers making incorrect assumptions and buffering packets for too long in cases where they should be dropped. This can lead to TCP's congestion-avoidance algorithms breaking, causing problems such as high and variable latency, and choking network bottlenecks for all other flows as the buffer becomes full of the packets of one TCP stream and other packets are then dropped.^[5] The buffers then take some time to drain, before the TCP connection ramps back up to speed and then floods the buffers again.^{[citation needed]}

Under falling prices of memory and the perception that dropping packets should be avoided if possible, network buffers in operating systems and network devices have grown.^{[when?][citation needed]}

A bloated buffer has an effect only when this buffer actually fills up. In other words, over-sized buffers have a damaging effect only when the link they buffer for becomes a bottleneck. When the current bottleneck on the route from/to another host is not contended then it is easy to tell if it's bloated or not using just the ping utility provided by most operating systems. First, the other host should be pinged continuously. Then a several seconds long download from it should be started and stopped a few times. By design, the TCP congestion avoidance algorithm rapidly fills up the bottleneck on the route. If downloading (resp. uploading) correlates with a direct and important increase of the round trip time reported by ping, then it proves that the buffer of the current bottleneck in the download (resp. upload) direction is bloated. Since the increase of the round trip time is caused by the buffer on the bottleneck, the maximum increase gives a rough estimation of its size in milliseconds.^{[citation needed]}

Using instead of ping in the above an advanced traceroute tool (like for instance MTR) will not just demonstrate the existence of a bloated buffer on the bottleneck but will also pinpoint its the network location. traceroute achieves this by pinging every router, which shows the latency added by every link on the route.^{[citation needed]}

Mechanism

The TCP congestion avoidance algorithm relies on packet drops to determine the bandwidth available. It speeds up the data transfer until packets start to drop, then slows down the connection. Ideally it speeds up and slows down until it finds an equilibrium equal to the speed of the link. However, for this to work the packet drops must occur in a timely manner, so that the algorithm can select a suitable transfer speed. With a large buffer that has been filled, the packets will arrive at their destination, but with a higher latency. The packet is not dropped, so TCP does not slow down once the uplink has been saturated, further filling the buffer. Newly arriving packets are dropped only when the buffer is fully saturated. TCP may even decide that the path of the connection has changed, and again go into more aggressive search for a new operating point^{[clarification needed][citation needed]}.

In a network buffer, packets are queued before being transmitted. In the problematic situation packets are only dropped if the buffer is full. On older routers, buffers were fairly small so they filled quickly and therefore packets began to drop shortly after the link became saturated, so the TCP protocol could adjust, and the issue wouldn't become apparent. On newer routers buffers have become large enough to hold several megabytes of data, which translates to 10 seconds or more at a 1 Mbit/s line rate used for residential Internet access. This causes the TCP algorithm that shares bandwidth on a link to react very slowly as its behavior is quadratic in the amount of buffering^{[clarification needed]}.

The problem also affects other protocols. All packets passing through a simple buffer implemented as a single queue will experience the same delay, so the latency of any connection that passes through a filled buffer will be affected. This also reduces the interactivity of applications using other network protocols, including UDP or any other datagram protocol used in latency sensitive applications like VoIP and games^{[citation needed]}. In extreme cases bufferbloat may cause failures in essential protocols such as DNS.

Applications

• Low latency: Any type of service which requires consistent low latency and jitter (whether low or high bandwidth), be it VoIP, networked gaming, video chat programs, and interactive application such as text, remote login become almost impossible.
• Latency has been identified as more important than bandwidth for many years.^[6]
• Other: When bufferbloat is present and the network is under load, latency and/or throughput sensitive uses are affected, for example a symptom is that normal web page loads can take many seconds to complete.

Mitigations

The problem may be mitigated by reducing the buffer size on the OS^[7] and network hardware; however, this is not configurable on most home routers, broadband equipment and switches, nor even feasible in today's broadband and wireless systems.^[7]

• DOCSIS was modified^[8] to allow smaller buffers in cable modems^[7]
• Using HTTP pipelining or SPDY instead of HTTP ^[7]

Traffic shaping

DiffServ can be used to prioritise traffic, which uses multiple buffers (queues) for each traffic class. This does not fundamentally change the situation, as although HTTP and VoIP may be buffered independently, each buffer will still be independently susceptible to bufferbloat. In practice though this may help mitigate,^[7] for example as a result of one large buffer being split into multiple smaller buffers, or isolation of bufferbloat queues combined with prioritisation.

Active queue management

Full solutions require active queue management (AQM). Additionally users have no control when bufferbloat occurs within the networks of their ISPs and other third-party networks throughout the Internet.

• CeroWrt is an open source project based on OpenWrt with AQM.^[7]
• CoDel, is an active queue management algorithm

Tools and projects

The ICSI Netalyzr^[9] is an on-line tool to check the user's own network for bufferbloat.^[10]

See also

• Bandwidth-delay product
• Congestion window
• Explicit Congestion Notification
• Head-of-line blocking
• Network congestion
• Packet loss
• TCP congestion avoidance algorithm

References

External links

• Bufferbloat.net, a central repository for bufferbloat researchers
• Dashboard over Bufferbloat, a monitoring tool for self-interested people
• BufferBloat: What's Wrong with the Internet? A discussion with Vint Cerf, Van Jacobson, Nick Weaver, and Jim Gettys
• Google Tech Talk on YouTube April, 2011, by Jim Gettys, introduction by Vint Cerf
• Bufferbloat: Dark Buffers in the Internet — Demonstrations Only on YouTube April, 2011, by Jim Gettys, introduction by Vint Cerf
• Bufferbloat: Dark Buffers in the Internet — Demonstrations and Discussions on YouTube 21 minute demonstration and explanation of typical broadband bufferbloat
• LACNIC - BufferBloat on YouTube May 2012, by Fred Baker (IETF chair) in Spanish, English slides available [1]
• tech-net - network queue lengths, a March 2009 discussion concerned with buffer bloat