CCIE 400-101: Network Principles - Latency, Windowing, BDP, Global Synchronization

on

Latency

  • Latency is most often used to describe the delay between the time that data is requested and the time when it arrives (also known as round-trip time or RTT for short).
  • However, there are essentially four components that contribute to total latency:
    1. Transmission delay: the time to transmit data on the link.
    2. Propagation delay: the time for a packet to travel accross the link.
    3. Processing delay:  the time for a device to process the data.
    4. Queuing delay: the time the data spent waiting in queue (memory) inside a device.
  • The combination of transmission, propagation, processing, and queuing delays often produces a complex and variable network latency profile.


  • Different delays can be calculated as follows:
    • Transmission delay (t) = number of bits to send (b) / link data rate (r). (t = b/r)
    • Propagation delay (p) = link distance (d) / speed of signal (s). (p = d/s)
      • Unless otherwise stated, s = c = 3 x 10^8 (the speed of light)
    • Processing delay and queuing delay are variable and depend on many factors.

Example: The latency is indicated in the 'time' column of the ping output. The ping statistics also list the minimum, maximum, and average latencies.

Pinging www.google.com [74.125.224.82] with 1500 bytes of data:
Reply from 74.125.224.82: bytes=1500 time=68ms TTL=52
Reply from 74.125.224.82: bytes=1500 time=68ms TTL=52
Reply from 74.125.224.82: bytes=1500 time=65ms TTL=52
Reply from 74.125.224.82: bytes=1500 time=66ms TTL=52
Reply from 74.125.224.82: bytes=1500 time=70ms TTL=52

Ping statistics for 74.125.224.82:
    Packets: Sent = 5, Received = 5, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
    Minimum = 65ms, Maximum = 70ms, Average = 67ms

  • A networking technology with very high throughput and bad (high) latency can be worse for some applications than one with relatively low throughput but good (low) latency. 
  • See: Ping vs. Latency: What's the Difference?

Windowing

  • Essentially, the TCP window size is the amount of ("in flight") data that a sender can send before it needs to wait for an acknowledgement from the recipient. 
  • If the sender has not received acknowledgement, it will stop and wait. If this wait time exceeds a certain limit, a retransmission will occur. This is how TCP achieves reliable data transmission. 
  • Even if there is no packet loss in the network, windowing can limit throughput. 
  • Because TCP transmits data up to the window size before waiting for the acknowledgement, the full bandwidth of the network may not always get used. 
  • The window size is communicated via a 16-bit field in the TCP header, giving it a maximum size of 64 KB. 
  • Window scale provides a scale factor which is required for TCP to support window sizes that are larger than 64 KB. 
  • Essentially, window scaling simply extends the 16-bit window field to 32 bits. 
  • This is done by specifying a 1-byte shift count in the header options field. 
  • The true receive window size is left shifted by the value in the shift count. 
  • The maximum value of the shift count is 14, resulting in the maximum window size of 1 GB!


  • The window scaling option may be sent only once during a connection by each host, in its SYN packet.
  • The window size can be dynamically adjusted by modifying the value of the window field in the TCP header, but the scale factor remains static for the duration of the TCP connection.
  • Scaling is only in effect if both ends include the option; if only one end of the connection supports window scaling, it will not be enabled in either direction.

 

Bandwidth Delay Product (BDP)

  • In short, bandwidth delay product (BDP) is the optimum window size to fully utilize link bandwidth. 
  • The best possible network performance is achieved when the link between the sender and the receiver forwards a constant stream of data. 
  • Analogy: a water hose. To achieve maximum water flow, the hose should be full. As the hose increases in diameter and length, the volume of water to keep it full also increases. 
  • In networks, diameter equates to bandwidth, length is measured as round-trip time, and the TCP window size is analogous to the volume of water necessary to keep the hose full. 
  • On fast networks with large round-trip times, the TCP window size must be increased to achieve maximum TCP bandwidth. 
  • Theoretically, the TCP window size should be set to the BDP, which is simply computed as bandwidth * delay. 
  • Example: For 1 Gbps link with 70 ms delay, the BDP (optimum TCP window size) is 1,000,000,000 * 0.07 = 70,000,000 (bits). Convert that to bytes, you get 70,000,000 / 8 = 8,750,000 bytes = 8,75 MB.  
  • In order to fully utilize 1 Gbps link with 70 ms, the TCP window size must be 8,75 MB.  
  • Based on these calculations, it is easy to see why the typical default buffer size of 64 KBytes would be completely inadequate for this connection. With 64 KBytes you would get only 0.1% of the available bandwidth.

Global Synchronization

  • During times of congestion, routers can use a technique called tail drop to simply discard packets when the interface queue is filled to its maximum.
  • When tail drop happens, TCP has an automatic recovery from the dropped packets, which causes the sender to reduce its sending rate.
  • Bandwidth utilization drops significantly and interface queue becomes less congested.
  • After a certain amount of time, the sender starts increasing the rate again.
  • The problem is that almost all the senders will use the same time delay. When these delays expire, at the same time, all senders will send additional packets, the interface queue will overflow again, more packets will be dropped, the senders will all back off for a fixed delay.. and the cycle repeats.
  • This pattern of each sender decreasing and increasing transmission rates at the same time as other senders is referred to as "global synchronization".
  • The result is a sawtooth-shaped bandwidth utilization, causing inefficient use of available bandwidth and inconsistent application throughput.


References

Transmission and Propagation Delay
Congestion Avoidance
Tuning TCP for High Bandwidth-Delay Networks
A User's Guide to TCP Windows
TCP Windows and Window Scaling

Comments

Post a Comment