When no need 3 handshakes in TCP?

  sonic0002        2024-09-29 03:19:35       1,426        0    

Overview

In the previous article titled Why TCP needs 3 handshakes simple answers were provided to the following three questions:

  1. Can application data be carried during the first handshake?
  2. Can application data be carried during the second handshake?
  3. Can application data be carried during the third handshake?

Briefly, traditional TCP requires a three-way handshake to establish a connection, and during these three handshakes, only simple SYN and ACK packets are sent.

From the perspective of utilizing network bandwidth resources, the TCP header at the transport layer plus the IP header at the network layer amount to a minimum of 40 bytes. To send just a few bytes of packet data, an additional 40 bytes of headers are assembled, which is somewhat akin to the so-called silly window syndrome.

From the perspective of application optimization, since application-layer data can only be sent after the TCP connection is established via the three-way handshake, this can cause a certain delay in the initial data transmission of applications, especially in scenarios involving short-lived connections and mobile devices, exacerbating the side effects.

How can this issue be solved? The solution is to use TFO (TCP Fast Open).

PS: Due to differences between versions of the TCP protocol stack, the premise of this article is that data is not transmitted during the three-way handshake (traditional three-way handshake). In fact, many cloud service providers offer optimized Linux distributions that allow data to be sent directly during the third handshake of TCP. When readers perform packet captures themselves, there may be some differences from the results presented in this article.

TFO (TCP Fast Open)

TCP Fast Open (TFO) optimizes the traditional three-way handshake by allowing data to be sent during the handshake process, thereby reducing the latency of the first data transmission and improving the performance of network applications.

Implementation Principle

The core principle of TFO is to introduce a Cookie mechanism in the communication between the sender and receiver, allowing the sender to simplify the TCP three-way handshake when reconnecting to the receiver later.

Just like how Cookies work in the Web application layer, the first visit requires authentication, and then the server validates it. Subsequent visits can carry the Cookie directly without needing to log in again.

1. Initial Connection

  • When the sender establishes a TCP connection with the receiver for the first time, it sends a SYN packet.
  • The receiver responds with a SYN-ACK packet, including a randomly generated identifier called the TFO Cookie to the sender.
  • After receiving the SYN-ACK packet, the sender saves the TFO Cookie, sends an ACK packet to the receiver, completes the three-way handshake, and starts transmitting data.

2. Subsequent Connections

  • When the sender reconnects to the same receiver, it can include the previously saved TFO Cookie in the SYN packet and also attach application-layer data (i.e., send data directly during the first handshake).
  • The receiver validates the sender's TFO Cookie, forwards the data to the application layer for processing, and responds with a SYN-ACK packet (and can also send data).
  • Upon receiving the SYN-ACK packet, the sender sends an ACK packet to the receiver, completing the three-way handshake.

Advantages

With TFO, the sender can attach data directly with the SYN packet, allowing the receiver to process data during the first handshake and send data during the second handshake. Ultimately:

  • The sender reduces the latency by 1.5 RTTs when sending data for the first time.
  • The receiver reduces the latency by 1 RTT when sending data for the first time.

Limitations

Compatibility

Both communicating parties need to support TFO. If one party does not support TFO, the connection falls back to the traditional TCP connection establishment process. Moreover, forwarding devices in the communication link (such as NAT and firewalls) must also support this compatibility mechanism.

Security

Although the TFO Cookie is generated by the receiver and sent to the sender and each cookie is associated with the sender, it increases the attack surface for the receiver, potentially leading to security risks such as "TCP SYN Flood" amplification attacks.

If an attacker obtains a valid TFO Cookie from a compromised host and forges a large number of data-carrying packets, the receiver would need a significant amount of memory to temporarily store the application data, eventually leading to memory exhaustion.

Deployment Environment Requirements

Kernel version requirements apply, and kernel parameters need to be adjusted.

Large Application Data

If the data the sender needs to transmit initially exceeds the Maximum Segment Size (MSS) of TCP, it still needs to be fragmented and sent in multiple packets. When the application data is large, the advantages brought by TCP Fast Open (reduced RTT) become negligible.

Simulation Environment

To demonstrate the functionality of TFO (TCP Fast Open), the author used two Linux servers as the sender and receiver, respectively. The corresponding distribution and kernel versions are listed below.

TFO requires Linux kernel version ≥ 3.7.

Sender Side

Distribution (WSL2 Environment)

PRETTY_NAME="Debian GNU/Linux 11 (bullseye)"
NAME="Debian GNU/Linux"
VERSION_ID="11"
VERSION="11 (bullseye)"
VERSION_CODENAME=bullseye
ID=debian

Kernel Version

5.10.0-21-amd64

Receiver Side

Distribution

NAME="CentOS Linux"
VERSION="7 (Core)"
ID="centos"
ID_LIKE="rhel fedora"
VERSION_ID="7"
PRETTY_NAME="CentOS Linux 7 (Core)"

Kernel Version

3.10.0-1160.53.1.el7.x86_64

Kernel Parameter Adjustment

Enabling TFO requires modifying the default kernel parameters:

  • 0: Disable TFO
  • 1: Enable TFO in sender mode
  • 2: Enable TFO in receiver mode
  • 3: Enable TFO in both sender and receiver modes

Enabling TFO on the Sender Side

echo 1 | sudo tee /proc/sys/net/ipv4/tcp_fastopen

Enabling TFO on the Receiver Side

echo 3 | sudo tee /proc/sys/net/ipv4/tcp_fastopen

For the parameters to persist across reboots, edit /etc/sysctl.conf and add the line net.ipv4.tcp_fastopen=3. Then run sysctl -p to apply the changes, ensuring they remain effective after reboot.

Program Code

If your curl version supports TFO, you can enable it directly with the following command:

curl --tcp-fastopen http://example.com

To check if your curl version supports TFO, run:

curl -V | grep -i TFO

Since the curl version on the author's server is relatively old, a Python script is provided here. The key part of the code is setting the socket options for TFO.

Receiver (Server) Code

The receiver acts as the server program, binding to and listening on a specified port, and accepting incoming TCP connections from the sender (client).

# service.py

import socket

def listen():
    # Initialize the server socket object
    listener = socket.socket(socket.AF_INET, socket.SOCK_STREAM)

    # Set the option for TCP Fast Open queue length
    listener.setsockopt(socket.SOL_TCP, socket.TCP_FASTOPEN, 32)

    # Bind to port 12345
    listener.bind(('0.0.0.0', 12345))
    # Set the maximum number of pending connections to 1024
    listener.listen(1024)

    print("Server is listening on port 12345...")

    # Poll for new TCP connections
    while True:
        conn, addr = listener.accept()
        print(f"Accepted connection from {addr}")
        print(f"Received data: {conn.recv(1024)}")

        conn.send(b"Hello, Client")
        conn.close()

        print(f"Closed connection with {addr}")

if __name__ == "__main__":
    try:
        # Start listening
        listen()
    except KeyboardInterrupt:
        # Handle Ctrl + C to terminate the program
        print("Server shutting down...")

Sender (Client) Code

Initialize the client socket object and send data to the server using TFO option.

# client.py

import socket

# Initialize the client socket object
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)

# Set the option for TCP Fast Open queue length
sock.setsockopt(socket.SOL_TCP, socket.TCP_FASTOPEN, 32)

# Send data to the server with Fast Open option
sock.sendto(b"Hello, Server", ("104.21.71.166", 12345))

print(f"Sent data to the server.")

sock.close()

Note: The sendto method with the MSG_FASTOPEN flag is used to indicate that the socket should attempt to use TFO. However, the Python standard library does not directly support the MSG_FASTOPEN flag, and thus the example above uses sendto without the flag, which might work depending on the underlying system implementation. For precise control over TFO, you might need to use lower-level interfaces or libraries that support this functionality directly.

Running Program Experiment

The core code (just a couple of lines) is ready. Let's proceed with running the program to verify the TCP Fast Open process.

Public IP of the Server: 104.21.71.166

Step-by-Step Instructions

1. Start the Server Program and Verify Listening Status

# Start the Server Program in One Terminal
$ python3 server.py

# Check the Listening Status in Another Terminal
$ netstat -ant | grep 12345 | grep LISTEN
tcp        0      0 0.0.0.0:12345            0.0.0.0:*               LISTEN

2. Begin Packet Capture on the Client

   Open Wireshark and start capturing packets on the appropriate network interface.

3. Run the Client Program

   Run the client script several times with intervals of 3-5 seconds to ensure the effect is observable.

$ python3 client.py

4. Check TCP Connection Status on the Client

   Use netstat to check the status of the TCP connections created by the client program.

$ netstat -ant | grep 12345 | grep TIME_WAIT

   Output should show multiple connections in the TIME_WAIT state if the client script was run multiple times.

tcp        0      0 10.0.0.53:38084         104.21.71.166:12345       TIME_WAIT  
tcp        0      0 10.0.0.53:37530         104.21.71.166:12345       TIME_WAIT  
tcp        0      0 10.0.0.53:37528         104.21.71.166:12345       TIME_WAIT  
tcp        0      0 10.0.0.53:38076         104.21.71.166:12345       TIME_WAIT  
tcp        0      0 10.0.0.53:38078         104.21.71.166:12345       TIME_WAIT  

...

Analyzing Wireshark Capture Results

First, apply the filter tcp.options.tfo to quickly find packets related to TCP Fast Open.

Initial Connection Establishment

When the sender establishes the initial TCP connection with the receiver, it sends a SYN packet with the TCP Fast Open option set. No data is sent yet, so the packet length (`Len`) is zero.

The receiver responds with a SYN-ACK packet, including a randomly generated TFO Cookie.

Upon receiving the SYN-ACK packet, the sender saves the TFO Cookie and sends an ACK packet to complete the three-way handshake.

Subsequent Connections

From the screenshots, it can be seen that when the client and server subsequently establish a TCP connection, the TFO Cookie is carried in the first handshake along with immediate data transmission, resulting in a Len = 13 in the Wireshark capture results.

So what is this 13? It is the data sent by the client, exactly 13 bytes.

conn.sendto(b"Hello, Server", ...)

During the subsequent TCP connection establishment (first handshake), data can be sent directly (due to space limitations, only two packet capture details are shown here):

The value of the TFO Cookie in each packet is d82d9074a6105a13, which is the same value that was included in the SYN-ACK message sent by the server during the initial TCP connection setup.

This experiment demonstrates the benefits of TCP Fast Open by reducing the latency involved in establishing new TCP connections, particularly for subsequent connections where a TFO Cookie is available.

References

Translated from https://dbwu.tech/posts/network/what-is-tcp-fast-open/

NETWORK  TCP  REASON  EXPLANATION 

       

  RELATED

  0 COMMENT

No comment for this article.


  RANDOM FUN

When push code directly to production