TCP and Its Three-Way Handshake
Thilan Dissanayaka Computer Networking May 01, 2020

TCP and Its Three-Way Handshake

Every time you open a website, send an email, or download a file, something happens before a single byte of actual data is transferred. Two machines — your computer and a server somewhere on the internet — go through a quick negotiation. They introduce themselves, agree on some ground rules, and only then start exchanging data.

This negotiation is the TCP three-way handshake, and it’s one of the most fundamental processes in computer networking. If you’ve ever captured packets in Wireshark and seen those SYN, SYN-ACK, ACK packets at the start of every connection — that’s what we’re going to break down today.

What is TCP?

TCP (Transmission Control Protocol) is one of the core protocols in the Internet Protocol Suite. It lives at the Transport Layer (Layer 4) of the OSI model, sitting between the application layer (HTTP, SMTP, FTP) and the network layer (IP).

The key word with TCP is reliable. Unlike its sibling UDP (User Datagram Protocol), which just fires packets and hopes for the best, TCP guarantees:

  • Reliable delivery — Every byte you send will arrive at the other end, or you’ll know about it
  • Ordered transmission — Data arrives in the exact sequence it was sent
  • Error detection — Corrupted packets are detected and retransmitted
  • Flow control — The sender won’t overwhelm the receiver with more data than it can handle

This makes TCP the protocol of choice for applications where data integrity matters:

Application Protocol Why TCP?
Web browsing HTTP/HTTPS A missing packet means a broken web page
Email SMTP/IMAP You can’t have half an email
File transfer FTP/SFTP A corrupted file is useless
SSH SSH Every keystroke must arrive correctly

But all of this reliability comes at a cost — overhead. Before TCP can deliver any data, it needs to set up a connection. And that setup process is the three-way handshake.

The TCP Header — What’s Inside a TCP Packet?

Before we dive into the handshake, it helps to understand what a TCP packet actually looks like. Every TCP segment carries a header with critical metadata:

 0                   1                   2                   3
 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|          Source Port          |       Destination Port        |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|                        Sequence Number                        |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|                    Acknowledgment Number                      |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|  Data |       |U|A|P|R|S|F|                                   |
| Offset| Rsrvd |R|C|S|S|Y|I|            Window Size            |
|       |       |G|K|H|T|N|N|                                   |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|           Checksum            |         Urgent Pointer        |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

The fields that matter most for the handshake are:

  • Sequence Number — Tracks the position of data in the byte stream
  • Acknowledgment Number — Tells the other side “I’ve received everything up to this byte”
  • Flags — Single-bit flags that control the connection state:
    • SYN — Synchronize sequence numbers (used to initiate a connection)
    • ACK — Acknowledgment field is valid
    • FIN — Sender is finished sending data
    • RST — Reset the connection
    • PSH — Push data to the application immediately
  • Window Size — How much data the receiver can accept (flow control)

Now that we know what’s inside a TCP packet, let’s see how the handshake uses these fields.

The Three-Way Handshake — Step by Step

The purpose of the handshake is simple — both sides need to:

  1. Synchronize their sequence numbers — So they can track every byte of data
  2. Confirm both sides are ready — The client wants to send, the server is ready to receive
  3. Establish a reliable session — Before any application data flows

Let’s walk through it with concrete numbers.

Step 1: SYN — Client → Server

The client initiates the connection by sending a SYN (synchronize) packet. It picks a random Initial Sequence Number (ISN) — let’s say 1000.

Client → Server
┌─────────────────────────────────┐
│  SYN = 1                        │
│  SEQ = 1000                     │
│  ACK = 0                        │
│  "Hey server, I want to talk.   │
│   My starting number is 1000."  │
└─────────────────────────────────┘

At this point, the client enters the SYN_SENT state. It’s waiting for the server to respond.

Why a random ISN instead of starting at 0? Security. If sequence numbers were predictable, an attacker could forge packets by guessing the next sequence number. Randomized ISNs make TCP sequence prediction attacks significantly harder.

Step 2: SYN-ACK — Server → Client

The server receives the SYN, and if it’s willing to accept the connection, it responds with a SYN-ACK. It does two things simultaneously:

  1. Acknowledges the client’s sequence number by setting ACK = 1001 (client’s ISN + 1)
  2. Sends its own ISN — let’s say 5000
Server → Client
┌─────────────────────────────────────┐
│  SYN = 1,  ACK = 1                  │
│  SEQ = 5000                         │
│  ACK = 1001                         │
│  "Got it. My starting number is     │
│   5000. I expect your next byte     │
│   to be 1001."                      │
└─────────────────────────────────────┘

The server enters the SYN_RECEIVED state. It’s acknowledged the client and is now waiting for the final confirmation.

Step 3: ACK — Client → Server

The client receives the SYN-ACK and sends back a final ACK to acknowledge the server’s sequence number.

Client → Server
┌─────────────────────────────────────┐
│  ACK = 1                            │
│  SEQ = 1001                         │
│  ACK = 5001                         │
│  "Got your number. We're good       │
│   to go."                           │
└─────────────────────────────────────┘

Both sides enter the ESTABLISHED state. The connection is ready, and data can now flow in both directions.

The Complete Picture

    Client                                          Server
      |                                               |
      |  ──── SYN, SEQ=1000 ──────────────────────>   |
      |        (SYN_SENT)                              |
      |                                               |
      |  <──── SYN-ACK, SEQ=5000, ACK=1001 ────────   |
      |                              (SYN_RECEIVED)    |
      |                                               |
      |  ──── ACK, SEQ=1001, ACK=5001 ────────────>   |
      |       (ESTABLISHED)          (ESTABLISHED)     |
      |                                               |
      |  ═══════ DATA TRANSFER BEGINS ════════════    |

Three packets. That’s all it takes to establish a reliable, bidirectional communication channel. Every HTTP request, every SSH session, every file download starts with this exact dance.

Why Sequence Numbers Matter

Sequence numbers are the backbone of TCP’s reliability. Without them, TCP couldn’t do any of the things that make it reliable.

Tracking Data

Every byte in a TCP stream has a sequence number. If the client sends 500 bytes of data starting at sequence number 1001, the bytes are numbered 1001 through 1500. The receiver can then acknowledge: “I’ve received everything up to byte 1501 — send me the next chunk.”

Client sends:  SEQ=1001, 500 bytes of data
Server replies: ACK=1501  ("I got bytes 1001-1500, send 1501 next")

Client sends:  SEQ=1501, 300 bytes of data
Server replies: ACK=1801  ("I got bytes 1501-1800, send 1801 next")

Duplicate Detection

Networks aren’t perfect. Packets can take different routes, arrive out of order, or even be duplicated. If the receiver gets a packet with SEQ=1001 twice, it knows it’s a duplicate because it’s already acknowledged bytes up to 1501. It simply discards the duplicate.

Retransmission

If the sender doesn’t receive an ACK within a timeout period, it assumes the packet was lost and retransmits it. The sequence number tells the receiver exactly which bytes are being resent, so it can slot them into the correct position in the stream.

Real-World Analogy

Imagine you’re mailing chapters of a book to a friend, one envelope at a time:

  • You number each envelope (sequence numbers)
  • Your friend sends you a postcard after receiving each one: “Got chapter 5, send me chapter 6” (acknowledgments)
  • If chapter 3 gets lost in the mail, your friend says: “I have chapters 1, 2, 4, 5 — please resend chapter 3” (retransmission)
  • If chapter 4 arrives twice, your friend just throws away the extra copy (duplicate detection)

That’s exactly what TCP does, but at the byte level, and billions of times faster.

Seeing It in Action — Wireshark and tcpdump

Theory is great, but nothing beats seeing the handshake happen in real time. Let’s capture it.

Using tcpdump

Open a terminal and start capturing on your network interface:

$ sudo tcpdump -i eth0 -n tcp port 443 -c 3

Now open a website in another terminal:

$ curl https://example.com

You’ll see something like this:

10:32:01.123456 IP 192.168.1.10.54321 > 93.184.216.34.443: Flags [S], seq 1847291356, win 65535, length 0
10:32:01.145678 IP 93.184.216.34.443 > 192.168.1.10.54321: Flags [S.], seq 2981473625, ack 1847291357, win 65535, length 0
10:32:01.145789 IP 192.168.1.10.54321 > 93.184.216.34.443: Flags [.], ack 2981473626, win 65535, length 0

There it is — the three-way handshake in three lines:

  1. [S] — SYN from client (seq 1847291356)
  2. [S.] — SYN-ACK from server (seq 2981473625, ack 1847291357)
  3. [.] — ACK from client (ack 2981473626)

Notice how the ACK numbers are always the other side’s sequence number + 1. That’s the pattern you’ll see in every single TCP connection.

Using Wireshark

If you prefer a GUI, Wireshark makes the handshake even more visual. Open Wireshark, start a capture, and filter by:

tcp.flags.syn == 1

This shows you every SYN and SYN-ACK packet — the beginning of every TCP connection on your machine. Click on any packet, and you can inspect the full TCP header — sequence numbers, flags, window size, everything we discussed above.

Connection Teardown — The Four-Way Handshake

We’ve talked about how connections are established. But how do they end? TCP uses a four-way handshake for connection teardown, because the connection is bidirectional — each side needs to close independently.

    Client                                          Server
      |                                               |
      |  ──── FIN, SEQ=5000 ──────────────────────>   |
      |        (FIN_WAIT_1)                            |
      |                                               |
      |  <──── ACK, ACK=5001 ──────────────────────   |
      |        (FIN_WAIT_2)        (CLOSE_WAIT)        |
      |                                               |
      |  <──── FIN, SEQ=8000 ──────────────────────   |
      |                              (LAST_ACK)        |
      |                                               |
      |  ──── ACK, ACK=8001 ──────────────────────>   |
      |        (TIME_WAIT)           (CLOSED)          |
      |                                               |
      |  ~~~ waits 2×MSL ~~~                          |
      |        (CLOSED)                                |

The key difference from the setup: the teardown has four packets instead of three. Why? Because when the client sends a FIN, it’s saying “I’m done sending.” But the server might still have data to send. So the server first ACKs the client’s FIN, finishes sending any remaining data, and then sends its own FIN.

The TIME_WAIT state at the end is also important — the client waits for twice the Maximum Segment Lifetime (2×MSL, typically 60 seconds) before fully closing. This ensures any delayed packets from the old connection don’t interfere with a new connection on the same port.

TCP States — The Full Picture

Throughout a connection’s lifecycle, both client and server move through a series of states. Here’s the complete TCP state machine:

State Description
CLOSED No connection exists
LISTEN Server is waiting for incoming connections
SYN_SENT Client has sent SYN, waiting for SYN-ACK
SYN_RECEIVED Server received SYN, sent SYN-ACK, waiting for ACK
ESTABLISHED Connection is open, data can flow
FIN_WAIT_1 Initiator sent FIN, waiting for ACK
FIN_WAIT_2 Initiator received ACK for FIN, waiting for other side’s FIN
CLOSE_WAIT Received FIN, sent ACK, waiting for app to close
LAST_ACK Sent FIN, waiting for final ACK
TIME_WAIT Waiting for delayed packets to expire (2×MSL)

You can see the current TCP states on your machine right now:

$ netstat -an | grep -c ESTABLISHED
47

$ netstat -an | grep -c TIME_WAIT
12

If you see a lot of TIME_WAIT connections on a server, it usually means the server is initiating a lot of short-lived connections — common with microservices or reverse proxies.

In a Security Context

Understanding the three-way handshake isn’t just academic — it has real implications for security.

SYN Flood Attack

One of the oldest and most well-known denial-of-service attacks exploits the handshake directly. The attacker sends a flood of SYN packets with spoofed source IP addresses. The server responds to each with a SYN-ACK and allocates resources for the half-open connection. But the final ACK never comes — because the spoofed IP didn’t initiate the connection.

Attacker → Server:  SYN (spoofed IP: 1.2.3.4)
Attacker → Server:  SYN (spoofed IP: 5.6.7.8)
Attacker → Server:  SYN (spoofed IP: 9.10.11.12)
... thousands more ...

Server → 1.2.3.4:   SYN-ACK  (no response — IP is spoofed)
Server → 5.6.7.8:   SYN-ACK  (no response — IP is spoofed)
Server → 9.10.11.12: SYN-ACK (no response — IP is spoofed)

The server’s connection table fills up with half-open connections, and legitimate users can’t connect. This is a SYN flood.

Defenses:

  • SYN cookies — The server doesn’t allocate resources until the handshake is complete. It encodes the connection state in the sequence number of the SYN-ACK itself.
  • Rate limiting — Limit the number of SYN packets from a single source.
  • Firewall rules — Drop suspicious SYN patterns.

Port Scanning

Tools like Nmap use the handshake for port scanning. A SYN scan (also called a half-open scan) sends a SYN to a target port:

  • If the port responds with SYN-ACK → the port is open
  • If the port responds with RST → the port is closed
  • If there’s no response → the port is filtered (likely behind a firewall)
$ nmap -sS -p 80,443,22 target.com

PORT    STATE    SERVICE
22/tcp  open     ssh
80/tcp  open     http
443/tcp open     https

The clever part? The scanner never sends the final ACK, so no full connection is established. This makes SYN scans stealthier than full TCP connect scans — the target’s application logs won’t record the connection.

TCP Reset Attacks

If an attacker can guess the correct sequence number for an active TCP connection, they can inject a RST (reset) packet and terminate the connection. This is why randomized ISNs are important — they make it exponentially harder to guess the right sequence number.

TCP vs UDP — When to Choose What

We’ve been talking about TCP, but it’s worth understanding when not to use it.

Aspect TCP UDP
Connection Connection-oriented (handshake required) Connectionless (just send)
Reliability Guaranteed delivery Best-effort delivery
Ordering Preserves order No ordering guarantee
Overhead Higher (headers + handshake) Lower (minimal headers)
Speed Slower (reliability costs time) Faster (no setup, no retransmission)
Use cases Web, email, file transfer, SSH DNS, video streaming, gaming, VoIP

UDP skips the handshake entirely — it just fires packets. That makes it faster but unreliable. For real-time applications like video calls or gaming, a dropped frame or two is acceptable — but the latency of TCP’s handshake and retransmission is not.

Final Thoughts

The three-way handshake might seem like a small detail, but it’s the foundation that makes reliable internet communication possible. Every web page you load, every file you download, every SSH session you open — it all starts with those three packets: SYN, SYN-ACK, ACK.

Understanding how it works gives you a deeper appreciation for what’s happening beneath the surface. And if you’re into security — understanding the handshake is essential for recognizing SYN floods, port scans, and connection manipulation techniques.

Next time you fire up Wireshark or tcpdump, filter for SYN packets and watch the handshake unfold in real time. There’s something satisfying about seeing those three packets dance across the wire.

Thanks for reading!

ALSO READ
Blockchain 0x000 – Understanding the Fundamentals
May 21, 2020 Web3 Development

Imagine a world where strangers can exchange money, share data, or execute agreements without ever needing to trust a central authority. No banks, no intermediaries, no single point of failure yet...

Identity and Access Management (IAM)
May 11, 2020 Identity & Access Management

Who are you — and what are you allowed to do? That's the fundamental question every secure system must answer. And it's exactly what Identity and Access Management (IAM) is built to solve.

How I built a web based CPU Simulator
May 07, 2020 Pet Projects

As someone passionate about computer engineering, reverse engineering, and system internals, I've always been fascinated by what happens "under the hood" of a computer. This curiosity led me to...

Writing a Shell Code for Linux
Apr 21, 2020 Exploit Development

Shellcode is a small piece of machine code used as the payload in exploit development. In this post, we write Linux shellcode from scratch — starting with a simple exit, building up to spawning a shell, and explaining every decision along the way.

Exploiting a Stack Buffer Overflow on Windows
Apr 12, 2020 Exploit Development

In a previous tutorial we discusses how we can exploit a buffer overflow vulnerability on a Linux machine. I wen through all theories in depth and explained each step. Now today we are going to jump...

Access Control Models
Apr 08, 2020 Identity & Access Management

Access control is one of the most fundamental concepts in security. Every time you set file permissions, assign user roles, or restrict access to a resource, you're implementing some form of access control. But not all access control is created equal...

Exploiting a  Stack Buffer Overflow  on Linux
Apr 01, 2020 Exploit Development

Have you ever wondered how attackers gain control over remote servers? How do they just run some exploit and compromise a computer? If we dive into the actual context, there is no magic happening....

Basic concepts of Cryptography
Mar 01, 2020 Cryptography

Ever notice that little padlock icon in your browser's address bar? That's cryptography working silently in the background, protecting everything you do online. Whether you're sending an email,...

Common Web Application Attacks
Feb 05, 2020 Application Security

Web applications are one of the most targeted surfaces by attackers. This is primarily because they are accessible over the internet, making them exposed and potentially vulnerable. Since these...

Remote Code Execution (RCE)
Jan 02, 2020 Application Security

Remote Code Execution (RCE) is the holy grail of application security vulnerabilities. It allows an attacker to execute arbitrary code on a remote server — and the consequences are as bad as it sounds. In this post, we'll go deep into RCE across multiple languages, including PHP, Java, Python, and Node.js.