SHAM: Reliable Transport over UDP

The problem

UDP is best-effort by design. Hand it a datagram and it may arrive, arrive twice, arrive after the datagram you sent later, or silently vanish, and the protocol will never tell you which happened. There is no connection, no acknowledgement, no ordering, no flow control. Every guarantee that makes a byte stream feel dependable lives one layer up, in TCP, and most software rightly never looks inside it.

The point of SHAM was to refuse that shortcut once: rebuild the reliability layer myself, in C, directly on UDP sockets (connection establishment, sequencing, acknowledgement, retransmission, reassembly, flow control), so that nothing about how data survives a lossy network is magic anymore. If a segment disappears mid-path, my code has to notice, time out, and resend it; if segments arrive scrambled, my code has to put them back in order before the application sees a single byte.

Interactive: drop packets, watch it recover

This is a miniature implementation of SHAM's two state machines, sender on the left and receiver on the right, with every packet a live object in flight between them. The presets script the network conditions, but you can play the network too: destroy any packet mid-flight and the protocol recovers on its own. Constants are rescaled for the screen (window of 3 instead of 10, RTO stretched to 4.5 s); the log mirrors the real SND/RCV/RETX/DROP traces the implementation writes to client_log.txt and server_log.txt.

How it works

SHAM is a user-space transport written in C against raw UDP sockets. Every packet on the wire is a UDP datagram carrying a fixed 12-byte SHAM header followed by at most SHAM_MSS_DATA (1024) payload bytes.

A 12-byte header, packed by hand

The header is four fields: seq_num, ack_num, flags, and window_size. Each is serialized explicitly with sham_pack() / sham_unpack() using htonl/htons and memcpy, so the wire format is network byte order at fixed offsets rather than whatever the compiler decides a struct looks like in memory.

Connection lifecycle

A connection opens with a 3-way handshake (SYN → SYN-ACK → ACK) using random initial sequence numbers on each side, and closes with a 4-way FIN teardown so both directions of the stream shut down cleanly.

Sliding window, cumulative ACKs, flow control

The sender keeps up to SHAM_SND_WIN_PKTS (10) unacknowledged segments in flight. ACKs are cumulative: ACK=k means everything through k has been received, so one arriving ACK can slide the window past several segments at once, and duplicate ACKs are detected as a signal that something in between is missing. Flow control is receiver-driven: the receiver's free buffer (SHAM_RECV_BUF_CAP) rides in every header's window_size, and the sender never puts more data in flight than the receiver has advertised room for.

Loss recovery and reassembly

Unacknowledged segments are retransmitted after a fixed timeout, SHAM_RTO_MS (500 ms), which restarts whenever an ACK makes progress. On the other side, segments that arrive early are buffered and the application is handed bytes strictly in sequence. Retransmission and reassembly together turn a lossy, reordering channel back into a stream. The same transport runs in two modes: file transfer, where the server prints the MD5 of the received file for an end-to-end integrity check, and interactive chat.

Validation: reproducible packet loss

Rather than depending on a genuinely lossy network, both endpoints run every received packet through a loss filter (should_drop_rx) gated by a loss_rate command-line argument, so any loss level is reproducible on loopback. Dropped packets are logged, the sender's RTO fires, segments are retransmitted, and both sides write timestamped packet logs (server_log.txt / client_log.txt) tracing every SND/RCV/RETX/DROP. A run at 20% loss still ends with the received file's MD5 matching the source.

Design decisions & trade-offs

Fixed RTO instead of adaptive estimation. The timeout is a single constant in sham.h, which makes retransmission behavior deterministic and tunable in one line, the right property for a protocol whose goal is to make every mechanism observable. Real TCP estimates the timeout from measured round-trip times (Jacobson/Karels: a smoothed RTT plus a variance term), and that is exactly what an adaptive RTO would buy here: no spurious retransmissions on slow paths and faster recovery on fast ones. At teaching scale on loopback, the fixed constant is the honest choice; on real, variable-latency paths it would be the first thing to replace.

Cumulative ACKs instead of SACK. A single integer summarizes the entire receiver state, which keeps the header fixed-size and duplicate-ACK detection trivial. The cost is sender blindness past the first gap: after a timeout it may retransmit data the receiver has already buffered out of order. Selective acknowledgements would avoid that redundant traffic, at the price of variable-length header options and considerably more sender bookkeeping.

Loss injected at the receiver, not the network. Because the drop filter runs inside the endpoints, every test is repeatable at an exact loss rate with full packet logs on both sides. The tests exercise the protocol, not the flakiness of a particular network setup.

Honest scope

SHAM implements reliability, ordering, and receiver-advertised flow control. It does not implement congestion control: the in-flight window is a fixed constant rather than something that adapts to what the network can carry.

That is why I don't describe it as "TCP over UDP". It makes no claim to TCP's congestion behavior. The natural next step is AIMD: slow start, congestion avoidance, and treating loss as a congestion signal, so the window grows and shrinks with the path instead of being a compile-time constant.

Try it yourself

make                        # builds ./server and ./client (needs OpenSSL headers)

# terminal 1 - receive into out.bin, dropping 20% of incoming packets
./server 9000 0.2

# terminal 2 - send big.bin across the deliberately lossy link
./client 127.0.0.1 9000 big.bin out.bin

md5sum big.bin              # matches the MD5 the server prints for out.bin

There is also an interactive chat mode: run ./server 9000 --chat on one side and ./client 127.0.0.1 9000 --chat on the other to exercise the same reliable stream in both directions.