<aside> 🧭
Module 11 · Inter-process communication
Every module so far has treated processes as isolated. They are not — they talk constantly, and almost all of it goes through three mechanisms. This module covers what a pipe really is, why a socket is just a file descriptor, how shared memory avoids copying entirely, and what is actually happening when a program is stuck waiting for something that never arrives.
🧠 concept → 🧩 real-world analogy → 🧪 exercise → ✅ expected result (hidden) → 🎯 interview questions (hidden)
</aside>
<aside> ✅
Before you start, you should already know:
From Module 02 — fork, exec, and that a child inherits its parent's open files.
From Module 03 — file descriptors, the lowest-unused-fd rule, redirection, inodes, and RLIMIT_NOFILE.
From Module 04 — signals and dispositions, and that SIGPIPE's default action is to kill the process.
From Module 08 — that two processes can map the same physical page.
From Module 09 — what Shmem is, and why memory counted inside Cached is not always reclaimable.
Tools used here that are not installed by default on Ubuntu: sudo apt-get install -y lsof strace socat netcat-openbsd.
</aside>
<aside> 📖
Official docs: pipe(7) · pipe(2) · fifo(7)
</aside>
A pipe is a fixed-size buffer in kernel memory with a file descriptor at each end. That is the whole thing. One descriptor can only be written to, the other can only be read from, and the data never touches a disk.
When you type ls | wc -l, the shell calls pipe() to create that buffer, then forks twice. In the first child it makes the pipe's write end become file descriptor 1 and runs ls; in the second it makes the read end become descriptor 0 and runs wc. Neither program knows a pipe is involved — ls writes to stdout exactly as it always does. This is the redirection mechanism from Module 03, applied to a pipe instead of a file.
Two consequences that surprise people:
The stages run at the same time. ls does not finish and then hand its output to wc. Both run concurrently, and the pipe buffer is what lets them run at different speeds.
A pipe has an inode, but no name. ls -l /proc/PID/fd shows entries like pipe:[482913]. That number is an inode in a virtual filesystem, and it is the only way to work out which two processes are connected — matching inode numbers is how you find the other end.
A FIFO, also called a named pipe, is the same object with a filename attached so that unrelated processes can find it. The name is a doorway; the data still lives only in kernel memory and never touches the disk the name sits on.
<aside> 🧠
Counter-intuitive. Creating a FIFO with mkfifo makes a file you can see with ls, and it always shows size 0 no matter how much data has flowed through it. Nothing is ever stored there. If you cat a FIFO with no writer attached, you do not get an empty file — you block, possibly forever, which is the single most common way a shell script hangs with no error message.
</aside>
<aside> 🧩
Real-world analogy — the pass-through hatch
A kitchen and a dining room share a hatch in the wall with a shelf in it. The chef puts plates on the shelf; the waiter takes them off. Neither ever sees the other.
The shelf holds a fixed number of plates, and that single fact produces almost all pipe behaviour. If the chef is faster, the shelf fills and the chef has to stop and wait — not because anything is broken, but because there is nowhere to put the next plate. If the waiter is faster, the shelf empties and the waiter waits.
Both work at once. The chef does not cook the entire service and then call the waiter. That is why a pipeline starts producing output immediately instead of after the first command finishes.
An anonymous pipe is a hatch built into the wall between two specific rooms when they were built — only those two rooms can use it, and it exists only while they do. A FIFO is a hatch with a label on the corridor side, so anyone who knows the label can walk up and use it. The label is on the wall; the plates are still only ever on the shelf.
Where the analogy stops working. A waiter can see how full the shelf is. Neither end of a pipe can ask how much data is buffered — you find out only by blocking.
</aside>
🧪 Exercise A1.1 — Find both ends of a pipe
# A pipeline whose two halves stay alive so we can inspect them
( sleep 60 | sleep 60 ) &
sleep 1
# Find the two processes
pgrep -a sleep | tail -2
P1=$(pgrep sleep | tail -2 | head -1)
P2=$(pgrep sleep | tail -1)
# Their file descriptors. Look for pipe:[NNNN].
echo "--- writer (PID $P1) ---"
ls -l /proc/$P1/fd
echo "--- reader (PID $P2) ---"
ls -l /proc/$P2/fd
# The inode number is the join key. Find everything using it.
INODE=$(readlink /proc/$P1/fd/1 | sed 's/[^0-9]//g')
echo "--- everything holding pipe inode $INODE ---"
# Compare each symlink target directly. `ls -l` over many directories prints
# a header per directory, and grep would throw those headers away - leaving
# you knowing that two descriptors exist but not which processes hold them.
for f in /proc/[0-9]*/fd/*; do
[ "$(readlink "$f" 2>/dev/null)" = "pipe:[$INODE]" ] && echo "$f"
done 2>/dev/null
# lsof does the same job in one command, and names the processes
sudo lsof 2>/dev/null | awk -v i="$INODE" '$0 ~ ("FIFO") && $0 ~ i'
kill $P1 $P2 2>/dev/null
<aside> 🏭
Now imagine this at 500 hosts. Pipe inodes are how you untangle a stuck pipeline in production, and it is worth knowing before you need it. A common failure: a log-shipping pipeline where the consumer has died, the producer is blocked writing, and the service looks hung with nothing in its logs — because the log line it is trying to emit is the one that will not fit. ls -l /proc/PID/fd plus a matching inode search finds it in under a minute, with no tooling installed.
</aside>