<aside> 🔀
Module 4 — Streams, Redirection, and Pipes. Logs, /dev/null, 2>&1, and pipelines — how data actually flows between commands, and how to send it where you want.
🧠 concept → 🧩 analogy → 🧪 exercise → ✅ expected result (hidden) → 🎯 interview questions (hidden)
</aside>
<aside> 📋
Before you start. You need Modules 1–3: scripts, quoted variables, and exit codes (this module keeps using verdicts, and finally explains where error messages live). Tools: nothing new. Keep ~/bash-course — the users.txt file from Module 3 gets reused.
</aside>
<aside> 📖
Official docs: Bash manual — Redirections
</aside>
Every running command is born holding three open channels, numbered like phone extensions: 0 — standard input (stdin), where it reads from; 1 — standard output (stdout), where it sends its results; 2 — standard error (stderr), where it sends complaints and commentary. In a terminal, 1 and 2 both happen to point at your screen — which is why you have never had to tell them apart. But they are two genuinely separate channels, and every command you have run in this track has been using both: ls hello.sh /nonexistent sends hello.sh out of mouth 1 and the cannot access complaint out of mouth 2, interleaved on the same screen.
Why two mouths? So that results stay clean even when complaints happen. A command's stdout may be feeding a file or another program (Part C); its grumbling must not contaminate the data. The entire Unix toolset is built on this discipline: results → 1, diagnostics → 2.
<aside> 🧩
Real-world analogy — the invoice and the phone call. A supplier fulfilling your order sends two kinds of communication: the paperwork (invoice, delivery note — the result, filed neatly), and the phone call ("your usual brand was out of stock" — the commentary). You would never want the phone call transcribed into the middle of the invoice; accounting software reading the invoice would choke on it. Separate channels exist so machines can read the paperwork while humans hear the phone calls.
Where the analogy stops working. On your terminal screen the two channels arrive visually identical — nothing marks a line as stdout or stderr, and their interleaving order isn't even guaranteed (the channels are buffered differently). You cannot tell them apart by looking; you tell them apart by redirecting one and seeing what moves. That experiment is Exercise 4.1, and it is the fastest stderr diagnosis there is.
</aside>
🧪 Exercise 4.1 — split the invisible
The ls fails on purpose — half of it.
cd ~/bash-course
ls hello.sh /nonexistent # both channels hit the screen, mixed
ls hello.sh /nonexistent > /tmp/out.txt # > captures channel 1 into a file
cat /tmp/out.txt
> and >>: overwrite vs append<aside> 📖
Official docs: Bash manual — Redirections
</aside>
command > file sends stdout into a file — and if the file exists, its old contents are destroyed first, before the command even runs. command >> file appends to the end instead, creating the file if needed. That single-character difference is the difference between a log and an amnesiac: a nightly job writing > report.log keeps only the last night forever.
<aside> 🧠
Counter-intuitive: the truncation happens before the command runs — bash prepares the redirection first, then starts the command (it must: the command needs its channels already connected when it is born). Two consequences worth engraving. First, wc -l data.txt > data.txt destroys data.txt before wc reads a single line — the file is emptied by the redirect, then counted: 0. Never redirect a command's output onto its own input file. Second, > file with no command at all is a legal line that just empties a file — occasionally useful, frequently an accident.
</aside>
<aside> 🧩
Real-world analogy — the whiteboard and the logbook. > writes on the meeting-room whiteboard: whoever writes next erases you first — the board only ever shows the latest meeting. >> writes in the ship's logbook: every entry goes below the previous one, and the history is the point.
Where the analogy stops working. A human erases a whiteboard when they need the space. Bash erases at setup time, every time — even if the command then produces no output at all, the old contents are already gone. An erased-but-never-rewritten whiteboard (an empty log after a failed job) is a state the analogy never predicts, and it is a real thing you will meet in production.
</aside>
🧪 Exercise 4.2 — the amnesiac and the historian
The second command destroys data on purpose.