<aside> ✂️
Module 7 — Text Surgery: sed and awk
Module 6 found the text; this module changes and computes with it. sed edits streams — replace, delete, extract, at any scale. awk treats text as columns and rows — filter, reshape, count, sum. With their supporting cast (sort, uniq, cut, tr), they turn logs and configs into answers. These are the commands that make interviewers sit up.
Legend used throughout: 🧠 concept → 🧩 analogy → 🧪 exercise → ✅ expected result (hidden) → 🎯 interview questions (hidden)
</aside>
<aside> 🚦
Before you start
You need Modules Module 1 — What Linux Is–Module 6 — Finding Things: find and grep — regexes, pipes, quoting, and Module 6's ~/search playground, which this module extends. Add the web-access log it will dissect (same paste-the-block technique as Module 6):
cd ~/search
cat > app/logs/access.log <<'EOF'
10.0.0.5 GET /api/users 200 123
10.0.0.7 GET /api/orders 200 87
10.0.0.5 POST /api/orders 500 431
10.0.0.9 GET /health 200 5
10.0.0.5 GET /api/users 200 119
10.0.0.7 POST /api/login 401 12
10.0.0.9 GET /health 200 4
10.0.0.5 GET /api/orders 500 502
EOF
Columns, left to right: client IP, method, path, status code, response time (ms). Every exercise reads them.
</aside>
<aside> ℹ️
This page uses Mermaid diagram blocks. Notion shows them as code by default — click the block and set it to Preview to see the diagram. This reminder appears once per page.
</aside>
<aside> 📖
Official docs: GNU sed manual · sed(1)
</aside>
🧠 sed 'script' file reads the file line by line, applies the script to each line, prints the result — the file itself is untouched; you are editing the stream. The script you will write most: s/old/new/ — substitute. Add g (s/old/new/g) to replace every occurrence on a line rather than just the first. The pattern side is a regex (Module 6's language, working overtime here); the script travels in single quotes (Module 5's rule — sed's $, *, and friends must reach sed intact). When your pattern contains slashes — file paths! — swap the delimiter for anything: s|/old/path|/new/path|.
<aside> 🧩
Real-world analogy — find-and-replace in the photocopier
sed is a photocopier with find-and-replace built in: originals feed in one side, corrected copies emerge from the other, the originals returning to their folder unmarked. Editing happens in the copy stream — which is why sed composes with pipes so naturally.
Where the analogy stops working. A copier's replace would surely fix every occurrence on a page. sed replaces only the first match per line unless you append g — a pure convention with no physical intuition behind it, and the number-one sed bug: the report that fixed one of the two hostnames on the line.
</aside>
🧪 Exercise A1.1 — Replace, and mind the g
cd ~/search
sed 's/db-01/db-03/' app/config/app.conf # stream out an edited copy
cat app/config/app.conf # original: untouched
echo "aaa bbb aaa" | sed 's/aaa/XXX/' # first match only...
echo "aaa bbb aaa" | sed 's/aaa/XXX/g' # ...vs all matches
<aside> 📖
Official docs: GNU sed manual
</aside>
🧠 Any sed command can be restricted to an address: a line number (2), a range (1,3), or a pattern (/^#/). Combined with the commands p (print — used with -n, which silences the default line-echo) and d (delete from the stream), you get surgical extraction: sed -n '2p' — only line 2; sed -n '1,3p' — lines 1–3; sed '/^#/d' — everything except comments. Address + s/// also composes: sed '/^db/s/-01/-03/' — substitute only on lines starting with db.
<aside> 🧩
Real-world analogy — the editor's blue pencil
A copy editor marks a manuscript: "strike every line beginning with a stage direction", "print me only paragraphs two through four". Addresses are the margin instructions; commands are the pencil strokes.
Where the analogy stops working. An editor flips back and forth through the manuscript. sed's pages stream past exactly once — an address can never mean "the line before the match" or "go back and fix line 3" (there are dark arts for lookback, but the honest answer is: that's awk's or an editor's job). One pass, forward only, is the price of handling infinite streams.
</aside>
🧪 Exercise A2.1 — Extract and strike
cd ~/search
sed -n '2p' app/config/app.conf # line 2 only
sed -n '1,3p' app/config/app.conf # a range
sed '/^#/d' app/config/app.conf # strike the comments