Text Processing (grep/sed/awk)
18 min
Three tools cover most day-to-day text processing: grep PATTERN file
prints only the lines matching a pattern (-n adds line numbers);
sed 's/OLD/NEW/' replaces the FIRST match of OLD with NEW on each
line (add a trailing g to replace every match, not just the first);
awk '{print $N}' prints just the Nth whitespace-separated field of each
line (1-indexed — $1 is the first word, $2 the second). All three read
from a file OR from a pipe, which is what makes them so composable with
each other.
echo "alice 29" > people.txt
echo "bob 34" >> people.txt
awk '{print $1}' people.txt
# alice
# bob
sed 's/alice/ALICE/' people.txt
# ALICE 29
# bob 34
awk pulls out just the score column; piping that into sort -n turns it into a sorted list of just the numbers, with no separate temp file needed anywhere in between.
A file `log.txt` already exists. Redact every occurrence of the word `secret` to `***`, then keep only the lines that contain `ERROR`, and write those (already-redacted) lines to `redacted-errors.txt`.
What does the trailing `g` mean in `sed 's/a/b/g'`?
You can filter, substitute, and extract fields from text using grep, sed, and awk, and chain them together with pipes.