Manhattan Melodrama - Wikipedia

Introduction

So you’ve heard about sed and awk, two legendary tools from the golden age of UNIX. But now you’re stuck in a classic dilemma: Which one should you use? Or, should you use both?

It’s like choosing between a sword (sed) and a magic staff (awk). One is great for quick, powerful cuts, while the other is an entire spellbook for data manipulation. This guide will help you figure out when to reach for sed, when to unleash awk, and when to combine them for ultimate text-processing domination.

A Brief History of AWK and SED

Both awk and sed were born at Bell Labs, the legendary birthplace of UNIX.

• sed (Stream Editor) was created in 1973 by Lee E. McMahon. It was designed as a non-interactive text editor, perfect for batch editing and automation.
• awk was created in 1977 by Alfred Aho, Peter Weinberger, and Brian Kernighan. Unlike sed, it wasn’t just for substitution—it had built-in programming constructs, making it a lightweight scripting language for text processing.

For the history buffs, here are the Wikipedia pages:

• SED: https://en.wikipedia.org/wiki/Sed
• AWK: https://en.wikipedia.org/wiki/AWK

Both tools became core components of UNIX and Linux, and today, they are still widely used.

When to Use sed vs awk

Here’s the quick and dirty comparison:

Rule of thumb:

• If you just need to replace, delete, or filter text, use sed.
• If you need complex text processing, calculations, or column-based operations, use awk.
• If you want to automate everything like a wizard, use them together!

SED vs AWK: Side-by-Side Examples

1. Replace Text

Using sed (Better for simple substitutions):

1

sed 's/error/fix/g' logs.txt

Using awk (Overkill for this task):

1

awk '{ gsub("error", "fix"); print }' logs.txt

2. Extract a Column

Using awk (Perfect for this task):

1

awk '{ print $2 }' file.txt

(Prints the second column from a file)

Using sed (Hacky, not recommended):

1

sed -E 's/^[^ ]+ ([^ ]+).*/
/' file.txt

3. Delete Blank Lines

Using sed (Best for this):

1

sed '/^$/d' file.txt

Using awk (Also works, but overkill):

1

awk 'NF > 0' file.txt

4. Sum a Column of Numbers

Using awk (Best choice, as sed can’t do math):

1

awk '{ sum += $2 } END { print sum }' numbers.txt

Using sed (Not possible—sed doesn’t do math! 😢)

Using sed and awk Together

Why choose when you can have both? Here’s how to combine sed and awk for maximum efficiency.

1. Pre-process with sed, then Format with awk

1

sed 's/,/ /g' data.csv | awk '{ print $1, $3 }'

(Replaces commas with spaces, then prints columns 1 and 3)

2. Extract Lines with sed, Process with awk

1

sed -n '5,10p' file.txt | awk '{ print $2 }'

(Extracts lines 5-10, then prints column 2)

3. Remove Duplicates with sed, then Summarize with awk

1

sed 's/foo/bar/g' data.txt | awk '{ sum += $2 } END { print sum }'

(Replaces “foo” with “bar”, then sums column 2)

Syntax Tables

SED Syntax Reference

AWK Syntax Reference

Conclusion

• Use sed for quick substitutions, deletions, and simple text processing.
• Use awk for working with structured data, arithmetic, and complex text transformations.
• Use both together for the ultimate power combo.

Now, go forth and automate!

References

1. SED Wikipedia: https://en.wikipedia.org/wiki/Sed
2. AWK Wikipedia: https://en.wikipedia.org/wiki/AWK
3. GNU Sed Manual: https://www.gnu.org/software/sed/manual/sed.html
4. GNU AWK Manual: https://www.gnu.org/software/gawk/manual/gawk.html