Quick Reference

Cheatsheets

Practical command references for Linux, networking, servers, containers, databases, and more.

Cheatsheet#sed-awk

Sed & Awk Cheatsheet

sed (Stream Editor) and awk (Pattern Scanning and Text Processing Language) are essential POSIX CLI tools for manipulating text files, parsing logs, extracting fields, and modifying code streams.

Sed (Stream Editor)

sed operates line-by-line, applying pattern matching rules to modify or filter text streams.

Search and Replace

# Replace first occurrence of 'foo' with 'bar' per line
sed 's/foo/bar/' file.txt
 
# Global replacement (all occurrences per line)
sed 's/foo/bar/g' file.txt
 
# Case-insensitive replacement
sed 's/foo/bar/gi' file.txt
 
# Replace on a specific line (e.g. line 5 only)
sed '5 s/foo/bar/' file.txt
 
# Replace on lines matching a pattern
sed '/pattern/ s/foo/bar/g' file.txt
 
# In-place file editing (modifies file directly)
sed -i 's/foo/bar/g' file.txt
 
# In-place file editing with backup creation (.bak)
sed -i.bak 's/foo/bar/g' file.txt

Deleting Lines

# Delete line 1 (header)
sed '1d' file.txt
 
# Delete lines 5 to 10
sed '5,10d' file.txt
 
# Delete last line ($)
sed '$d' file.txt
 
# Delete empty or whitespace-only lines
sed '/^$/d' file.txt
sed '/^[[:space:]]*$/d' file.txt
 
# Delete lines matching a regular expression
sed '/DEBUG/d' app.log

Line Insertion & Extraction

# Print only lines 10 through 20 (-n suppresses default output, p prints)
sed -n '10,20p' file.txt
 
# Insert text BEFORE line 1
sed '1i # Title Header' file.txt
 
# Append text AFTER line matching 'pattern'
sed '/pattern/a New appended line' file.txt
 
# Print lines between two patterns (Inclusive range)
sed -n '/START_TAG/,/END_TAG/p' file.txt

Awk (Text Processing Language)

awk reads input line-by-line and splits each line into fields based on a separator (default: whitespace).

  • $0 : The entire current line.
  • $1, $2, $3 : Field 1, Field 2, Field 3.
  • NF : Number of Fields in current line ($NF is the last field).
  • NR : Number of Records (Current line number).

Field Extraction & Formatting

# Print 1st and 3rd columns
awk '{print $1, $3}' file.txt
 
# Print last column of every line
awk '{print $NF}' file.txt
 
# Print line numbers along with lines
awk '{print NR, $0}' file.txt
 
# Change Output Field Separator (OFS) to comma
awk 'BEGIN {OFS=","} {print $1, $2, $3}' file.txt

Field Separators (-F)

# Use colon as separator (e.g. /etc/passwd usernames and home dirs)
awk -F: '{print $1, $6}' /etc/passwd
 
# Use comma as separator (CSV file processing)
awk -F',' '{print $1, $4}' data.csv
 
# Multi-character delimiter (e.g. " - ")
awk -F' - ' '{print $1}' server.log

Pattern Matching & Conditional Filtering

# Filter lines where 2nd column equals 'ERROR'
awk '$2 == "ERROR" {print $0}' app.log
 
# Filter HTTP 500 errors from Nginx access log (Status code in column 9)
awk '$9 == 500 {print $1, $7}' access.log
 
# Regex match: Filter lines where 1st column starts with '192.168.'
awk '$1 ~ /^192\.168\./ {print $1}' access.log
 
# Filter numeric values (Column 3 > 500)
awk '$3 > 500 {print $1, $3}' metrics.txt

Practical Awk One-Liners

# Calculate sum of numbers in 1st column
awk '{sum += $1} END {print "Total:", sum}' data.txt
 
# Calculate average of numbers in 2nd column
awk '{sum += $2; count++} END {print "Average:", sum/count}' data.txt
 
# Count unique occurrences of IP addresses (like sort | uniq -c)
awk '{count[$1]++} END {for (ip in count) print count[ip], ip}' access.log | sort -nr | head -n 10