awk · wleeaf field noteshttps://articles.wleeaf.dev/tags/awk/Hands-on guides and build logs from the wleeaf studio: self-hosting, Linux, networking, Docker, the command line and reverse engineering.en-usSat, 04 Apr 2026 00:00:00 +0000awk in 10 Minutes With Practical Exampleshttps://articles.wleeaf.dev/posts/awk-practical-examples/Sat, 04 Apr 2026 00:00:00 +0000https://articles.wleeaf.dev/posts/awk-practical-examples/Nobody reads the awk manual. Here are the patterns that cover 90% of real-world use.awkclilinuxawk is one of those tools that looks intimidating until you learn five patterns. Then you use it every day. I am not going to explain the full language. Here are the patterns that actually matter.

Pattern 1: Print a Column#

awk splits each line into fields. $1 is the first, $2 is the second, $0 is the whole line:

bash
# Print the first column of ps output
ps aux | awk '{print $1}'

# Print columns 1 and 11 (user and command)
ps aux | awk '{print $1, $11}'

The default field separator is whitespace. Change it with -F:

bash
# Parse /etc/passwd (colon-separated)
awk -F: '{print $1, $7}' /etc/passwd

# Parse CSV
awk -F, '{print $2}' data.csv

Pattern 2: Filter Rows#

Put a condition before the action:

bash
# Lines where column 3 is greater than 100
awk '$3 > 100' data.txt

# Lines containing "ERROR"
awk '/ERROR/' app.log

# Lines where the first field matches a pattern
awk '$1 ~ /^192\.168/' access.log

Combine conditions:

bash
# Memory usage over 50% AND the process is not root
ps aux | awk '$4 > 50.0 && $1 != "root" {print $1, $4, $11}'

Pattern 3: Sum a Column#

bash
# Total bytes transferred (column 10 in access logs)
awk '{sum += $10} END {print sum}' access.log

# Average response time
awk '{sum += $NF; n++} END {print sum/n}' response_times.log

END runs after all lines are processed. NF is the number of fields on the current line, so $NF is the last field.

Pattern 4: Count and Group#

bash
# Count HTTP status codes
awk '{count[$9]++} END {for (c in count) print c, count[c]}' access.log | sort -rn -k2

# Output:
# 200 45231
# 304 12044
# 404 891
# 500 23

This uses an associative array. count[$9]++ increments the count for each unique value in column 9.

Top 10 IP addresses by request count:

bash
awk '{count[$1]++} END {for (ip in count) print count[ip], ip}' access.log | sort -rn | head -10

Pattern 5: Transform Fields#

bash
# Convert bytes to megabytes
awk '{printf "%s %.1fMB\n", $1, $2/1048576}' sizes.txt

# Add a prefix to every line
awk '{print "[INFO]", $0}' app.log

# Swap two columns
awk '{temp=$1; $1=$2; $2=temp; print}' data.txt

Real-World Examples#

Parse Docker ps output#

bash
docker ps --format '{{.Names}}\t{{.Status}}\t{{.Ports}}' | \
  awk -F'\t' '{printf "%-20s %-15s %s\n", $1, $2, $3}'

Find the largest Docker images#

bash
docker images --format '{{.Repository}}:{{.Tag}}\t{{.Size}}' | \
  awk -F'\t' '{
    size = $2
    if (size ~ /GB/) {
      gsub(/GB/, "", size)
      mb = size * 1024
    } else if (size ~ /MB/) {
      gsub(/MB/, "", size)
      mb = size
    } else {
      mb = 0
    }
    printf "%8.0fMB  %s\n", mb, $1
  }' | sort -rn | head -10

Calculate uptime percentage#

Given a log with timestamps and status:

bash
awk '
  /UP/   { up++ }
  /DOWN/ { down++ }
  END {
    total = up + down
    if (total > 0)
      printf "Uptime: %.2f%% (%d/%d checks)\n", (up/total)*100, up, total
  }
' monitoring.log

Extract unique error messages#

bash
awk '/ERROR/ {
  msg = ""
  for (i=4; i<=NF; i++) msg = msg " " $i
  errors[msg]++
}
END {
  for (e in errors) printf "%5d %s\n", errors[e], e
}' app.log | sort -rn | head -20

Built-in Variables#

Variable Meaning
NR Current line number
NF Number of fields on current line
FS Field separator
OFS Output field separator
$0 Entire line
$1..$n Individual fields
FILENAME Current filename

When to Use awk vs. Other Tools#

  • cut: When you just need a column and nothing else. cut -d: -f1 /etc/passwd is simpler than awk for trivial extraction.
  • grep: When you just need to filter lines by a pattern.
  • sed: When you need to do find-and-replace on text.
  • awk: When you need to do math, grouping, conditional logic, or work with multiple fields at once.

The mental model is simple. awk reads lines, splits them into fields, and lets you write rules about what to do with each line. Once that clicks, you start seeing awk solutions everywhere.

]]>