What grep Is and How to Use It

The other day, Habr user simpleadmin posted a pretty useful note about grep. From here on, it’s the author’s words:

So… Summer… Friday… Let’s talk a bit about grep.

Knowing the local crowd, and to avoid any unnecessary insinuations, I should mention that everything below applies to

# grep -version | grep grep  
grep (GNU grep) 2.5.1-FreeBSD

This matters because of

# man grep | grep -iB 2 freebsd  
  -P, -perl-regexp Interpret PATTERN as a Perl regular expression. This option is not supported in FreeBSD.

First, about how we usually grep files.
Using cat:

root@nm3:/ # cat /var/run/dmesg.boot | grep CPU:  
CPU: Intel(R) Core(TM)2 Quad CPU Q9550 @ 2.83GHz (2833.07-MHz K8-class CPU)

But why? You can just do this instead:

root@nm3:/ # grep CPU: /var/run/dmesg.boot  
CPU: Intel(R) Core(TM)2 Quad CPU Q9550 @ 2.83GHz (2833.07-MHz K8-class CPU)

Or like this (I hate this construct):

root@nm3:/ # </var/run/dmesg.boot grep CPU: CPU: Intel(R) Core(TM)2 Quad CPU Q9550 @ 2.83GHz (2833.07-MHz K8-class CPU)

For some reason we count the matched lines with wc:

root@nm3:/ # grep WARNING /var/run/dmesg.boot | wc -l  
3

Though you can just:

root@nm3:/ # grep WARNING /var/run/dmesg.boot -c  
3

Let’s make a test file:

one two three
seven eight one eight three
thirteen fourteen fifteen
sixteen seventeen eighteen seven
sixteen seventeen eighteen
twenty seven
one 504 one
one 503 one
one 504 one
one 504 one
#comment UP
twentyseven
#comment down
twenty1
twenty3
twenty5
twenty7

And let’s start searching:
The -w option lets you search for a whole word:

root@nm3:/ # grep -w 'seven' test.txt
seven eight one eight three
sixteen seventeen eighteen seven
twenty seven

What if you need the start or end of a word?

root@nm3:/ # grep 'seven\>' test.txt
seven eight one eight three
sixteen seventeen eighteen seven
twenty seven
twentyseven

At the start or end of a line?

root@nm3:/ # grep '^seven' test.txt
seven eight one eight three
root@nm3:/ # grep 'seven$' test.txt
sixteen seventeen eighteen seven
twenty seven
twentyseven

Want to see the lines around a match?

root@nm3:/ # grep -C 1 twentyseven test.txt
#comment UP
twentyseven
#comment down

Only below or above?

root@nm3:/ # grep -A 1 twentyseven test.txt
twentyseven
#comment down

root@nm3:/ # grep -B 1 twentyseven test.txt
#comment UP
twentyseven

We can also do this

root@nm3:/ # grep `twenty[1-4]` test.txt
twenty1
twenty3

And the opposite, excluding these

root@nm3:/ # grep `twenty[^1-4]` test.txt
twenty seven
twentyseven
twenty5
twenty7

Of course grep also supports the usual quantifiers, metacharacters and other regex goodies. A couple of practical examples:

root@nm3:/ # cat /etc/resolv.conf
#options edns0
#nameserver 127.0.0.1
nameserver 8.8.8.8
nameserver 77.88.8.8
nameserver 8.8.4.4

Let’s pick out only the lines with an IP:

root@nm3:/ # grep -E `[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}` /etc/resolv.conf
#nameserver 127.0.0.1
nameserver 8.8.8.8
nameserver 77.88.8.8
nameserver 8.8.4.4

Works, but this looks nicer:

root@nm3:/ # grep -E '\b[0-9]{1,3}(\.[0-9]{1,3}){3}\b' /etc/resolv.conf
#nameserver 127.0.0.1
nameserver 8.8.8.8
nameserver 77.88.8.8
nameserver 8.8.4.4

Let’s drop the commented-out line?

root@nm3:/ # grep -E '\b[0-9]{1,3}(\.[0-9]{1,3}){3}\b' /etc/resolv.conf | grep -v #
nameserver 8.8.8.8
nameserver 77.88.8.8
nameserver 8.8.4.4

Now let’s pull out just the IPs themselves

root@nm3:/ # grep -oE '\b[0-9]{1,3}(\.[0-9]{1,3}){3}\b' /etc/resolv.conf | grep -v #
127.0.0.1
8.8.8.8
77.88.8.8
8.8.4.4

Here’s the catch… The commented-out line is back. That’s due to how pattern matching works. What do we do? Like this:

root@nm3:/ # grep -v # /etc/resolv.conf | grep -oE '\b[0-9]{1,3}(\.[0-9]{1,3}){3}\b'
8.8.8.8
77.88.8.8
8.8.4.4

Let’s pause here on inverting the search with the -v flag. Say we need to run “ps -afx | grep ttyv”

root@nm3:/ # ps -afx | grep ttyv
1269 v1 Is+ 0:00.00 /usr/libexec/getty Pc ttyv1
1270 v2 Is+ 0:00.00 /usr/libexec/getty Pc ttyv2
1271 v3 Is+ 0:00.00 /usr/libexec/getty Pc ttyv3
1272 v4 Is+ 0:00.00 /usr/libexec/getty Pc ttyv4
1273 v5 Is+ 0:00.00 /usr/libexec/getty Pc ttyv5
1274 v6 Is+ 0:00.00 /usr/libexec/getty Pc ttyv6
1275 v7 Is+ 0:00.00 /usr/libexec/getty Pc ttyv7
48798 2 S+ 0:00.00 grep ttyv

That would be fine, except we don’t want the line “48798 2 S+ 0:00.00 grep ttyv”. Let’s use -v

root@nm3:/ # ps -afx | grep ttyv | grep -v grep
1269 v1 Is+ 0:00.00 /usr/libexec/getty Pc ttyv1
1270 v2 Is+ 0:00.00 /usr/libexec/getty Pc ttyv2
1271 v3 Is+ 0:00.00 /usr/libexec/getty Pc ttyv3
1272 v4 Is+ 0:00.00 /usr/libexec/getty Pc ttyv4
1273 v5 Is+ 0:00.00 /usr/libexec/getty Pc ttyv5
1274 v6 Is+ 0:00.00 /usr/libexec/getty Pc ttyv6
1275 v7 Is+ 0:00.00 /usr/libexec/getty Pc ttyv7

Ugly construct? Let’s do a little trick:

root@nm3:/ # ps -afx | grep `[t]tyv`
1269 v1 Is+ 0:00.00 /usr/libexec/getty Pc ttyv1
1270 v2 Is+ 0:00.00 /usr/libexec/getty Pc ttyv2
1271 v3 Is+ 0:00.00 /usr/libexec/getty Pc ttyv3
1272 v4 Is+ 0:00.00 /usr/libexec/getty Pc ttyv4
1273 v5 Is+ 0:00.00 /usr/libexec/getty Pc ttyv5
1274 v6 Is+ 0:00.00 /usr/libexec/getty Pc ttyv6
1275 v7 Is+ 0:00.00 /usr/libexec/getty Pc ttyv7

Also don’t forget about | (OR)

root@nm3:/ # vmstat -z | grep -E `(sock|ITEM)`
ITEM SIZE LIMIT USED FREE REQ FAIL SLEEP
socket: 696, 130295, 30, 65, 43764, 0, 0

and the same thing, another way:

root@nm3:/ # vmstat -z | grep `sock\|ITEM`
ITEM SIZE LIMIT USED FREE REQ FAIL SLEEP
socket: 696, 130295, 30, 65, 43825, 0, 0

Now, while a lot of people remember how to use regexes in grep, they somehow forget about POSIX classes, and those come in handy sometimes too. POSIX Let’s pick out lines with uppercase characters:

root@nm3:/ # grep `[[:upper:]]` test.txt
#comment UP

And a couple more tricks to wrap up. The first one is more academic. In 15 years I’ve never actually used it: We need to pick lines from our test file that contain six, seven or eight: So far it’s all simple:

root@nm3:/ # grep -E `(six|seven|eight)` test.txt
seven eight one eight three
sixteen seventeen eighteen seven
sixteen seventeen eighteen
twenty seven
twentyseven

Now only the lines where six, seven or eight show up more than once. This trick is called Backreferences

root@nm3:/ # grep -E `(six|seven|eight).*\1` test.txt
seven eight one eight three
sixteen seventeen eighteen seven

And the second trick, a much more useful one. We need to print lines where 504 is bounded by tabs on both sides. Oh, how I miss PCRE support here… Using POSIX classes doesn’t save us:

root@nm3:/ # grep `[[:blank:]]504[[:blank:]]` test.txt
one 504 one
one 504 one
one 504 one

The [CTRL+V][TAB] trick comes to the rescue:

root@nm3:/ # grep ` 504 ` test.txt
one 504 one

What else haven’t I mentioned? Of course, grep can search files/directories, and of course recursively. Let’s find the code in the sources where Intel allows third-party SFPs. I don’t remember if it’s allow_unsupported_sfp or unsupported_allow_sfp. Well, never mind — that’s grep’s problem to sort out:

root@nm3:/ # grep -rni allow /usr/src/sys/dev/ | grep unsupp
/usr/src/sys/dev/ixgbe/README:75:of unsupported modules by setting the static variable 'allow_unsupported_sfp'
/usr/src/sys/dev/ixgbe/ixgbe.c:322:static int allow_unsupported_sfp = TRUE;
/usr/src/sys/dev/ixgbe/ixgbe.c:323:TUNABLE_INT(`hw.ixgbe.unsupported_sfp`, &allow_unsupported_sfp);
/usr/src/sys/dev/ixgbe/ixgbe.c:542: hw->allow_unsupported_sfp = allow_unsupported_sfp;
/usr/src/sys/dev/ixgbe/ixgbe_type.h:3249: bool allow_unsupported_sfp;
/usr/src/sys/dev/ixgbe/ixgbe_phy.c:1228: if (hw->allow_unsupported_sfp == TRUE) {

Hope I didn’t wear you out. And this was only the tip of the grep iceberg. Enjoy the read, and I’ll enjoy my appetite at the barbecue! And good luck with your grep-ing!

original article