Let’s face it, when writing code in the command line, you’ll occasionally encounter a problem. The problems I usually find are actually single instances of problems repeated elsewhere. For instance, I find a problem, I fix it. Then I determine if the same problem exists elsewhere, and fix it there.
Find and `grep` are useful tools, and their uses are almost limitless, so I will provide only a few examples that highlight some neat and powerful options. Here are some of the things you’ll learn:
- Finding files and interacting with them (delete, compress, etc)
- Using grep to search file contents
- Using grep to find a file missing a string
- Using sed and grep to insert a string into certain files
This article will start with a relatively simple example and build towards a more advanced one. Keep reading to learn more about find, sed and grep!
Find and Interact with Files
Let’s start with an example of using the `find` command to locate and remove some backup files that may be in the /etc directory. We’ll specify the location to look (the current directory: “.”), that we’re only interested in files (type -f) and specify that the name of the file must end with “bu” (-name “*bu”).
[akubacki@thelaptop ~]$ find . -type f -name "*bu"
If the results look good, you can easily pass the `-delete` option to get rid of them:
[akubacki@thelaptop ~]$ find . -type f -name "*bu" -delete
Pro Tip: Don’t delete without fully considering the consequences. Ask yourself what you’re trying to accomplish and always keep a backup of your files.
Need a good way to do this? Create a script and backup your files to SmartFile on a routine basis. Click here to try SmartFile for free, no credit card required.
However, you will not always want to delete files once you’ve found them, so it is important to know how to execute an arbitrary command. Using `-exec` and `rm`, we can perform the equivalent of `-delete` from the example above. For every file found, the `rm` command will be run, and `{}` will be expanded to the found file’s name:
[akubacki@thelaptop ~]$ find . -type f -name "*backup" -exec rm -f {} \;Now you can do all kinds of things to the files you find! An important note, if you are searching a remote location (say you’re in /root, but searching another user’s directory) you will need to use -execdir instead of -exec, which will cause `{}` to be expanded to the full path of the files.
[akubacki@thelaptop ~]$ find /home/curtis -type f -name "*.log" -execdir gzip {} \;Searching and Interacting with File Contents Using Grep and Sed
Rather than executing commands that interact with the complete file (delete, compress, etc.), we may often wish to interact with the contents of files. `sed` is a powerful tool we can leverage for this purpose. In short, `sed` is a text processor that can make changes to a file based on regular expressions. There are entire books dedicated to its use, but I will point out two use cases which demonstrate adding a line to files and searching and replacing a string.
Let’s say I have a coworker with a bad habit of not putting a shebang (#!/bin/sh) at the beginning of their scripts. I can easily use `sed` to add a new line containing the shebang to the beginning of a file:
[akubacki@thelaptop ~]$ sed -i "1s/^/#\!\/bin\/sh \n/" script.sh
Now, you may want to do this for lots of files, if you have a history of making this mistake. You could use `find` as shown above to find any file with the extension “.sh” and -exec to do the insertion, but you likely don’t need to do it for every script as some may already have a shebang, and it would be pointless to have two. So what can we do?
First, let’s look at identifying how we’re using shebangs in our existing files:
[akubacki@thelaptop ~]$ grep "#\!/bin" *.sh checkssl.sh:#!/bin/sh migrate.sh:#!/bin/sh
By default, `grep` outputs the name of the file, followed by the line that contains the text match. Note that I needed to use \ to escape the ! in the search string.
Also, I grep’ed all the files in the current folder ending in “.sh” only. This is great, I can see that two files have shebangs, but I know there are more files that need them! Grep’s “-L” option will return file names that are MISSING the string we’re looking for:
[akubacki@thelaptop ~]$ grep -L "#\!/bin" *sh dummy.sh badscript.sh
At this point, we have a command we want to run (see example above) against a list of files (the `grep -L` results). We can use a pipe (|) and `xargs` to perform the `sed` on every file returned by `grep -L`.
[akubacki@thelaptop ~]$ grep -L "#\!/bin" *sh | xargs sed -i "1s/^/#\!\/bin\/sh \n/"
Phew! It probably would have been easier to just put the shebang in correctly the first time!
Concluding Thoughts
Overall, `find`, `sed` and `grep` are great tools to use for search and modify files and their contents. These are just some instances of their use cases, and there are tons more worth exploring as well. Let us know some of your favorites in the comments below.