Let’s face it, when writing code in the command line, you’ll occasionally encounter a problem. The problems I usually find are actually single instances of problems repeated elsewhere. For instance, I find a problem, I fix it. Then I determine if the same problem exists elsewhere, and fix it there.

Find and grep are useful tools, and their uses are almost limitless, so I will provide only a few examples that highlight some neat and powerful options. Here are some of the things you’ll learn:

  • Finding files and interacting with them (delete, compress, etc)
  • Using grep to search file contents
  • Using grep to find a file missing a string
  • Using sed and grep to insert a string into certain files

This article will start with a relatively simple example and build towards a more advanced one. Keep reading to learn more about find, sed and grep!

Find and Interact with Files

Let’s start with an example of using the find command to locate and remove some backup files that may be in the /etc directory. We’ll specify the location to look (the current directory: “.”), that we’re only interested in files (type -f) and specify that the name of the file must end with “bu” (-name “*bu”).

If the results look good, you can easily pass the -delete option to get rid of them:

hacker mentality

Pro Tip: Don’t delete without fully considering the consequences. Ask yourself what you’re trying to accomplish and always keep a backup of your files.

Need a good way to do this? Create a script and backup your files to SmartFile on a routine basis. Click here to try SmartFile for free, no credit card required.

However, you will not always want to delete files once you’ve found them, so it is important to know how to execute an arbitrary command. Using -exec and rm, we can perform the equivalent of -delete from the example above. For every file found, the rm command will be run, and {} will be expanded to the found file’s name:

Now you can do all kinds of things to the files you find! An important note, if you are searching a remote location (say you’re in /root, but searching another user’s directory) you will need to use -execdir instead of -exec, which will cause {} to be expanded to the full path of the files.

Searching and Interacting with File Contents Using Grep and Sed

Rather than executing commands that interact with the complete file (delete, compress, etc.), we may often wish to interact with the contents of files. sed is a powerful tool we can leverage for this purpose. In short, sed is a text processor that can make changes to a file based on regular expressions. There are entire books dedicated to its use, but I will point out two use cases which demonstrate adding a line to files and searching and replacing a string.

Let’s say I have a coworker with a bad habit of not putting a shebang (#!/bin/sh) at the beginning of their scripts. I can easily use sed to add a new line containing the shebang to the beginning of a file:

Now, you may want to do this for lots of files, if you have a history of making this mistake. You could use find as shown above to find any file with the extension “.sh” and -exec to do the insertion, but you likely don’t need to do it for every script as some may already have a shebang, and it would be pointless to have two. So what can we do?

First, let’s look at identifying how we’re using shebangs in our existing files:

By default, grep outputs the name of the file, followed by the line that contains the text match. Note that I needed to use \ to escape the ! in the search string.

Also, I grep’ed all the files in the current folder ending in “.sh” only. This is great, I can see that two files have shebangs, but I know there are more files that need them! Grep’s “-L” option will return file names that are MISSING the string we’re looking for:

At this point, we have a command we want to run (see example above) against a list of files (the grep -L results). We can use a pipe (|) and xargs to perform the sed on every file returned by grep -L.

Phew! It probably would have been easier to just put the shebang in correctly the first time!

Concluding Thoughts

Overall, find, sed and grep are great tools to use for search and modify files and their contents. These are just some instances of their use cases, and there are tons more worth exploring as well. Let us know some of your favorites in the comments below.

Protect Your Files Before Making Changes

While these tools can make it easier to take action on files, you always need multiple backups. Leverage SmartFile to backup these files off your file system manually or automatically with scripts. Try it today for free, no credit card required!

Sign Up Now

Related Posts

Related Topics & Tags: Quick Tips

About Alex Kubacki

Alex is the Linux Administrator at SmartFile. And yes, that is his real life photo, he's a cartoon that lives in the real world, like Roger Rabbit.

Leave a Reply

Your email address will not be published. Required fields are marked *