cli-walkthrough/05-scripting

CLI Part V: Scripting

CHEG 667-013 — Chemical Engineering with Computers
Department of Chemical and Biomolecular Engineering, University of Delaware

---

Key idea

Use wildcards, shell history, and shell scripts to work efficiently and automate tasks.

Key goals

• Select multiple files with wildcards (*, ?)
• Make directory backups with cp -r
• Use shell history to recall and re-execute commands
• Write and run a shell script

---

1. Wildcards and matching

In earlier sections, we used mv, cp, and rm to manipulate files one at a time. We can select more than one file to act on by using wildcards:

• * — match a string of characters
• ? — match one character

For example, compare the output of these commands:

$ ls /dev/tty*

$ ls /dev/tty?

What files are listed in the first example? What is the difference in the second?

Exercise 1: Practice listing certain files. Can you list all of the /dev/tty files that begin with tty1? How about all of the programs in /usr/bin that begin with the letter p? Try some other letters!

Exercise 2: Count the number of files that begin with the letter p in /usr/bin by typing ls /usr/bin/p* | wc -l. What is the program wc? (RTFM!)

Delete all files, recursively

Clear out a directory structure for deletion using the command:

$ rm -r *

Be careful! Remember, there is no undo!

2. Making a backup of a directory

What if I have a directory ~/foobar with important files? I want to make a copy of that directory. Can I use the following command?

$ cp foobar foobar_backup

Why or why not? Try it!

The copy command will not act on a directory. However, we can copy all of the directory contents to a new directory using the recursion option, cp -r:

$ cp -r foobar foobar_backup

Now we should have a backup of foobar with all of the files (and directories). Here's the original directory:

$ ls -l foobar
total 20
-rwxrwxr-x 1 furst furst 15960 Mar 31 13:52 a.out*
-rw-rw-r-- 1 furst furst    61 Mar 31 13:52 hello.c

and here is the backup:

$ ls -l foobar_backup/
total 20
-rwxrwxr-x 1 furst furst 15960 Mar 31 18:07 a.out*
-rw-rw-r-- 1 furst furst    61 Mar 31 18:07 hello.c

Something interesting happened: when we copy the files, they have new modification times. This might be undesirable for a backup. But I can preserve the old modification times with the -p option:

$ cp -rp foobar foobar_backup

Now the copies of those files should preserve their original modification times. This holds for a number of other file transfer and copying commands, like rsync and sftp.

3. Shell history

Typing in a command again and again can be a drag. Luckily, most shells save a history of previous commands. You can refer back to this history and even execute previous commands.

• history — print the shell command history
• ! and !! — execute a previous command or the most recent command

Type history (or maybe history | less):

$ history
...
 2013  ls
 2014  less weather.sh
 2015  man whoami
 2016  whoami
 2017  man history
 2018  history

Now, if I type !less ("bang less"):

$ !less

it will execute the last less instruction in my command history (in this case, less weather.sh). This is useful when commands are long and have a lot of options or if you need to repeatedly refer back to a text file, like the example here.

Typing !! ("bang-bang") will execute the last command in the history. This is also handy!

4. Shell programming

The shell is the program that is managing our input and output in the terminal. There are several shell programs to choose from including the Bourne shell sh, C shell csh, Korn shell ksh, Z shell zsh, but bash is a common default.

We can also write scripts in the shell. These may be used to run other programs or a combination of tasks. Programming the shell is a subject in itself, but a powerful tool. Here's one (perhaps useful) example to play with.

Type in the following and save it as weather.sh (or use the copy in this directory):

#!/bin/bash
# weather.sh [-s STATE] [ZONE]

usage() { echo "Usage: $0 [-s STATE] [ZONE]" 1>&2; exit 1; }

STATE="de"
ZONE="001"

while getopts "s:" flag; do
  case "$flag" in
    s) STATE="$OPTARG" ;;
    *) usage ;;
  esac
done
shift $((OPTIND - 1))

# Allow 0 or 1 positional arg (ZONE)
if [ "$#" -gt 1 ]; then
  usage
fi

if [ "$#" -eq 1 ]; then
  if [[ ! "$1" =~ ^[0-9]{3}$ ]]; then
    usage
  fi
  ZONE="$1"
fi

STATE="${STATE,,}"  # lowercase

curl -s "https://tgftp.nws.noaa.gov/data/forecasts/zone/${STATE}/${STATE}z${ZONE}.txt"

Now change the permissions to make this script executable:

$ chmod u+x weather.sh

You now have a short script that downloads the latest weather forecast! Who needs the web!

macOS Easter egg: ./weather.sh | tail -n +15 | say

The shell script above shows how you can write your own Unix utility. It processes command options like the programs we've learned about and provides some user information, including fairly reasonable error processing.

Exercise 3: Try a few different states and zones. (You'll have to look them up.) Redirect each report to a unique file. Can you concatenate these into one large file?

Exercise 4: Write your own shell script. It can be simple — maybe one that creates a backup of a directory, or one that greets the user with the current date and time.