Keyboard shortcuts

Press ← or → to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

Bash Scripting

By the end of this section, you will:

  1. Understand Bash as both a command and scripting language: Recognize the dual functionality of Bash, allowing you to automate tasks and manage scripts efficiently within a Linux environment.
  2. Work with variables and arrays in Bash: Learn to declare and use variables, apply command substitution, and manage arrays for more complex scripting tasks.
  3. Apply conditional expressions for decision-making: Use conditionals such as if; then; else statements to control the flow of your scripts based on conditions and outcomes.
  4. Implement loops to automate repetitive tasks: Utilize looping structures, such as for loops, to automate actions that need to be repeated under certain conditions or across arrays.
  5. Write and execute Bash scripts with the correct structure: Include essential elements like the shebang (#!/usr/bin/env bash) at the start of your scripts, to ensure portability and clarity in execution.
  6. Test conditions in Bash scripts: Understand how to test for specific conditions in scripts, such as file existence or the comparison of variables, to build more reliable and functional scripts.
  7. Validate and improve Bash scripts: Learn how to use tools like shellcheck to check for errors in your Bash scripts and ensure adherence to best practices through style guides.

Getting Started

So far, we’ve been working on the Linux commandline. Specifically, we have been working in the Bash shell. Bash serves as both an interactive command language and a scripting language. The same language we’ve been using at the command line can therefore be used to write programs and automate Linux tasks.

Unix-like operating systems, including Linux, offer many shells, each with different advantages. Besides bash, a few examples include:

  • sh
  • tcsh
  • csh
  • ksh
  • zsh

Other operating systems may include their own shells. Windows OS has PowerShell, which includes the older cmd.exe (Command Prompt) shell as well as the much more capable PowerShell.

Shells serve two purposes. They provide a mechanism for interactively using the operating system from a command prompt, as we’ve been using it so far, and they provide programming capabilities for automation, or job control.

The following is a brief intro to the main parts of scripting in Bash. If you have ever programmed before, many of these concepts will not be new to you.

Shebang or Hashbang

When we start to write scripts, the first thing we add is a shebang or hashbang at line one. The {she,hash}bang tells the shell what program needs to run.

In the example below, the #!/usr/bin/env bash form uses env to locate bash using the current PATH. This can make the script more portable across systems where Bash is installed in different locations.

#!/usr/bin/env bash

As an alternate example, if we were writing a Python script, then we could declare it to be: #!/usr/bin/env python3.

However, you may see the following forms, too:

#!/usr/bin/bash

Or:

#!/bin/bash

On POSIX compliant systems, bash may be located in different paths. On some Linux distributions, it’s located at /usr/bin/bash. On others, it may be located at /bin/bash. On BSD OSes, like FreeBSD, bash might be installed at /usr/local/bin/bash. Thus, by using the #!/usr/bin/env bash shebang, you help ensure that your bash program is portable across different OSes.

Style

To help ensure that your scripts are clean and readable, even for small scripts, it’s helpful to follow a consistent style. Just like when we write academic papers, where we might use APA, Chicago, MLA, or some other style guide, style guides exist for programming and scripting languages, too. Adhering to a style guide helps us to write clean, visually appealing scripts that are easier to maintain and understand. Consider checking out style guides early on, like the Google Shell Style Guide.

Comments

Like other programming languages, we can add comments to our code. These might be notes we leave to remind us or others of the intent of a function or some other aspect of our program. In Bash, we use the pound sign # to indicate a comment. For example, the following function adds two integers, and the line beginning with a pound sign describes that:

#!/usr/bin/env bash

# this function adds two integers
add() {
    local x="$1"
    local y="$2"
    result=$((x + y))
    echo $result
}

# call function
add "$@"

Variables

One of the most important abilities of any programming or scripting language is to declare a variable. Variables enable us to attach some value to a name. Variables are often used to pass information to other parts of a program.

In Bash, we declare variables first with the name of the variable, an equal sign, and then the value of the variable (the assignment) within double quotes. Unlike other programming languages, we do not insert spaces between the variable and assignment. In the following code snippet, which can be entered on the commandline, I create a variable named first_name and assign it the value Sean. I create another variable named backup and assign it the value /media, representing a path on my filesystem. Then I use the echo and cd commands to test the variables:

first_name="Sean"
backup="/media"
echo "My name is ${first_name}"
echo "${backup}"
cd "${backup}"
pwd
cd

Variables may include values that may change given some context. For example, if we want a variable to refer to today’s day of week, we can use command substitution. This “allows the output of a command to replace the command name” (see man bash). In the following, I use the date +%A command to assign the current day of the week to the variable named today. The output at the time this variable is set will differ if it is set on a different day.

# run the command to test the output
date +%A
# assign the command to the variable named `today` using command substitution
today="$(date +%A)"
echo "${today}"

Curly braces are not strictly necessary when calling a Bash variable, but it is good practice to use them for a variety of reasons (e.g., array variables and parameter expansion). See:

We also use curly braces in brace expansion to generate strings:

echo {1..5}
echo {5..1}
echo {a..l}
echo {l..a}

More on arrays below, but we can assign the values in a brace expansion to an array:

my_array=( {1..5} )
echo "${my_array[1]}"

Using brace notation, we can generate multiple sub-directories at once. The following command creates a directory called homework/ that contains two subdirectories: drafts/ and notes/. In the following example, the -p option to mkdir creates the parent directory along with the subdirectories:

mkdir -p $HOME/homework/{drafts,notes}
cd homework
ls
drafts/ notes/

Lists / Arrays

I can assign a list or an array to a variable using () parentheses. To create a variable named seasons that contains multiple values, such as winter spring summer fall, I declare my variable as follows:

seasons=(winter spring summer fall)

Bash lets me access parts of that array. In the following example, [@] refers to the entire array and [n] refers to a subscript in the array. Like other programming languages, the first subscript in an array begins with 0.

echo "${seasons[@]}"
winter spring summer fall
echo "${seasons[0]}"
winter
echo "${seasons[1]}"
spring
echo "${seasons[2]}"
summer

I can also access subscripts in an array from right to left, starting with -1:

echo "${seasons[-1]}"
fall
echo "${seasons[-2]}"
summer

See Parameter expansions for more advanced techniques.

Variable Naming

When naming variables, use lowercase or snake_case: e.g., firstname or first_name is more appropriate than FIRSTNAME or FIRST_NAME. By convention, variables created by your script generally use lowercase or snake_case. Uppercase names are commonly used for environmental variables and sometimes for values treated as constants. Using lowercase for ordinary variables also helps avoid accidentally conflicting with special shell or environmental variables such as PATH, HOME, USER, and PWD.

Environmental variables are pre-set variables that store configuration data for your Bash session. If curious, you can examine these variables by echoing their values: echo $HOME, etc. To see all your environmental variables, run the env command.

Looping

Looping is a common way to repeat an instruction until some specified condition is met. There are several looping methods in Bash that include: for, while, until, and select.

for Loops

The for loop is often the most useful. In the following toy looping example, we instruct bash to assign the letter i to the sequence 1 2 3 4 5. Each time it assigns i to those numbers, it echos them to standard output:

for i in {1..5} ; do
  echo "i = ${i}"
done

Note that I take advantage of brace expansion in the above for loop.

You might notice in the echo statement above that I use ${i} instead of $i. Braces explicitly delimit the variable name. For example, ${i}th expands variable i followed by th, but $ith asks Bash to expand a variable $ith. Braces help to avoid this confusion.

Using the above for loop, we can create a rudimentary timer by calling the sleep command to pause after each count. Once the for loop closes, the final echo statement runs:

for i in {5..1} ; do
  echo "T minus ${i}"
  sleep 1
done ; echo "BLAST OFF!"

Note that I take advantage of brace expansion again, but this time reversing the ordering, as well as conditional execution.

The sleep command is particularly useful in automation tasks where you want to pause execution between steps, such as monitoring scripts, where you might poll a resource at intervals, or in timed alerts. The command sleep 1 pauses the shell for one second, the command sleep 2 pauses the shell for two seconds, and so forth.

while Loops

We can use a while loop instead of a for loop. Before entering a while loop, we initialize a counter with count=5 (the variable count could be named something else). The while statement then tests a condition before each iteration. As long as that condition is true, the loop continues.

count=5

while [[ "${count}" -ge 1 ]] ; do
    echo "${count}"
    ((count--))
    sleep 1
done ; echo "BLAST OFF!"

until Loops

while loops terminate when the condition becomes false. In the above example, the condition (initially count=5) becomes false when $count is no longer greater than or equal to one.

The until loop works in the reverse order, and terminates when the condition becomes true. In this example, the condition becomes true when $count equals zero.

count=5

until [[ "${count}" -eq 0 ]] ; do
    echo "${count}"
    ((count--))
    sleep 1
done ; echo "BLAST OFF!"

Both while and until loops are useful when you need an indeterminate number of loops.

Looping Arrays

We can loop through variable arrays (or lists). In the following for loop, I loop through the seasons variable first introduced above:

#!/usr/bin/env bash

seasons=(winter spring summer fall)

for i in "${seasons[@]}" ; do
  echo "I hope you have a nice ${i}."
done

Note that I added the {she,hash}bang in the above example. I do this to make it clear that this is the kind of for loop that I would want to write in a text editor.

Testing

Sometimes we will want to test certain conditions. There are two parts to this, we can use if; then ; else commands, and we can also use the double square brackets: [[. There are a few ways to get documentation on these functions. See the following:

man test
help test
help [
help [[
help if

Between [ and [[, I generally prefer to use the [[ syntax, as demonstrated below. It’s less error-prone and allows for more complex conditional checks. However, [[ is a bash specific conditional construct and is not standard POSIX sh syntax, like [ is. Since we are writing Bash scripts, we generally prefer it for its versatility.

We can test integers:

if [[ 5 -ge 3 ]] ; then
  echo "true"
else
  echo "false"
fi

Reverse it to return the else statement:

if [[ 3 -ge 5 ]] ; then
  echo "true"
else
  echo "false"
fi

We can test file conditions. Let’s first create a file called paper.txt and a file called paper.bak. We will add some trivial content to paper.txt but not to the paper.bak. The following if statement will test if paper.txt has a more recent modification date. If so, it’ll back up the file with the cp and echo back its success.

First, create the two files and add a bit of text to the text file:

touch "$HOME/paper.bak"
sleep 1

# creating the paper.txt file after the paper.bak file, for this example, means paper.txt is newer than paper.bak
echo "My paper" > "$HOME/paper.txt"

And then run the test and do the backup if it passes:

if [[ "$HOME/paper.txt" -nt "$HOME/paper.bak" ]] ; then
  cp "$HOME/paper.txt" "$HOME/paper.bak" && echo "Paper is backed up."
fi

Here’s a script that tests whether today is either Tue or Thu. If it is either of those days, it prints that class is at 9:30am. If it’s neither Tue or Thu, then it prints that there is no class today. Let’s save it in a text file and call it schedule.sh:

#!/usr/bin/env bash

day1="Tue"
day2="Thu"
today="$(date +%a)"

if [[ "${today}" = "${day1}" ]] ; then
  printf "\nIf %s is %s, then ICT 418: Linux Systems Administration is at 9:30am.\n" "${today}" "${day1}"
elif [[ "${today}" = "${day2}" ]] ; then
  printf "\nIf %s is %s, then ICT 418: Linux Systems Administration is at 9:30am.\n" "${today}" "${day2}"
else
  printf "\nThere is no class today.\n"
fi

printf is an alternate way of displaying text on the screen to the echo command.

Nested Conditions: Combining Loops and Tests

We can test strings and nest conditions within loops. Consider our array example above. In the following, if the value of the array equals fall or spring, then the script will wish you an appropriate greeting:

#!/usr/bin/env bash

seasons=(winter spring summer fall)

for i in "${seasons[@]}" ; do
    if [[ "${i}" == "spring" || "${i}" == "fall" ]] ; then
        echo "I hope you have a nice ${i} semester."
    fi
done

Functions

Functions are blocks of code. We use functions to organize or re-use code in other programs or scripts. In bash, we define a function using the following syntax:

function_name() {
    command 1
    command 2
}

In the above greetings script, I can place the code within a function that I might call student_greeting, and add another function I will call break_greeting. Since I am re-using the array variable in both functions, I can move it outside and before the functions. This makes it a global variable, since it is available to all the functions in the script. We still have the $i and $j variables within the functions. I can declare those to be local variables within their functions to make it explicit. Then we call the functions at the end of the script:

#!/usr/bin/env bash
 
SEASONS=(winter spring summer fall)

student_greeting() {
	local i
	for i in "${SEASONS[@]}" ; do
		if [[ "${i}" == "spring" || "${i}" == "fall" ]] ; then
			echo "I hope you have a nice ${i} semester."
		fi
	done
}

break_greeting() {
	local j
	for j in "${SEASONS[@]}" ; do
		if [[ "${j}" == "winter" || "${j}" == "summer" ]] ; then
			echo "I hope you have a nice ${j} break."
		fi
	done
}

student_greeting
break_greeting

Global Versus Local Variables

By default, variables in Bash are global. Global variables are variables accessible by all functions in a script; and local variables are variables accessible only within a function. Since SEASONS, in the example above, is a script-wide value that we are treating as fixed, I use an uppercase name (although it will work in lower case, too). The loop variables i and j, on the other hand, are explicitly declared local inside each function. This is not necessary, but it makes for better and safer code. If you are working within functions and want a variable to only be available within the function, you can declare it as local using local var_name=value or local var_name.

Checking Scripts with ShellCheck

Finally, you can check your shell scripts using the shellcheck command. To use it, let’s say I have saved the above script in a file named greetings in my ~/bin directory.

To check for errors, run the following command:

shellcheck ~/bin/greetings

If there are errors, shellcheck will tell you what they are and provide a link to documentation on the error.

Execute

To make your script available for use at the command line, put the script file in your PATH, such as in $HOME/bin, and set it’s permissions to x:

chmod 766 ~/bin/greetings

case and select

This is only an introduction to bash scripting, and there is far more to learn. For example, bash also includes case and select statements. Both of these add interactive elements to our scripts and can make creating advanced programs fun. Read more about them at:

Conclusion

In this lecture, we’ve covered the basics of Bash scripting, including working with variables, loops, and conditionals. These tools form the foundation for automating tasks and creating powerful scripts. Continue practicing by writing small scripts for your own workflow, and explore the resources and style guides provided to deepen your understanding.

If you become interested in bash scripting, then you should check out the various style guides that exist. For example, see the Shell Style Guide that was authored by coders at Google and the additional resources listed below.

Resources

I encourage you to explore some useful guides and cheat sheets on Bash scripting: