Techniques for Writing Bash Scripts

2022-10-16

Being able to efficiently write Shell scripts should be a fundamental skill for any SRE or Infra engineer. Recently, I've learned a lot from pitfalls while writing Shell scripts, so here's a summary.

Tracing Script Execution Stack

For simple scripts, you can print information in the console by using the echo command. But for complex scripts, echo output isn't clear enough. The bash command provides a -x flag that shows the specific execution stack and arguments while running the script, helping us to debug more effectively.

For example, we have a script:

➜ ✗ cat demo.sh 
function g() {
    return 0
}

function f() {
    echo using arg: $1
    g $1
}

f hello

Run it with the -x flag:

➜ bash -x demo.sh
+ f hello
+ echo using arg: hello
using arg: hello
+ g hello
+ return 0

Error Handling

Shell scripts don't have a mechanism like Java's Exception to capture exception stacks. You can only determine whether a function executed successfully by checking its return code.

➜  cat demo.sh 
function f() {
    return 0
}

function g() {
    return 1
}

function execute() {
    $1
    if [[ $? -eq 0 ]]; then
        echo execute function $1 success
    else
        echo fail to execute function $1
    fi
}

execute f
execute g

Let's run it:

➜  bash -x demo.sh
+ execute f
+ f
+ return 0
+ [[ 0 -eq 0 ]]
+ echo execute function f success
execute function f success
+ execute g
+ g
+ return 1
+ [[ 1 -eq 0 ]]
+ echo fail to execute function g
fail to execute function g

Returning Text Content from Functions with echo

Shell functions must return numeric values in the range of 0-255. Normally they return 0 for success, and different numbers for different error states.

But what if we need to return text? See the example:

➜  cat demo.sh 
function f() {
    echo using arg: $1
}

output=$(f hello)

echo output: [$output]

Run it:

➜  bash -x demo.sh
++ f hello
++ echo using arg: hello
+ output='using arg: hello'
+ echo output: '[using' arg: 'hello]'
output: [using arg: hello]

Using `local` to Define Variables Inside Functions

This practice avoids changes to global variables caused by function execution, which can sometimes lead to hard-to-track bugs.

➜  cat demo.sh 
function f() {
    a_global_var="changed by f"
} 

a_global_var="init"
f
echo $a_global_var

➜  bash -x demo.sh
+ a_global_var=init
+ f
+ a_global_var='changed by f'
+ echo changed by f
changed by f

If we use local:

➜  cat demo.sh
function f() {
    local a_global_var="changed by f"
    echo inside the f function, the var is [$a_global_var]
} 

a_global_var="init"
f
echo $a_global_var

➜  bash -x demo.sh
+ a_global_var=init
+ f
+ local 'a_global_var=changed by f'
+ echo inside the f function, the var is '[changed' by 'f]'
inside the f function, the var is [changed by f]
+ echo init
init

Redirecting stderr to stdout

Many commands produce logs during execution, but these don't go to stdout — they go to stderr.

If the script's environment doesn't properly print stderr logs, you'll need to redirect them to stdout.

For example, we have a demo.sh script that calls a help.sh script. Part of help.sh's output goes to stderr.

➜  cat help.sh 
echo generated error >&2
echo hello
➜  cat demo.sh 
output=$(bash help.sh)
echo output: [$output]

Run demo.sh:

➜  bash -x demo.sh
++ bash help.sh
generated error
+ output=hello
+ echo output: '[hello]'
output: [hello]

Now let's modify demo.sh to redirect stderr to stdout:

➜  cat demo.sh
output=$(bash help.sh 2>&1)
echo output: [$output]

Run demo.sh again:

➜  bash -x demo.sh
++ bash help.sh
+ output='generated error
hello'
+ echo output: '[generated' error 'hello]'
output: [generated error hello]

Note the Difference Between ' and "

When the content inside quotes is a constant string, ' and " make no significant difference. But if the string needs variable interpolation, ' will not work.

For example:

➜  cat demo.sh
a="world"
b="hello,$a"
c='hello,$a'

echo $b
echo $c