Bash: Things That Make It Worthy

Bash is a shell that is the command interpreter for many Unix variants. Hence, Linux[], *BSD, and Mac OS all have Bash as their default login command interpreter. You can also run Bash on Windows with the help of the Windows subsystem of Linux.

The Thompson shell is the first original Unix shell. The first release was on 3rd November 1971, called Unix Version 1. It had the following tools:

cal
cat
chdir (later shortened to cd)
chmod/chown
cp
date
df
du

ed: this is the ancestor of vi/vim, with the ancestry being: ed(editor)
- em(editor for mortals)
- en
- ex (extended en)
- vi(visual)
- vim(vi improved)

ls
mkdir
su

The Thompson shell was minimally functional and insufficient for programming tasks, such as ‘if/else’. Bourne shell replaced the Thompson shell in Unix version 7 (1979). The shell looked more like a programming language, having features like flow control variables (if/else, case), i/o redirection (>&2, 2>my-errors.log), etc.

As copyright/licensing issues plagued the operating system, Richard Stallman (founder of the GNU project), started writing duplicates of the Unix kernel and tools without any Unix source code. These tools included the C compiler (gcc), the C standard library (glibc), the core binary executables such as ‘sed’, bundled in a package called ‘coreutills’, and Bash.

Overview

The upcoming paragraphs are for those who understand Bash. This is the ~/.profile, a place to assign Bash settings.

bind 'set show-all-if-ambiguous on'
bind 'TAB:menu-complete'
bind 'set completion-ignore-case on'
bind 'set visible-stats on'
bind 'set page-completions off'

Using TAB to autocomplete filenames is easier in this shell. Press TAB only once to autocomplete. Above all, there is no bell sound and it will show a list of files that match the autocomplete. Hence one can go through until they find the match.

Look at this example. It shows the complete list of files after typing “bold[TAB]”.

@mbp2:src $ bold
athena*      athena.user* email*
@mbp2:src $ bold athena

Exit codes in Bash

A Bash script will not exit if an error from a program has been thrown, which is different from a syntax error. This does not exist in other programming languages and may result in unexpected behavior in Bash scripts. A sensible code that can be used here is:

#!/usr/bin/env bash
set -euo pipefail

Now the script will fail if there is any error or unset variable.

Here are some more details on these settings: set -e (from man bash): Exit immediately if a pipeline, which can consist of a single shell command, exists with a non-zero status. set -u (from man bash): The return value is the value of the last command to exit with a non-zero status, or zero if all commands exit successfully. set -o pipefail (from man bash): The return value is the value of the last command to exit with a non-zero status, or zero if all commands exit successfully.

Just like any other programming language, Bash also features exit codes. The $? variables consist of the exit command of the prior function. Exit codes 1, -2, 126, 165 and 255 all have special meanings. Only if the command warrants the exit code, you should call the aforementioned exit codes. However, you can use any exit code as given below:

@mbp2:~ $ /usr/local/bin/bash -c 'exit 0'; echo $?
0
@mbp2:~ $ /usr/local/bin/bash -c 'exit 1'; echo $?
1
@mbp2:~ $ /usr/local/bin/bash -c 'exit 12'; echo $?
12

Variables

Seldom there is a misuse of variable expansions. Following are some essential rules:

• Always quote the variable extension. Since the quoting actually preserves newline characters embedded in the variable, if present, whereas non-quoted expansions will remove the newline characters.

Correct:

do_some_stuff "$thing"

Wrong:

do_some_stuff $thing

• Only use curly braces if adjacent to non-spaces, omit curly braces everywhere else. Curly braces are indented to explicitly signal the variable. If it is not explicitly set, the bash parser, as well as the reader, may be confused and assume the variable includes an adjacent character as the Bash variable name.

Correct:

url="${base_url}/${endpoint}?${query_params}"

msg="the api request returned : $result"

Wrong:

do_some_stuff ${thing}

do_some_stuff "${thing}"

• Use single quotes if there are no variables present and reserve double quotes for use in variable expansions.

Correct:

base_url='https://www.gnu.org/

Wrong:

base_url="https://www.gnu.org/"

Arguments

The shell uses positional arguments for function declaration and calls. Besides, many command line programs offer flags to pass in content or optional behavior. Let us see a code where there is this same behavior:

tmp_dir="$(mkdir -p /tmp/email && echo '/tmp/email')"
report=''
distro_list=''
html=''
date_override=''
body_override=''

usage(){
  echo "Usage: email: ${0} [--report <file_path>] [--distro-list <'[email protected]'>]" 1>&2
  echo "                   [--html] [--date-override <date>] [--body-override <body>]" 1>&2
  echo "       Do not use --html and body override in the same call.                 " 1>&2
  exit 1
}
while [[ $# -gt 0 ]]; do
  case "$1" in
    -r|--report)        report="$2";        shift ;;
    -l|--distro-list)   distro_list="$2";   shift ;;
    -h|--html)          html='y' ;;
    -d|--date-override) date_override="$2"; shift;;
    -b|--body-override) body_override="$2"; shift;;
    *) break ;;
  esac
  shift
done
if [[ -z $report ]] || [[ -z $distro_list ]]; then usage; fi
if [[ ! -z $html ]] && [[ ! -z $body_override ]]; then usage; fi

email(){
  local report="$1"
  local distro_list="$2"
  local html="$3"
  local date_override="$4"
  local body_override="$5"

  if [[ $(whoami) == 'root' ]]; then            # docker (k8s, odroid, pi)
    curl_email "$report" "$distro_list" "$html" "$date_override" "$body_override"
  elif [[ $(whoami) == 'sbx_'* ]]; then         # AWS Lambda
    curl_email "$report" "$distro_list" "$html" "$date_override" "$body_override"
  elif [[ $(whoami) == 'skilbjo' ]]; then       # mac OS
    curl_email "$report" "$distro_list" "$html" "$date_override" "$body_override"
  fi
}

email "$report" "$distro_list" "$html" "$date_override" "$body_override"

In the above code, the global functions are listed at the top but initialized with empty values. And a function is added that specifies usage and exits with an error, a common pattern in command line programs before the while statement is used to parse the “$@” arguments. Then the global variables are set with the appropriate arguments based on the flags. This is similar to argc/argv pattern in the C language’s main argument parsing. Then the next code checks whether the required arguments are set. If not, it calls the usage function and the script exits with an error code. Next, the script logic is declared as functions, and lastly, the main function is called, here it is email. Finally, the script logic is then executed, which is a script to send a custom email.

A method to partition a Bash application is to split the application into smaller files and load those files into memory at run-time. This can be achieved by:

file foo

bar(){
  echo 'I ran from a sourced file!'
}

file run-it

#!/usr/bin/env bash

source foo

bar

would return

$ ./run-it
I ran from a sourced file!
$

This method is similar to the way libraries work in other programming languages, like C, Python, and Clojure. In contrast, the Unix process model’s design is such that it aims more to execute independent programs. If the functions are helper functions or more of a library, they are sourced from the application’s entry point. If there are independent programs, they are invoked like any other Unix binary.

Aliases (inlining a function)

To set aliases, place aliases in a ~./aliases file, and use the given syntax:

alias h='cd ~'
alias mkdir='mkdir -p'
alias vim='vim -p'
alias man='function _(){ /usr/bin/man "$1" | col -xb | vim -;};_'
alias ytdl='function _(){ cd ~/Desktop/; youtube-dl -x --audio-format mp3 "$1" & cd -;};_' # download youtube songs
alias "psql.dw"='function _psql(){ psql "$db_uri" -c "$1"; };_psql'

alias x='exit'
alias a='cd ~/dev/aeon/'
alias m='cd ~/dev/markets-etl/'
alias b='cd ~/dev/bash-etl/'
alias d='cd ~/dev'
alias 'netstat.osx'='echo "Proto Recv-Q Send-Q  Local Address          Foreign Address        (state)" && netstat -an | grep LISTEN'

A red flag is that you cannot use aliases with sudo or watch.

@mbp2:cdmtr $ echo "alias 'docker.ssh'='docker-machine ssh default'" >>~/.aliases
@mbp2:cdmtr $ source ~/.aliases
@mbp2:cdmtr $ docker.ssh
docker@default:~$ exit

@mbp2:cdmtr $ watch -n5 docker.ssh
sh: docker.ssh: command not found
^C
@mbp2:cdmtr $

Tips for better application development

• Use as many functions as possible. The inclusion of any extra file can happen inside the function, as a setup function.
• Also, use local variables as much as possible. Besides, passing data to and from functions as arguments is better than using global variables.
• No tabs
• Indentation: 2 spaces
• Employ generic variables that perform a single function well.

Final word

Therefore, the investment you put in your core toolset will benefit you in the long run. As the programs go through changes regularly, a shell-like Bash is essential for daily work and it helps to become a better programmer.

Essential programs

• bash-completion: not a program per se, but a handy tool that lets you use TAB complete for program arguments, like git
• man: man – format and display the online manual pages
• htop: interactive process viewer / a more modern top command
• tree: an illustrated version of the 1s command.
• ack: quickly search for file contents of many files
• vim: vim – Vi IMproved, a programmer’s text editor
• tmux: tmux – terminal multiplexer
• bc: evaluate simple mathematical equations. For example, sleep "$(echo '60 * 5' | bc)" # sleep for 5 min
• gzip: compression/decompression tool. compress: gzip -9 [file]. decompress:
  gzip -d [file].gz
• watch: run the same command repeatedly in a ncurses window. watch -n1 'docker ps | grep 'my-container'

Shells to look for:

• ash/dash: Almquist shell / Debian Almquist shell
• zsh: Z shell
• ksh: Korn shell
• tcsh: C shell
• fish: A modern shell