Chapter 3. Special Characters

What makes a character special? If it has a meaning beyond its literal meaning, a meta-meaning, then we refer to it as a special character.

Special Characters Found In Scripts and Elsewhere

#

Comments. Lines beginning with a # (with the exception of #!) are comments and will not be executed.

   1 # This line is a comment.

Comments may also occur following the end of a command.

   1 echo "A comment will follow." # Comment here.
   2 #                            ^ Note whitespace before #

Comments may also follow whitespace at the beginning of a line.

   1 	# A tab precedes this comment.

Caution

A command may not follow a comment on the same line. There is no method of terminating the comment, in order for "live code" to begin on the same line. Use a new line for the next command.

Note

Of course, a quoted or an escaped # in an echo statement does not begin a comment. Likewise, a # appears in certain parameter-substitution constructs and in numerical constant expressions.
   1 echo "The # here does not begin a comment."
   2 echo 'The # here does not begin a comment.'
   3 echo The \# here does not begin a comment.
   4 echo The # here begins a comment.
   5 
   6 echo ${PATH#*:}       # Parameter substitution, not a comment.
   7 echo $(( 2#101011 ))  # Base conversion, not a comment.
   8 
   9 # Thanks, S.C.
The standard quoting and escape characters (" ' \) escape the #.

Certain pattern matching operations also use the #.

;

Command separator [semicolon]. Permits putting two or more commands on the same line.

   1 echo hello; echo there
   2 
   3 
   4 if [ -x "$filename" ]; then    #  Note that "if" and "then" need whitespace
   5                                #+ separation. Why?
   6   echo "File $filename exists."; cp $filename $filename.bak
   7 else
   8   echo "File $filename not found."; touch $filename
   9 fi; echo "File test complete."

Note that the ";" sometimes needs to be escaped.

;;

Terminator in a case option [double semicolon].

   1 case "$variable" in
   2   abc)  echo "\$variable = abc" ;;
   3   xyz)  echo "\$variable = xyz" ;;
   4 esac

.

"dot" command [period]. Equivalent to source (see Example 14-22). This is a bash builtin.

.

"dot", as a component of a filename. When working with filenames, a leading dot is the prefix of a "hidden" file, a file that an ls will not normally show.
 bash$ touch .hidden-file
 bash$ ls -l	      
 total 10
 -rw-r--r--    1 bozo      4034 Jul 18 22:04 data1.addressbook
 -rw-r--r--    1 bozo      4602 May 25 13:58 data1.addressbook.bak
 -rw-r--r--    1 bozo       877 Dec 17  2000 employment.addressbook
 
 
 bash$ ls -al	      
 total 14
 drwxrwxr-x    2 bozo  bozo      1024 Aug 29 20:54 ./
 drwx------   52 bozo  bozo      3072 Aug 29 20:51 ../
 -rw-r--r--    1 bozo  bozo      4034 Jul 18 22:04 data1.addressbook
 -rw-r--r--    1 bozo  bozo      4602 May 25 13:58 data1.addressbook.bak
 -rw-r--r--    1 bozo  bozo       877 Dec 17  2000 employment.addressbook
 -rw-rw-r--    1 bozo  bozo         0 Aug 29 20:54 .hidden-file
 	        

When considering directory names, a single dot represents the current working directory, and two dots denote the parent directory.

 bash$ pwd
 /home/bozo/projects
 
 bash$ cd .
 bash$ pwd
 /home/bozo/projects
 
 bash$ cd ..
 bash$ pwd
 /home/bozo/
 	        

The dot often appears as the destination (directory) of a file movement command, in this context meaning current directory.

 bash$ cp /home/bozo/current_work/junk/* .
 	        
Copy all the "junk" files to $PWD.

.

"dot" character match. When matching characters, as part of a regular expression, a "dot" matches a single character.

"

partial quoting [double quote]. "STRING" preserves (from interpretation) most of the special characters within STRING. See Chapter 5.

'

full quoting [single quote]. 'STRING' preserves all special characters within STRING. This is a stronger form of quoting than "STRING". See Chapter 5.

,

comma operator. The comma operator [1] links together a series of arithmetic operations. All are evaluated, but only the last one is returned.
   1 let "t2 = ((a = 9, 15 / 3))"  # Set "a =
   2 	       9" and "t2 = 15 / 3"

\

escape [backslash]. A quoting mechanism for single characters.

\X escapes the character X. This has the effect of "quoting" X, equivalent to 'X'. The \ may be used to quote " and ', so they are expressed literally.

See Chapter 5 for an in-depth explanation of escaped characters.

/

Filename path separator [forward slash]. Separates the components of a filename (as in /home/bozo/projects/Makefile).

This is also the division arithmetic operator.

`

command substitution. The `command` construct makes available the output of command for assignment to a variable. This is also known as backquotes or backticks.

:

null command [colon]. This is the shell equivalent of a "NOP" (no op, a do-nothing operation). It may be considered a synonym for the shell builtin true. The ":" command is itself a Bash builtin, and its exit status is true (0).

   1 :
   2 echo $?   # 0

Endless loop:

   1 while :
   2 do
   3    operation-1
   4    operation-2
   5    ...
   6    operation-n
   7 done
   8 
   9 # Same as:
  10 #    while true
  11 #    do
  12 #      ...
  13 #    done

Placeholder in if/then test:

   1 if condition
   2 then :   # Do nothing and branch ahead
   3 else     # Or else ...
   4    take-some-action
   5 fi

Provide a placeholder where a binary operation is expected, see Example 8-2 and default parameters.

   1 : ${username=`whoami`}
   2 # ${username=`whoami`}   Gives an error without the leading :
   3 #                        unless "username" is a command or builtin...

Provide a placeholder where a command is expected in a here document. See Example 18-10.

Evaluate string of variables using parameter substitution (as in Example 9-16).
   1 : ${HOSTNAME?} ${USER?} ${MAIL?}
   2 #  Prints error message
   3 #+ if one or more of essential environmental variables not set.

Variable expansion / substring replacement.

In combination with the > redirection operator, truncates a file to zero length, without changing its permissions. If the file did not previously exist, creates it.
   1 : > data.xxx   # File "data.xxx" now empty.	      
   2 
   3 # Same effect as   cat /dev/null >data.xxx
   4 # However, this does not fork a new process, since ":" is a builtin.
See also Example 15-15.

In combination with the >> redirection operator, has no effect on a pre-existing target file (: >> target_file). If the file did not previously exist, creates it.

Note

This applies to regular files, not pipes, symlinks, and certain special files.

May be used to begin a comment line, although this is not recommended. Using # for a comment turns off error checking for the remainder of that line, so almost anything may appear in a comment. However, this is not the case with :.
   1 : This is a comment that generates an error, ( if [ $x -eq 3] ).

The ":" also serves as a field separator, in /etc/passwd, and in the $PATH variable.
 bash$ echo $PATH
 /usr/local/bin:/bin:/usr/bin:/usr/X11R6/bin:/sbin:/usr/sbin:/usr/games

!

reverse (or negate) the sense of a test or exit status [bang]. The ! operator inverts the exit status of the command to which it is applied (see Example 6-2). It also inverts the meaning of a test operator. This can, for example, change the sense of equal ( = ) to not-equal ( != ). The ! operator is a Bash keyword.

In a different context, the ! also appears in indirect variable references.

In yet another context, from the command line, the ! invokes the Bash history mechanism (see Appendix J). Note that within a script, the history mechanism is disabled.

*

wild card [asterisk]. The * character serves as a "wild card" for filename expansion in globbing. By itself, it matches every filename in a given directory.

 bash$ echo *
 abs-book.sgml add-drive.sh agram.sh alias.sh
 	      

The * also represents any number (or zero) characters in a regular expression.

*

arithmetic operator. In the context of arithmetic operations, the * denotes multiplication.

A double asterisk, **, is the exponentiation operator.

?

test operator. Within certain expressions, the ? indicates a test for a condition.

In a double-parentheses construct, the ? can serve as an element of a C-style trinary operator, ?:.
   1 (( var0 = var1<98?9:21 ))
   2 #                ^ ^
   3 
   4 # if [ "$var1" -lt 98 ]
   5 # then
   6 #   var0=9
   7 # else
   8 #   var0=21
   9 # fi

In a parameter substitution expression, the ? tests whether a variable has been set.

?

wild card. The ? character serves as a single-character "wild card" for filename expansion in globbing, as well as representing one character in an extended regular expression.

$

Variable substitution (contents of a variable).
   1 var1=5
   2 var2=23skidoo
   3 
   4 echo $var1     # 5
   5 echo $var2     # 23skidoo

A $ prefixing a variable name indicates the value the variable holds.

$

end-of-line. In a regular expression, a "$" addresses the end of a line of text.

${}
$*, $@
$?

exit status variable. The $? variable holds the exit status of a command, a function, or of the script itself.

$$

process ID variable. The $$ variable holds the process ID [2] of the script in which it appears.

()

command group.
   1 (a=hello; echo $a)

Important

A listing of commands within parentheses starts a subshell.

Variables inside parentheses, within the subshell, are not visible to the rest of the script. The parent process, the script, cannot read variables created in the child process, the subshell.
   1 a=123
   2 ( a=321; )	      
   3 
   4 echo "a = $a"   # a = 123
   5 # "a" within parentheses acts like a local variable.

array initialization.
   1 Array=(element1 element2 element3)

{xxx,yyy,zzz,...}

Brace expansion.
   1 cat {file1,file2,file3} > combined_file
   2 # Concatenates the files file1, file2, and file3 into combined_file.
   3 
   4 
   5 cp file22.{txt,backup}
   6 # Copies "file22.txt" to "file22.backup"

A command may act upon a comma-separated list of file specs within braces. [3] Filename expansion (globbing) applies to the file specs between the braces.

Caution

No spaces allowed within the braces unless the spaces are quoted or escaped.

echo {file1,file2}\ :{\ A," B",' C'}

file1 : A file1 : B file1 : C file2 : A file2 : B file2 : C

{a..z}

Extended Brace expansion.
   1 echo {a..z} # a b c d e f g h i j k l m n o p q r s t u v w x y z
   2 # Echoes characters between a and z.
   3 
   4 echo {0..3} # 0 1 2 3
   5 # Echoes characters between 0 and 3.

The {a..z} extended brace expansion construction is a feature introduced in version 3 of Bash.

{}

Block of code [curly brackets]. Also referred to as an inline group, this construct, in effect, creates an anonymous function (a function without a name). However, unlike in a "standard" function, the variables inside a code block remain visible to the remainder of the script.

 bash$ { local a;
	      a=123; }
 bash: local: can only be used in a
function
 	      

   1 a=123
   2 { a=321; }
   3 echo "a = $a"   # a = 321   (value inside code block)
   4 
   5 # Thanks, S.C.

The code block enclosed in braces may have I/O redirected to and from it.


Example 3-1. Code blocks and I/O redirection

   1 #!/bin/bash
   2 # Reading lines in /etc/fstab.
   3 
   4 File=/etc/fstab
   5 
   6 {
   7 read line1
   8 read line2
   9 } < $File
  10 
  11 echo "First line in $File is:"
  12 echo "$line1"
  13 echo
  14 echo "Second line in $File is:"
  15 echo "$line2"
  16 
  17 exit 0
  18 
  19 # Now, how do you parse the separate fields of each line?
  20 # Hint: use awk, or . . .
  21 # . . . Hans-Joerg Diers suggests using the "set" Bash builtin.


Example 3-2. Saving the output of a code block to a file

   1 #!/bin/bash
   2 # rpm-check.sh
   3 
   4 #  Queries an rpm file for description, listing,
   5 #+ and whether it can be installed.
   6 #  Saves output to a file.
   7 # 
   8 #  This script illustrates using a code block.
   9 
  10 SUCCESS=0
  11 E_NOARGS=65
  12 
  13 if [ -z "$1" ]
  14 then
  15   echo "Usage: `basename $0` rpm-file"
  16   exit $E_NOARGS
  17 fi  
  18 
  19 { # Begin code block.
  20   echo
  21   echo "Archive Description:"
  22   rpm -qpi $1       # Query description.
  23   echo
  24   echo "Archive Listing:"
  25   rpm -qpl $1       # Query listing.
  26   echo
  27   rpm -i --test $1  # Query whether rpm file can be installed.
  28   if [ "$?" -eq $SUCCESS ]
  29   then
  30     echo "$1 can be installed."
  31   else
  32     echo "$1 cannot be installed."
  33   fi  
  34   echo              # End code block.
  35 } > "$1.test"       # Redirects output of everything in block to file.
  36 
  37 echo "Results of rpm test in file $1.test"
  38 
  39 # See rpm man page for explanation of options.
  40 
  41 exit 0

Note

Unlike a command group within (parentheses), as above, a code block enclosed by {braces} will not normally launch a subshell. [4]

{}

placeholder for text. Used after xargs -i (replace strings option). The {} double curly brackets are a placeholder for output text.

   1 ls . | xargs -i -t cp ./{} $1
   2 #            ^^         ^^
   3 
   4 # From "ex42.sh" (copydir.sh) example.

anchor id="semicolonesc">

{} \;

pathname. Mostly used in find constructs. This is not a shell builtin.

Note

The ";" ends the -exec option of a find command sequence. It needs to be escaped to protect it from interpretation by the shell.

[ ]

test.

Test expression between [ ]. Note that [ is part of the shell builtin test (and a synonym for it), not a link to the external command /usr/bin/test.

[[ ]]

test.

Test expression between [[ ]]. More flexible than the single-bracket [ ] test, this is a shell keyword.

See the discussion on the [[ ... ]] construct.

[ ]

array element.

In the context of an array, brackets set off the numbering of each element of that array.
   1 Array[1]=slot_1
   2 echo ${Array[1]}

[ ]

range of characters.

As part of a regular expression, brackets delineate a range of characters to match.

(( ))

integer expansion.

Expand and evaluate integer expression between (( )).

See the discussion on the (( ... )) construct.

> &> >& >> < <>

scriptname >filename redirects the output of scriptname to file filename. Overwrite filename if it already exists.

command &>filename redirects both the stdout and the stderr of command to filename.

command >&2 redirects stdout of command to stderr.

scriptname >>filename appends the output of scriptname to file filename. If filename does not already exist, it is created.

[i]<>filename opens file filename for reading and writing, and assigns file descriptor i to it. If filename does not exist, it is created.

(command)>

<(command)

In a different context, the "<" and ">" characters act as string comparison operators.

In yet another context, the "<" and ">" characters act as integer comparison operators. See also Example 15-9.

<<

redirection used in a here document.

<<<

redirection used in a here string.

<, >

ASCII comparison.
   1 veg1=carrots
   2 veg2=tomatoes
   3 
   4 if [[ "$veg1" < "$veg2" ]]
   5 then
   6   echo "Although $veg1 precede $veg2 in the dictionary,"
   7   echo -n "this does not necessarily imply anything "
   8   echo "about my culinary preferences."
   9 else
  10   echo "What kind of dictionary are you using, anyhow?"
  11 fi

\<, \>

bash$ grep '\<the\>' textfile

|

pipe. Passes the output (stdout of a previous command to the input (stdin) of the next one, or to the shell. This is a method of chaining commands together.

   1 echo ls -l | sh
   2 #  Passes the output of "echo ls -l" to the shell,
   3 #+ with the same result as a simple "ls -l".
   4 
   5 
   6 cat *.lst | sort | uniq
   7 # Merges and sorts all ".lst" files, then deletes duplicate lines.

The output of a command or commands may be piped to a script.
   1 #!/bin/bash
   2 # uppercase.sh : Changes input to uppercase.
   3 
   4 tr 'a-z' 'A-Z'
   5 #  Letter ranges must be quoted
   6 #+ to prevent filename generation from single-letter filenames.
   7 
   8 exit 0
Now, let us pipe the output of ls -l to this script.
 bash$ ls -l | ./uppercase.sh
 -RW-RW-R--    1 BOZO  BOZO       109 APR  7 19:49 1.TXT
 -RW-RW-R--    1 BOZO  BOZO       109 APR 14 16:48 2.TXT
 -RW-R--R--    1 BOZO  BOZO       725 APR 20 20:56 DATA-FILE
 	      

Note

The stdout of each process in a pipe must be read as the stdin of the next. If this is not the case, the data stream will block, and the pipe will not behave as expected.
   1 cat file1 file2 | ls -l | sort
   2 # The output from "cat file1 file2" disappears.

A pipe runs as a child process, and therefore cannot alter script variables.
   1 variable="initial_value"
   2 echo "new_value" | read variable
   3 echo "variable = $variable"     # variable = initial_value

If one of the commands in the pipe aborts, this prematurely terminates execution of the pipe. Called a broken pipe, this condition sends a SIGPIPE signal.

>|

force redirection (even if the noclobber option is set). This will forcibly overwrite an existing file.

||

OR logical operator. In a test construct, the || operator causes a return of 0 (success) if either of the linked test conditions is true.

&

Run job in background. A command followed by an & will run in the background.

 bash$ sleep 10 &
 [1] 850
 [1]+  Done                    sleep 10
 	      

Within a script, commands and even loops may run in the background.


Example 3-3. Running a loop in the background

   1 #!/bin/bash
   2 # background-loop.sh
   3 
   4 for i in 1 2 3 4 5 6 7 8 9 10            # First loop.
   5 do
   6   echo -n "$i "
   7 done & # Run this loop in background.
   8        # Will sometimes execute after second loop.
   9 
  10 echo   # This 'echo' sometimes will not display.
  11 
  12 for i in 11 12 13 14 15 16 17 18 19 20   # Second loop.
  13 do
  14   echo -n "$i "
  15 done  
  16 
  17 echo   # This 'echo' sometimes will not display.
  18 
  19 # ======================================================
  20 
  21 # The expected output from the script:
  22 # 1 2 3 4 5 6 7 8 9 10 
  23 # 11 12 13 14 15 16 17 18 19 20 
  24 
  25 # Sometimes, though, you get:
  26 # 11 12 13 14 15 16 17 18 19 20 
  27 # 1 2 3 4 5 6 7 8 9 10 bozo $
  28 # (The second 'echo' doesn't execute. Why?)
  29 
  30 # Occasionally also:
  31 # 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
  32 # (The first 'echo' doesn't execute. Why?)
  33 
  34 # Very rarely something like:
  35 # 11 12 13 1 2 3 4 5 6 7 8 9 10 14 15 16 17 18 19 20 
  36 # The foreground loop preempts the background one.
  37 
  38 exit 0
  39 
  40 #  Nasimuddin Ansari suggests adding    sleep 1
  41 #+ after the   echo -n "$i"   in lines 6 and 14,
  42 #+ for some real fun.

Caution

A command run in the background within a script may cause the script to hang, waiting for a keystroke. Fortunately, there is a remedy for this.

&&

AND logical operator. In a test construct, the && operator causes a return of 0 (success) only if both the linked test conditions are true.

-

option, prefix. Option flag for a command or filter. Prefix for an operator. Prefix for a default parameter in parameter substitution.

COMMAND -[Option1][Option2][...]

ls -al

sort -dfu $filename

   1 if [ $file1 -ot $file2 ]
   2 then #      ^
   3   echo "File $file1 is older than $file2."
   4 fi
   5 
   6 if [ "$a" -eq "$b" ]
   7 then      ^
   8   echo "$a is equal to $b."
   9 fi
  10 
  11 if [ "$c" -eq 24 -a "$d" -eq 47 ]
  12 then      ^              ^
  13   echo "$c equals 24 and $d equals 47."
  14 fi
  15 
  16 
  17 param2=${param1:-$DEFAULTVAL}
  18 #               ^

--

The double-dash -- prefixes long (verbatim) options to commands.

sort --ignore-leading-blanks

Used with a Bash builtin, it means the end of options to that particular command.

Tip

This provides a handy means of removing files whose names begin with a dash.
 bash$ ls -l
 -rw-r--r-- 1 bozo bozo 0 Nov 25 12:29 -badname
 
 
 bash$ rm -- -badname
 
 bash$ ls -l
 total 0

The double-dash is also used in conjunction with set.

set -- $variable (as in Example 14-18)

-

redirection from/to stdin or stdout [dash].

 bash$ cat -
 abc
 abc
 
 ...
 
 Ctl-D

As expected, cat - echoes stdin, in this case keyboarded user input, to stdout. But, does I/O redirection using - have real-world applications?

   1 (cd /source/directory && tar cf - . ) | (cd /dest/directory && tar xpvf -)
   2 # Move entire file tree from one directory to another
   3 # [courtesy Alan Cox <a.cox@swansea.ac.uk>, with a minor change]
   4 
   5 # 1) cd /source/directory
   6 #    Source directory, where the files to be moved are.
   7 # 2) &&
   8 #   "And-list": if the 'cd' operation successful,
   9 #    then execute the next command.
  10 # 3) tar cf - .
  11 #    The 'c' option 'tar' archiving command creates a new archive,
  12 #    the 'f' (file) option, followed by '-' designates the target file
  13 #    as stdout, and do it in current directory tree ('.').
  14 # 4) |
  15 #    Piped to ...
  16 # 5) ( ... )
  17 #    a subshell
  18 # 6) cd /dest/directory
  19 #    Change to the destination directory.
  20 # 7) &&
  21 #   "And-list", as above
  22 # 8) tar xpvf -
  23 #    Unarchive ('x'), preserve ownership and file permissions ('p'),
  24 #    and send verbose messages to stdout ('v'),
  25 #    reading data from stdin ('f' followed by '-').
  26 #
  27 #    Note that 'x' is a command, and 'p', 'v', 'f' are options.
  28 #
  29 # Whew!
  30 
  31 
  32 
  33 # More elegant than, but equivalent to:
  34 #   cd source/directory
  35 #   tar cf - . | (cd ../dest/directory; tar xpvf -)
  36 #
  37 #     Also having same effect:
  38 # cp -a /source/directory/* /dest/directory
  39 #     Or:
  40 # cp -a /source/directory/* /source/directory/.[^.]* /dest/directory
  41 #     If there are hidden files in /source/directory.

   1 bunzip2 -c linux-2.6.16.tar.bz2 | tar xvf -
   2 #  --uncompress tar file--    | --then pass it to "tar"--
   3 #  If "tar" has not been patched to handle "bunzip2",
   4 #+ this needs to be done in two discrete steps, using a pipe.
   5 #  The purpose of the exercise is to unarchive "bzipped" kernel source.

Note that in this context the "-" is not itself a Bash operator, but rather an option recognized by certain UNIX utilities that write to stdout, such as tar, cat, etc.

 bash$ echo "whatever" | cat -
 whatever 

Where a filename is expected, - redirects output to stdout (sometimes seen with tar cf), or accepts input from stdin, rather than from a file. This is a method of using a file-oriented utility as a filter in a pipe.

 bash$ file
 Usage: file [-bciknvzL] [-f namefile] [-m magicfiles] file...
 	      
By itself on the command line, file fails with an error message.

Add a "-" for a more useful result. This causes the shell to await user input.
 bash$ file -
 abc
 standard input:              ASCII text
 
 
 
 bash$ file -
 #!/bin/bash
 standard input:              Bourne-Again shell script text executable
 	      
Now the command accepts input from stdin and analyzes it.

The "-" can be used to pipe stdout to other commands. This permits such stunts as prepending lines to a file.

Using diff to compare a file with a section of another:

grep Linux file1 | diff file2 -

Finally, a real-world example using - with tar.


Example 3-4. Backup of all files changed in last day

   1 #!/bin/bash
   2 
   3 #  Backs up all files in current directory modified within last 24 hours
   4 #+ in a "tarball" (tarred and gzipped file).
   5 
   6 BACKUPFILE=backup-$(date +%m-%d-%Y)
   7 #                 Embeds date in backup filename.
   8 #                 Thanks, Joshua Tschida, for the idea.
   9 archive=${1:-$BACKUPFILE}
  10 #  If no backup-archive filename specified on command line,
  11 #+ it will default to "backup-MM-DD-YYYY.tar.gz."
  12 
  13 tar cvf - `find . -mtime -1 -type f -print` > $archive.tar
  14 gzip $archive.tar
  15 echo "Directory $PWD backed up in archive file \"$archive.tar.gz\"."
  16 
  17 
  18 #  Stephane Chazelas points out that the above code will fail
  19 #+ if there are too many files found
  20 #+ or if any filenames contain blank characters.
  21 
  22 # He suggests the following alternatives:
  23 # -------------------------------------------------------------------
  24 #   find . -mtime -1 -type f -print0 | xargs -0 tar rvf "$archive.tar"
  25 #      using the GNU version of "find".
  26 
  27 
  28 #   find . -mtime -1 -type f -exec tar rvf "$archive.tar" '{}' \;
  29 #         portable to other UNIX flavors, but much slower.
  30 # -------------------------------------------------------------------
  31 
  32 
  33 exit 0

Caution

Filenames beginning with "-" may cause problems when coupled with the "-" redirection operator. A script should check for this and add an appropriate prefix to such filenames, for example ./-FILENAME, $PWD/-FILENAME, or $PATHNAME/-FILENAME.

If the value of a variable begins with a -, this may likewise create problems.
   1 var="-n"
   2 echo $var		
   3 # Has the effect of "echo -n", and outputs nothing.

-

previous working directory. A cd - command changes to the previous working directory. This uses the $OLDPWD environmental variable.

Caution

Do not confuse the "-" used in this sense with the "-" redirection operator just discussed. The interpretation of the "-" depends on the context in which it appears.

-

Minus. Minus sign in an arithmetic operation.

=

Equals. Assignment operator
   1 a=28
   2 echo $a   # 28

In a different context, the "=" is a string comparison operator.

+

Plus. Addition arithmetic operator.

In a different context, the + is a Regular Expression operator.

+

Option. Option flag for a command or filter.

Certain commands and builtins use the + to enable certain options and the - to disable them. In parameter substitution, the + prefixes an alternate value that a variable expands to.

%

modulo. Modulo (remainder of a division) arithmetic operation.

   1 let "z = 5 % 3"
   2 echo $z  # 2

In a different context, the % is a pattern matching operator.

~

home directory [tilde]. This corresponds to the $HOME internal variable. ~bozo is bozo's home directory, and ls ~bozo lists the contents of it. ~/ is the current user's home directory, and ls ~/ lists the contents of it.
 bash$ echo ~bozo
 /home/bozo
 
 bash$ echo ~
 /home/bozo
 
 bash$ echo ~/
 /home/bozo/
 
 bash$ echo ~:
 /home/bozo:
 
 bash$ echo ~nonexistent-user
 ~nonexistent-user
 	      

~+

current working directory. This corresponds to the $PWD internal variable.

~-

previous working directory. This corresponds to the $OLDPWD internal variable.

=~

regular expression match. This operator was introduced with version 3 of Bash.

^

beginning-of-line. In a regular expression, a "^" addresses the beginning of a line of text.

Control Characters

change the behavior of the terminal or text display. A control character is a CONTROL + key combination (pressed simultaneously). A control character may also be written in octal or hexadecimal notation, following an escape.

Control characters are not normally useful inside a script.

  • Ctl-A

    Moves cursor to beginning of line of text (on the command-line).

  • Ctl-B

    Backspace (nondestructive).

  • Ctl-C

    Break. Terminate a foreground job.

  • Ctl-D

    Log out from a shell (similar to exit).

    EOF (end-of-file). This also terminates input from stdin.

    When typing text on the console or in an xterm window, Ctl-D erases the character under the cursor. When there are no characters present, Ctl-D logs out of the session, as expected. In an xterm window, this has the effect of closing the window.

  • Ctl-E

    Moves cursor to end of line of text (on the command-line).

  • Ctl-F

    Moves cursor forward one character position (on the command-line).

  • Ctl-G

    BEL. On some old-time teletype terminals, this would actually ring a bell. In an xterm it might beep.

  • Ctl-H

    Rubout (destructive backspace). Erases characters the cursor backs over while backspacing.

       1 #!/bin/bash
       2 # Embedding Ctl-H in a string.
       3 
       4 a="^H^H"                  # Two Ctl-H's -- backspaces
       5                           # ctl-V ctl-H, using vi/vim
       6 echo "abcdef"             # abcdef
       7 echo
       8 echo -n "abcdef$a "       # abcd f
       9 #  Space at end  ^              ^  Backspaces twice.
      10 echo
      11 echo -n "abcdef$a"        # abcdef
      12 #  No space at end               ^ Doesn't backspace (why?).
      13                           # Results may not be quite as expected.
      14 echo; echo
      15 
      16 # Constantin Hagemeier suggests trying:
      17 # a=$'\010\010'
      18 # a=$'\b\b'
      19 # a=$'\x08\x08'
      20 # But, this does not change the results.

  • Ctl-I

    Horizontal tab.

  • Ctl-J

    Newline (line feed). In a script, may also be expressed in octal notation -- '\012' or in hexadecimal -- '\x0a'.

  • Ctl-K

    Vertical tab.

    When typing text on the console or in an xterm window, Ctl-K erases from the character under the cursor to end of line. Within a script, Ctl-K may behave differently, as in Lee Lee Maschmeyer's example, below.

  • Ctl-L

    Formfeed (clear the terminal screen). In a terminal, this has the same effect as the clear command. When sent to a printer, a Ctl-L causes an advance to end of the paper sheet.

  • Ctl-M

    Carriage return.

       1 #!/bin/bash
       2 # Thank you, Lee Maschmeyer, for this example.
       3 
       4 read -n 1 -s -p \
       5 $'Control-M leaves cursor at beginning of this line. Press Enter. \x0d'
       6            # Of course, '0d' is the hex equivalent of Control-M.
       7 echo >&2   #  The '-s' makes anything typed silent,
       8            #+ so it is necessary to go to new line explicitly.
       9 
      10 read -n 1 -s -p $'Control-J leaves cursor on next line. \x0a'
      11            #  '0a' is the hex equivalent of Control-J, linefeed.
      12 echo >&2
      13 
      14 ###
      15 
      16 read -n 1 -s -p $'And Control-K\x0bgoes straight down.'
      17 echo >&2   #  Control-K is vertical tab.
      18 
      19 # A better example of the effect of a vertical tab is:
      20 
      21 var=$'\x0aThis is the bottom line\x0bThis is the top line\x0a'
      22 echo "$var"
      23 #  This works the same way as the above example. However:
      24 echo "$var" | col
      25 #  This causes the right end of the line to be higher than the left end.
      26 #  It also explains why we started and ended with a line feed --
      27 #+ to avoid a garbled screen.
      28 
      29 # As Lee Maschmeyer explains:
      30 # --------------------------
      31 #  In the [first vertical tab example] . . . the vertical tab
      32 #+ makes the printing go straight down without a carriage return.
      33 #  This is true only on devices, such as the Linux console,
      34 #+ that can't go "backward."
      35 #  The real purpose of VT is to go straight UP, not down.
      36 #  It can be used to print superscripts on a printer.
      37 #  The col utility can be used to emulate the proper behavior of VT.
      38 
      39 exit 0

  • Ctl-N

    Erases a line of text recalled from history buffer [6] (on the command-line).

  • Ctl-O

    Issues a newline (on the command-line).

  • Ctl-P

    Recalls last command from history buffer (on the command-line).

  • Ctl-Q

    Resume (XON).

    This resumes stdin in a terminal.

  • Ctl-R

    Backwards search for text in history buffer (on the command-line).

  • Ctl-S

    Suspend (XOFF).

    This freezes stdin in a terminal. (Use Ctl-Q to restore input.)

  • Ctl-T

    Reverses the position of the character the cursor is on with the previous character (on the command-line).

  • Ctl-U

    Erase a line of input, from the cursor backward to beginning of line. In some settings, Ctl-U erases the entire line of input, regardless of cursor position.

  • Ctl-V

    When inputting text, Ctl-V permits inserting control characters. For example, the following two are equivalent:
       1 echo -e '\x0a'
       2 echo <Ctl-V><Ctl-J>

    Ctl-V is primarily useful from within a text editor.

  • Ctl-W

    When typing text on the console or in an xterm window, Ctl-W erases from the character under the cursor backwards to the first instance of whitespace. In some settings, Ctl-W erases backwards to first non-alphanumeric character.

  • Ctl-X

    In certain word processing programs, Cuts highlighted text and copies to clipboard.

  • Ctl-Y

    Pastes back text previously erased (with Ctl-K or Ctl-U).

  • Ctl-Z

    Pauses a foreground job.

    Substitute operation in certain word processing applications.

    EOF (end-of-file) character in the MSDOS filesystem.

Whitespace

functions as a separator, separating commands or variables. Whitespace consists of either spaces, tabs, blank lines, or any combination thereof. [7] In some contexts, such as variable assignment, whitespace is not permitted, and results in a syntax error.

Blank lines have no effect on the action of a script, and are therefore useful for visually separating functional sections.

$IFS, the special variable separating fields of input to certain commands, defaults to whitespace.

To preserve whitespace within a string or in a variable, use quoting.

UNIX filters can target and operate on whitespace using the POSIX character class [:space:].

Notes

[1]

An operator is an agent that carries out an operation. Some examples are the common arithmetic operators, + - * /. In Bash, there is some overlap between the concepts of operator and keyword.

[2]

A PID, or process ID, is a number assigned to a running process. The PIDs of running processes may be viewed with a ps command.

Definition: A process is an executing program, sometimes referred to as a job.

[3]

The shell does the brace expansion. The command itself acts upon the result of the expansion.

[4]

Exception: a code block in braces as part of a pipe may run as a subshell.

   1 ls | { read firstline; read secondline; }
   2 #  Error. The code block in braces runs as a subshell,
   3 #+ so the output of "ls" cannot be passed to variables within the block.
   4 echo "First line is $firstline; second line is $secondline"  # Won't work.
   5 
   6 # Thanks, S.C.

[5]

Even as in olden times a philtre denoted a potion alleged to have magical transformative powers, so does a UNIX filter transform its target in (rougly) analogous fashion. (The coder who comes up with a "love philtre" that runs on a Linux machine will likely win accolades and honors.)

[6]

Bash stores a list of commands previously issued from the command-line in a buffer, or memory space, for recall with the builtin history commands.

[7]

A linefeed (newline) is also a whitespace character. This explains why a blank line, consisting only of a linefeed, is considered whitespace.