|
Chapter 3. Special CharactersWhat makes a character special? If it has a meaning beyond its literal meaning, a meta-meaning, then we refer to it as a special character. Along with commands and keywords, special characters are building blocks of Bash scripts. Special Characters Found In Scripts and Elsewhere - #
# This line is a comment. | Comments may also occur following the end of a command. echo "A comment will follow." # Comment here.# ^ Note whitespace before # | Comments may also follow whitespace at the beginning of a line. # A tab precedes this comment. | Comments may even be embedded within a pipe. initial=( `cat "$startfile" | sed -e '/#/d' | tr -d '' |# Delete lines containing '#' comment character. sed -e 's/./. /g' -e 's/_/_ /g'` )# Excerpted from life.sh script | | A command may not follow a comment on the same line. There is no method of terminating the comment, in order for "live code" to begin on the same line. Use a new line for the next command. | | Of course, a quoted or an escaped # in an echo statement does not begin a comment. Likewise, a # appears in certain parameter-substitution constructs and in numerical constant expressions. echo "The # here does not begin a comment." echo 'The # here does not begin a comment.'echo The # here does not begin a comment.echo The # here begins a comment.echo ${PATH#*:} # Parameter substitution, not a comment.echo $(( 2#101011 )) # Base conversion, not a comment.# Thanks, S.C. | The standard quoting and escape characters (" ' ) escape the #. | Certain pattern matching operations also use the #. - ;
echo hello; echo thereif [ -x "$filename" ]; then # Note the space after the semicolon.#+ ^^ echo "File $filename exists."; cp $filename $filename.bakelse # ^^ echo "File $filename not found."; touch $filenamefi; echo "File test complete." | Note that the ";" sometimes needs to be escaped. - ;
case "$variable" in abc) echo "$variable = abc" ; xyz) echo "$variable = xyz" ;esac | - ;&, ;&
- .
- .
When considering directory names, a singledot represents the current working directory,and two dots denote the parentdirectory. bash$ pwd/home/bozo/projectsbash$ cd .bash$ pwd/home/bozo/projectsbash$ cd ..bash$ pwd/home/bozo/ | The dot often appears as the destination (directory) of a file movement command, in this context meaning current directory. bash$ cp /home/bozo/current_work/junk/* . | Copy all the "junk" files to$PWD.- .
- "
- '
- ,
The comma operator can also concatenate strings. for file in /{,usr/}bin/*calc# ^ Find all executable files ending in "calc" #+ in /bin and /usr/bin directories.do if [ -x "$file" ] then echo $file fidone# /bin/ipcalc# /usr/bin/kcalc# /usr/bin/oidcalc# /usr/bin/oocalc# Thank you, Rory Winston, for pointing this out. | - ,, ,
Xescapes the characterX. This has the effect of"quoting" X, equivalentto 'X'. The maybe used to quote " and ',so they are expressed literally. See Chapter 5 for an in-depth explanation of escaped characters. - /
This is also the division arithmetic operator. - `
- :
Endless loop: while :do operation-1 operation-2 ... operation-ndone# Same as:# while true# do# ...# done | Placeholder in if/then test: if conditionthen : # Do nothing and branch aheadelse # Or else ... take-some-actionfi | Provide a placeholder where a binary operation is expected, see Example 8-2 and default parameters. : ${username=`whoami`}# ${username=`whoami`} Gives an error without the leading :# unless "username" is a command or builtin...: ${1?"Usage: $0 ARGUMENT"} # From "usage-message.sh example script. | Provide a placeholder where a command is expected in a here document. See Example 19-10. Evaluate string of variables usingparameter substitution(as in Example 10-7). : ${HOSTNAME?} ${USER?} ${MAIL?}# Prints error message#+ if one or more of essential environmental variables not set. | Variable expansion / substring replacement. In combination with the > redirection operator, truncates a file to zero length, without changing its permissions. If the file did not previously exist, creates it. : > data.xxx # File "data.xxx" now empty. # Same effect as cat /dev/null >data.xxx# However, this does not fork a new process, since ":" is a builtin. | See also Example 16-15.In combination with the >> redirection operator, has no effect on a pre-existing target file (: >> target_file). If the file did not previously exist, creates it. | This applies to regular files, not pipes, symlinks, and certain special files. | May be used to begin a comment line, although this is not recommended. Using # for a comment turns off error checking for the remainder of that line, so almost anything may appear in a comment. However, this is not the case with :. : This is a comment that generates an error, ( if [ $x -eq 3] ). | The ":" serves as a field separator, in /etc/passwd, and in the $PATH variable. bash$ echo $PATH/usr/local/bin:/bin:/usr/bin:/usr/X11R6/bin:/sbin:/usr/sbin:/usr/games |
A colon is acceptable as a function name. :(){ echo "The name of this function is "$FUNCNAME" # Why use a colon as a function name? # It's a way of obfuscating your code.}:# The name of this function is : | This is not portable behavior, and therefore not a recommended practice.
A colon can serve as a placeholder in an otherwise empty function. not_empty (){ :} # Contains a : (null command), and so is not empty. | - !
In a different context, the ! also appears in indirect variablereferences. In yet another context, from the commandline, the ! invokes theBash history mechanism (see Appendix L). Note that within a script,the history mechanism is disabled. - *
bash$ echo *abs-book.sgml add-drive.sh agram.sh alias.sh |
The * also represents any number (or zero) characters in a regular expression. - *
** A double asterisk can represent the exponentiation operator or extended file-match globbing. - ?
In a double-parentheses construct, the ?can serve as an element of a C-styletrinary operator. condition?result-if-true:result-if-false (( var0 = var1<98?9:21 ))# ^ ^# if [ "$var1" -lt 98 ]# then# var0=9# else# var0=21# fi | In a parameter substitution expression, the ?tests whether a variable has beenset. - ?
- $
A $ prefixing a variable nameindicates the value the variableholds. - $
- ${}
- $' ... '
- $*, $@
- $?
- $$
- ()
| A listing of commands within parentheses starts a subshell. Variables inside parentheses, within the subshell, are not visible to the rest of the script. The parent process, the script, cannot read variables created in the child process, the subshell. a=123( a=321; ) echo "a = $a" # a = 123# "a" within parentheses acts like a local variable. | | - {xxx,yyy,zzz,...}
A command may act upon a comma-separated list of file specs within braces. Filename expansion (globbing) applies to the file specs between the braces. | No spaces allowed within the braces unless the spaces are quoted or escaped. echo {file1,file2} :{ A," B",' C'} file1 : A file1 : B file1 : C file2 : A file2 : B file2 : C | - {a..z}
The {a..z} extended brace expansion construction is a feature introduced in version 3 of Bash. - {}
bash$ { local a; a=123; }bash: local: can only be used in afunction | a=123{ a=321; }echo "a = $a" # a = 321 (value inside code block)# Thanks, S.C. |
The code block enclosed in braces may have I/O redirected to and from it. Example 3-1. Code blocks and I/O redirection #!/bin/bash# Reading lines in /etc/fstab.File=/etc/fstab{read line1read line2} < $Fileecho "First line in $File is:" echo "$line1" echoecho "Second line in $File is:" echo "$line2" exit 0# Now, how do you parse the separate fields of each line?# Hint: use awk, or . . .# . . . Hans-Joerg Diers suggests using the "set" Bash builtin. |
Example 3-2. Saving the output of a code block to a file #!/bin/bash# rpm-check.sh# Queries an rpm file for description, listing,#+ and whether it can be installed.# Saves output to a file.# # This script illustrates using a code block.SUCCESS=0E_NOARGS=65if [ -z "$1" ]then echo "Usage: `basename $0` rpm-file" exit $E_NOARGSfi { # Begin code block. echo echo "Archive Description:" rpm -qpi $1 # Query description. echo echo "Archive Listing:" rpm -qpl $1 # Query listing. echo rpm -i --test $1 # Query whether rpm file can be installed. if [ "$?" -eq $SUCCESS ] then echo "$1 can be installed." else echo "$1 cannot be installed." fi echo # End code block.} > "$1.test" # Redirects output of everything in block to file.echo "Results of rpm test in file $1.test" # See rpm man page for explanation of options.exit 0 | | Unlike a command group within (parentheses), as above, a code block enclosed by {braces} will not normally launch a subshell. | - {}
ls . | xargs -i -t cp ./{} $1# ^^ ^^# From "ex42.sh" (copydir.sh) example. |
- {} ;
| The ";" endsthe -exec option of afind command sequence. It needsto be escaped to protect it from interpretation by theshell. | - [ ]
Test expression between [ ].Note that [ is part of the shell builtin test (and a synonym for it), not a link to the external command /usr/bin/test. - [[ ]]
Test expression between [[ ]]. More flexible than the single-bracket [ ] test, this is a shell keyword. See the discussion on the [[ ... ]] construct. - [ ]
In the context of an array, brackets set off the numbering of each element of that array. Array[1]=slot_1echo ${Array[1]} | - [ ]
As part of a regular expression, brackets delineate a range of characters to match. - $[ ... ]
Evaluate integer expression between $[ ]. a=3b=7echo $[$a+$b] # 10echo $[$a*$b] # 21 | Note that this usage is deprecated, and has been replaced by the (( ... )) construct. - (( ))
Expand and evaluate integer expression between (( )). See the discussion on the (( ... )) construct. - > &> >& >> < <>
scriptname >filename redirects the output of scriptname to file filename. Overwrite filename if it already exists.
command &>filename redirects both the stdout and the stderr of command to filename. | This is useful for suppressing output when testing for a condition. For example, let us test whether a certain command exists. bash$ type bogus_command &>/dev/nullbash$ echo $?1 | Or in a script: command_test () { type "$1" &>/dev/null; }# ^cmd=rmdir # Legitimate command.command_test $cmd; echo $? # 0cmd=bogus_command # Illegitimate commandcommand_test $cmd; echo $? # 1 | |
command >&2 redirects stdout of command to stderr. scriptname >>filename appends the output of scriptname to file filename. If filename does not already exist, it is created.
[i]<>filename opens file filename for reading and writing, and assigns file descriptor i to it. If filename does not exist, it is created. (command)> <(command) In a different context, the "<" and ">" characters act as string comparison operators. In yet another context, the "<" and ">" characters act as integer comparison operators. See also Example 16-9. - <<
- <<<
- <, >
- <, >
bash$ grep '<the>' textfile - |
echo ls -l | sh# Passes the output of "echo ls -l" to the shell,#+ with the same result as a simple "ls -l".cat *.lst | sort | uniq# Merges and sorts all ".lst" files, then deletes duplicate lines. | The output of a command or commands may be piped to a script. #!/bin/bash# uppercase.sh : Changes input to uppercase.tr 'a-z' 'A-Z'# Letter ranges must be quoted#+ to prevent filename generation from single-letter filenames.exit 0 | Now, let us pipe the output of ls -l to this script. bash$ ls -l | ./uppercase.sh-RW-RW-R-- 1 BOZO BOZO 109 APR 7 19:49 1.TXT -RW-RW-R-- 1 BOZO BOZO 109 APR 14 16:48 2.TXT -RW-R--R-- 1 BOZO BOZO 725 APR 20 20:56 DATA-FILE | | The stdout of each process in a pipe must be read as the stdin of the next. If this is not the case, the data stream will block, and the pipe will not behave as expected. cat file1 file2 | ls -l | sort# The output from "cat file1 file2" disappears. | A pipe runs as a child process, and therefore cannot alter script variables. variable="initial_value" echo "new_value" | read variableecho "variable = $variable" # variable = initial_value | If one of the commands in the pipe aborts, this prematurely terminates execution of the pipe. Called a broken pipe, this condition sends a SIGPIPE signal. | - >|
- ||
- &
bash$ sleep 10 &[1] 850[1]+ Done sleep 10 | Within a script, commands and even loops may run in the background.
Example 3-3. Running a loop in the background #!/bin/bash# background-loop.shfor i in 1 2 3 4 5 6 7 8 9 10 # First loop.do echo -n "$i " done & # Run this loop in background. # Will sometimes execute after second loop.echo # This 'echo' sometimes will not display.for i in 11 12 13 14 15 16 17 18 19 20 # Second loop.do echo -n "$i " done echo # This 'echo' sometimes will not display.# ======================================================# The expected output from the script:# 1 2 3 4 5 6 7 8 9 10 # 11 12 13 14 15 16 17 18 19 20 # Sometimes, though, you get:# 11 12 13 14 15 16 17 18 19 20 # 1 2 3 4 5 6 7 8 9 10 bozo $# (The second 'echo' doesn't execute. Why?)# Occasionally also:# 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20# (The first 'echo' doesn't execute. Why?)# Very rarely something like:# 11 12 13 1 2 3 4 5 6 7 8 9 10 14 15 16 17 18 19 20 # The foreground loop preempts the background one.exit 0# Nasimuddin Ansari suggests adding sleep 1#+ after the echo -n "$i" in lines 6 and 14,#+ for some real fun. | | A command run in the background within a script may cause the script to hang, waiting for a keystroke. Fortunately, there is a remedy for this. | - &&
- -
COMMAND -[Option1][Option2][...] ls -al sort -dfu $filename if [ $file1 -ot $file2 ]then # ^ echo "File $file1 is older than $file2." fiif [ "$a" -eq "$b" ]then # ^ echo "$a is equal to $b." fiif [ "$c" -eq 24 -a "$d" -eq 47 ]then # ^ ^ echo "$c equals 24 and $d equals 47." fiparam2=${param1:-$DEFAULTVAL}# ^ |
-- The double-dash-- prefixes long(verbatim) options to commands. sort --ignore-leading-blanks Used with a Bash builtin, it means the end ofoptions to that particular command. | This provides a handy means of removing files whose names begin with a dash. bash$ ls -l-rw-r--r-- 1 bozo bozo 0 Nov 25 12:29 -badnamebash$ rm -- -badnamebash$ ls -ltotal 0 | | The double-dash is also used in conjunction with set. set -- $variable (as in Example 15-18) - -
bash$ cat -abcabc...Ctl-D | As expected, cat - echoes stdin, in this case keyboarded user input, to stdout. But, does I/O redirection using - have real-world applications? (cd /source/directory && tar cf - . ) | (cd /dest/directory && tar xpvf -)# Move entire file tree from one directory to another# [courtesy Alan Cox <[email protected]>, with a minor change]# 1) cd /source/directory# Source directory, where the files to be moved are.# 2) &&# "And-list": if the 'cd' operation successful,# then execute the next command.# 3) tar cf - .# The 'c' option 'tar' archiving command creates a new archive,# the 'f' (file) option, followed by '-' designates the target file# as stdout, and do it in current directory tree ('.').# 4) |# Piped to ...# 5) ( ... )# a subshell# 6) cd /dest/directory# Change to the destination directory.# 7) &&# "And-list", as above# 8) tar xpvf -# Unarchive ('x'), preserve ownership and file permissions ('p'),# and send verbose messages to stdout ('v'),# reading data from stdin ('f' followed by '-').## Note that 'x' is a command, and 'p', 'v', 'f' are options.## Whew!# More elegant than, but equivalent to:# cd source/directory# tar cf - . | (cd ../dest/directory; tar xpvf -)## Also having same effect:# cp -a /source/directory/* /dest/directory# Or:# cp -a /source/directory/* /source/directory/.[^.]* /dest/directory# If there are hidden files in /source/directory. | bunzip2 -c linux-2.6.16.tar.bz2 | tar xvf -# --uncompress tar file-- | --then pass it to "tar"--# If "tar" has not been patched to handle "bunzip2",#+ this needs to be done in two discrete steps, using a pipe.# The purpose of the exercise is to unarchive "bzipped" kernel source. | Note that in this context the "-" is not itself a Bash operator, but rather an option recognized by certain UNIX utilities that write to stdout, such as tar, cat, etc. bash$ echo "whatever" | cat -whatever | Where a filename is expected, - redirects output to stdout (sometimes seen with tar cf), or accepts input from stdin, rather than from a file. This is a method of using a file-oriented utility as a filter in a pipe. bash$ fileUsage: file [-bciknvzL] [-f namefile] [-m magicfiles] file... | By itself on the command-line, file fails with an error message. Add a "-" for a more useful result. This causes the shell to await user input. bash$ file -abcstandard input: ASCII textbash$ file -#!/bin/bashstandard input: Bourne-Again shell script text executable | Now the command accepts input from stdin and analyzes it. The "-" can be used to pipe stdout to other commands. This permits such stunts as prepending lines to a file. Using diff to compare a file with a section of another: grep Linux file1 | diff file2 - Finally, a real-world example using - with tar. Example 3-4. Backup of all files changed in last day #!/bin/bash# Backs up all files in current directory modified within last 24 hours#+ in a "tarball" (tarred and gzipped file).BACKUPFILE=backup-$(date +%m-%d-%Y)# Embeds date in backup filename.# Thanks, Joshua Tschida, for the idea.archive=${1:-$BACKUPFILE}# If no backup-archive filename specified on command-line,#+ it will default to "backup-MM-DD-YYYY.tar.gz." tar cvf - `find . -mtime -1 -type f -print` > $archive.targzip $archive.tarecho "Directory $PWD backed up in archive file "$archive.tar.gz"." # Stephane Chazelas points out that the above code will fail#+ if there are too many files found#+ or if any filenames contain blank characters.# He suggests the following alternatives:# -------------------------------------------------------------------# find . -mtime -1 -type f -print0 | xargs -0 tar rvf "$archive.tar" # using the GNU version of "find".# find . -mtime -1 -type f -exec tar rvf "$archive.tar" '{}' ;# portable to other UNIX flavors, but much slower.# -------------------------------------------------------------------exit 0 | | Filenames beginning with "-" may cause problems when coupled with the "-" redirection operator. A script should check for this and add an appropriate prefix to such filenames, for example ./-FILENAME, $PWD/-FILENAME, or $PATHNAME/-FILENAME. If the value of a variable begins with a -, this may likewise createproblems. var="-n" echo $var# Has the effect of "echo -n", and outputs nothing. | | - -
| Do not confuse the "-" used in thissense with the "-" redirectionoperator just discussed. The interpretation of the"-" depends on the context in which itappears. | - -
- =
In a different context, the "=" is a string comparison operator. - +
In a different context, the + is a Regular Expression operator. - +
Certain commands and builtins use the + to enable certain options and the - to disable them. In parameter substitution, the + prefixes an alternate value that a variable expands to. - %
let "z = 5 % 3" echo $z # 2 | In a different context, the % is a pattern matching operator. - ~
- ~+
- ~-
- =~
- ^
- ^, ^^
- Control Characters
Control characters are not normally useful inside a script. Ctl-A Moves cursor to beginning of line of text (on the command-line). Ctl-B Backspace (nondestructive).
Ctl-C Break. Terminate a foreground job.
Ctl-D Log out from a shell (similar to exit). EOF (end-of-file). This also terminates input from stdin. When typing text on the console or in an xterm window, Ctl-D erases the character under the cursor. When there are no characters present, Ctl-D logs out of the session, as expected. In an xterm window, this has the effect of closing the window. Ctl-E Moves cursor to end of line of text (on the command-line). Ctl-F Moves cursor forward one character position (on the command-line).
Ctl-G BEL. On someold-time teletype terminals, this would actually ringa bell. In an xterm it mightbeep.
Ctl-H Rubout (destructive backspace). Erases characters the cursor backs over while backspacing. #!/bin/bash# Embedding Ctl-H in a string.a="^H^H" # Two Ctl-H's -- backspaces # ctl-V ctl-H, using vi/vimecho "abcdef" # abcdefechoecho -n "abcdef$a " # abcd f# Space at end ^ ^ Backspaces twice.echoecho -n "abcdef$a" # abcdef# No space at end ^ Doesn't backspace (why?). # Results may not be quite as expected.echo; echo# Constantin Hagemeier suggests trying:# a=$' |
|
| |