The first commands a novice learns

ls

The basic file "list" command. It is all too easy to underestimate the power of this humble command. For example, using the -R, recursive option, ls provides a tree-like listing of a directory structure. Other useful options are -S, sort listing by file size, -t, sort by file modification time, -v, sort by (numerical) version numbers embedded in the filenames, [1] -b, show escape characters, and -i, show file inodes (see Example 16-4).

bash$ ls -l-rw-rw-r-- 1 bozo bozo 0 Sep 14 18:44 chapter10.txt -rw-rw-r-- 1 bozo bozo 0 Sep 14 18:44 chapter11.txt -rw-rw-r-- 1 bozo bozo 0 Sep 14 18:44 chapter12.txt -rw-rw-r-- 1 bozo bozo 0 Sep 14 18:44 chapter1.txt -rw-rw-r-- 1 bozo bozo 0 Sep 14 18:44 chapter2.txt -rw-rw-r-- 1 bozo bozo 0 Sep 14 18:44 chapter3.txt -rw-rw-r-- 1 bozo bozo 0 Sep 14 18:49 Chapter_headings.txt -rw-rw-r-- 1 bozo bozo 0 Sep 14 18:49 Preface.txtbash$ ls -lv total 0 -rw-rw-r-- 1 bozo bozo 0 Sep 14 18:49 Chapter_headings.txt -rw-rw-r-- 1 bozo bozo 0 Sep 14 18:49 Preface.txt -rw-rw-r-- 1 bozo bozo 0 Sep 14 18:44 chapter1.txt -rw-rw-r-- 1 bozo bozo 0 Sep 14 18:44 chapter2.txt -rw-rw-r-- 1 bozo bozo 0 Sep 14 18:44 chapter3.txt -rw-rw-r-- 1 bozo bozo 0 Sep 14 18:44 chapter10.txt -rw-rw-r-- 1 bozo bozo 0 Sep 14 18:44 chapter11.txt -rw-rw-r-- 1 bozo bozo 0 Sep 14 18:44 chapter12.txt

The ls command returns a non-zero exit status when attempting to list a non-existent file. bash$ ls abcls: abc: No such file or directorybash$ echo $?2

Example 16-1. Using ls to create a table of contentsfor burning a CDR disk

#!/bin/bash# ex40.sh (burn-cd.sh)# Script to automate burning a CDR.SPEED=10 # May use higher speed if your hardware supports it.IMAGEFILE=cdimage.isoCONTENTSFILE=contents# DEVICE=/dev/cdrom For older versions of cdrecordDEVICE="1,0,0" DEFAULTDIR=/opt # This is the directory containing the data to be burned. # Make sure it exists. # Exercise: Add a test for this.# Uses Joerg Schilling's "cdrecord" package:# http://www.fokus.fhg.de/usr/schilling/cdrecord.html# If this script invoked as an ordinary user, may need to suid cdrecord#+ chmod u+s /usr/bin/cdrecord, as root.# Of course, this creates a security hole, though a relatively minor one.if [ -z "$1" ]then IMAGE_DIRECTORY=$DEFAULTDIR # Default directory, if not specified on command-line.else IMAGE_DIRECTORY=$1fi# Create a "table of contents" file.ls -lRF $IMAGE_DIRECTORY > $IMAGE_DIRECTORY/$CONTENTSFILE# The "l" option gives a "long" file listing.# The "R" option makes the listing recursive.# The "F" option marks the file types (directories get a trailing /).echo "Creating table of contents." # Create an image file preparatory to burning it onto the CDR.mkisofs -r -o $IMAGEFILE $IMAGE_DIRECTORYecho "Creating ISO9660 file system image ($IMAGEFILE)." # Burn the CDR.echo "Burning the disk." echo "Please be patient, this will take a while." wodim -v -isosize dev=$DEVICE $IMAGEFILE# In newer Linux distros, the "wodim" utility assumes the#+ functionality of "cdrecord." exitcode=$?echo "Exit code = $exitcode" exit $exitcode

cat, tac

cat, an acronym for concatenate, lists a file to stdout. When combined with redirection (> or >>), it is commonly used to concatenate files.

# Uses of 'cat'cat filename # Lists the file.cat file.1 file.2 file.3 > file.123 # Combines three files into one.

The -n option to cat inserts consecutive numbers before all lines of the target file(s). The -b option numbers only the non-blank lines. The -v option echoes nonprintable characters, using ^ notation. The -s option squeezes multiple consecutive blank lines into a single blank line.

See also Example 16-28 and Example 16-24.

In a pipe, it may be more efficient to redirect the stdin to a file, rather than to cat the file. cat filename | tr a-z A-Ztr a-z A-Z < filename # Same effect, but starts one less process, #+ and also dispenses with the pipe.

tac, is the inverse of cat, listing a file backwards from its end.

rev

reverses each line of a file, and outputs to stdout. This does not have the same effect as tac, as it preserves the order of the lines, but flips each one around (mirror image).

bash$ cat file1.txtThis is line 1. This is line 2.bash$ tac file1.txtThis is line 2. This is line 1.bash$ rev file1.txt.1 enil si sihT .2 enil si sihT

cp

This is the file copy command. cp file1 file2 copies file1 to file2, overwriting file2 if it already exists (see Example 16-6).

Particularly useful are the -a archive flag (for copying an entire directory tree), the -u update flag (which prevents overwriting identically-named newer files), and the -r and -R recursive flags. cp -u source_dir/* dest_dir# "Synchronize" dest_dir to source_dir#+ by copying over all newer and not previously existing files.

mv

This is the file move command. It is equivalent to a combination of cp and rm. It may be used to move multiple files to a directory, or even to rename a directory. For some examples of using mv in a script, see Example 10-11 and Example A-2.

When used in a non-interactive script, mv takes the -f (force) option to bypass user input. When a directory is moved to a preexisting directory, it becomes a subdirectory of the destination directory. bash$ mv source_directory target_directorybash$ ls -lF target_directorytotal 1 drwxrwxr-x 2 bozo bozo 1024 May 28 19:20 source_directory/

rm

Delete (remove) a file or files. The -f option forces removal of even readonly files, and is useful for bypassing user input in a script.

The rm command will, byitself, fail to remove filenames beginning witha dash. Why? Because rmsees a dash-prefixed filename as anoption. bash$ rm -badnamerm: invalid option -- b Try `rm --help' for more information. One clever workaround is to precedethe filename with a " -- " (theend-of-options flag). bash$ rm -- -badname Another method to is to preface the filename to be removed with a dot-slash . bash$ rm ./-badname

When used with the recursive flag -r, this command removes files all the way down the directory tree from the current directory. A careless rm -rf * can wipe out a big chunk of a directory structure.

rmdir

Remove directory. The directory must be empty of all files -- including "invisible" dotfiles[2] -- for this command to succeed.

mkdir

Make directory, creates a new directory. For example, mkdir -p project/programs/December creates the named directory. The -p option automatically creates any necessary parent directories.

chmod

Changes the attributes of an existing file or directory (see Example 15-14).

chmod +x filename# Makes "filename" executable for all users.chmod u+s filename# Sets "suid" bit on "filename" permissions.# An ordinary user may execute "filename" with same privileges as the file's owner.# (This does not apply to shell scripts.)

chmod 644 filename# Makes "filename" readable/writable to owner, readable to others#+ (octal mode).chmod 444 filename# Makes "filename" read-only for all.# Modifying the file (for example, with a text editor)#+ not allowed for a user who does not own the file (except for root),#+ and even the file owner must force a file-save#+ if she modifies the file.# Same restrictions apply for deleting the file.

chmod 1777 directory-name# Gives everyone read, write, and execute permission in directory,#+ however also sets the "sticky bit".# This means that only the owner of the directory,#+ owner of the file, and, of course, root#+ can delete any particular file in that directory.chmod 111 directory-name# Gives everyone execute-only permission in a directory.# This means that you can execute and READ the files in that directory#+ (execute permission necessarily includes read permission#+ because you can't execute a file without being able to read it).# But you can't list the files or search for them with the "find" command.# These restrictions do not apply to root.chmod 000 directory-name# No permissions at all for that directory.# Can't read, write, or execute files in it.# Can't even list files in it or "cd" to it.# But, you can rename (mv) the directory#+ or delete it (rmdir) if it is empty.# You can even symlink to files in the directory,#+ but you can't read, write, or execute the symlinks.# These restrictions do not apply to root.

chattr

Change file attributes. This is analogous to chmod above, but with different options and a different invocation syntax, and it works only on ext2/ext3 filesystems.

One particularly interesting chattr option is i. A chattr +i filename marks the file as immutable. The file cannot be modified, linked to, or deleted, not even by root. This file attribute can be set or removed only by root. In a similar fashion, the a option marks the file as append only.

root# chattr +i file1.txtroot# rm file1.txtrm: remove write-protected regular file `file1.txt'? y rm: cannot remove `file1.txt': Operation not permitted

If a file has the s (secure) attribute set, then when it is deleted its block is overwritten with binary zeroes.[3]

If a file has the u (undelete) attribute set, then when it is deleted, its contents can still be retrieved (undeleted).

If a file has the c (compress) attribute set, then it will automatically be compressed on writes to disk, and uncompressed on reads.

The file attributes set with chattr do not show in a file listing (ls -l).

ln

Creates links to pre-existings files. A "link" is a reference to a file, an alternate name for it. The ln command permits referencing the linked file by more than one name and is a superior alternative to aliasing (see Example 4-6).

The ln creates only a reference, a pointer to the file only a few bytes in size.

The ln command is most often used with the -s, symbolic or "soft" link flag. Advantages of using the -s flag are that it permits linking across file systems or to directories.

The syntax of the command is a bit tricky. For example: ln -s oldfile newfile links the previously existing oldfile to the newly created link, newfile.

If a file named newfile has previously existed, an error message will result.

Which type of link to use? As John Macdonald explains it: Both of these [types of links] provide a certain measure of dual reference-- if you edit the contents of the file using any name,your changes will affect both the original name and eithera hard or soft new name. The differences between themoccurs when you work at a higher level. The advantage ofa hard link is that the new name is totally independentof the old name -- if you remove or rename the old name,that does not affect the hard link, which continuesto point to the data while it would leave a soft linkhanging pointing to the old name which is no longerthere. The advantage of a soft link is that it can referto a different file system (since it is just a referenceto a file name, not to actual data). And, unlike a hardlink, a symbolic link can refer to a directory.

Links give the ability to invoke a script (or any other type of executable) with multiple names, and having that script behave according to how it was invoked.

Example 16-2. Hello or Good-bye

#!/bin/bash# hello.sh: Saying "hello" or "goodbye" #+ depending on how script is invoked.# Make a link in current working directory ($PWD) to this script:# ln -s hello.sh goodbye# Now, try invoking this script both ways:# ./hello.sh# ./goodbyeHELLO_CALL=65GOODBYE_CALL=66if [ $0 = "./goodbye" ]then echo "Good-bye!" # Some other goodbye-type commands, as appropriate. exit $GOODBYE_CALLfiecho "Hello!" # Some other hello-type commands, as appropriate.exit $HELLO_CALL

man, info

These commands access the manual and information pages on system commands and installed utilities. When available, the info pages usually contain more detailed descriptions than do the man pages.

There have been various attempts at "automating" the writing of man pages. For a script that makes a tentative first step in that direction, see Example A-39.

Commands for more advanced users

find

-exec COMMAND ;

Carries out COMMAND on each file that find matches. The command sequence terminates with ; (the ";" is escaped to make certain the shell passes it to find literally, without interpreting it as a special character).

bash$ find ~/ -name '*.txt'/home/bozo/.kde/share/apps/karm/karmdata.txt /home/bozo/misc/irmeyc.txt /home/bozo/test-scripts/1.txt

If COMMAND contains {}, then find substitutes the full path name of the selected file for "{}".

find ~/ -name 'core*' -exec rm {} ;# Removes all core dump files from user's home directory.

find /home/bozo/projects -mtime -1# ^ Note minus sign!# Lists all files in /home/bozo/projects directory tree#+ that were modified within the last day (current_day - 1).#find /home/bozo/projects -mtime 1# Same as above, but modified *exactly* one day ago.## mtime = last modification time of the target file# ctime = last status change time (via 'chmod' or otherwise)# atime = last access timeDIR=/home/bozo/junk_filesfind "$DIR" -type f -atime +5 -exec rm {} ;# ^ ^^# Curly brackets are placeholder for the path name output by "find." ## Deletes all files in "/home/bozo/junk_files" #+ that have not been accessed in *at least* 5 days (plus sign ... +5).## "-type filetype", where# f = regular file# d = directory# l = symbolic link, etc.## (The 'find' manpage and info page have complete option listings.)

find /etc -exec grep '[0-9][0-9]*[.][0-9][0-9]*[.][0-9][0-9]*[.][0-9][0-9]*' {} ;# Finds all IP addresses (xxx.xxx.xxx.xxx) in /etc directory files.# There a few extraneous hits. Can they be filtered out?# Possibly by:find /etc -type f -exec cat '{}' ; | tr -c '.[:digit:]' '' | grep '^[^.][^.]*.[^.][^.]*.[^.][^.]*.[^.][^.]*$'## [:digit:] is one of the character classes#+ introduced with the POSIX 1003.2 standard. # Thanks, St�phane Chazelas.

The -exec option to find should not be confused with the exec shell builtin.

Example 16-3. Badname, eliminate file namesin current directory containing bad characters and whitespace.

#!/bin/bash# badname.sh# Delete filenames in current directory containing bad characters.for filename in *do badname=`echo "$filename" | sed -n /[+{;"=?~()<>&*|$]/p`# badname=`echo "$filename" | sed -n '/[+{;"=?~()<>&*|$]/p'` also works.# Deletes files containing these nasties: + { ; " = ? ~ ( ) < > & * | $# rm $badname 2>/dev/null# ^^^^^^^^^^^ Error messages deep-sixed.done# Now, take care of files containing all manner of whitespace.find . -name "* *" -exec rm -f {} ;# The path name of the file that _find_ finds replaces the "{}".# The '' ensures that the '' is interpreted literally, as end of command.exit 0#---------------------------------------------------------------------# Commands below this line will not execute because of _exit_ command.# An alternative to the above script:find . -name '*[+{;"=?~()<>&*|$ ]*' -maxdepth 0 -exec rm -f '{}' ;# The "-maxdepth 0" option ensures that _find_ will not search#+ subdirectories below $PWD.# (Thanks, S.C.)

Example 16-4. Deleting a file by its inode number

#!/bin/bash# idelete.sh: Deleting a file by its inode number.# This is useful when a filename starts with an illegal character,#+ such as ? or -.ARGCOUNT=1 # Filename arg must be passed to script.E_WRONGARGS=70E_FILE_NOT_EXIST=71E_CHANGED_MIND=72if [ $# -ne "$ARGCOUNT" ]then echo "Usage: `basename $0` filename" exit $E_WRONGARGSfi if [ ! -e "$1" ]then echo "File ""$1"" does not exist." exit $E_FILE_NOT_EXISTfi inum=`ls -i | grep "$1" | awk '{print $1}'`# inum = inode (index node) number of file# -----------------------------------------------------------------------# Every file has an inode, a record that holds its physical address info.# -----------------------------------------------------------------------echo; echo -n "Are you absolutely sure you want to delete "$1" (y/n)? " # The '-v' option to 'rm' also asks this.read answercase "$answer" in[nN]) echo "Changed your mind, huh?" exit $E_CHANGED_MIND ;*) echo "Deleting file "$1".";esacfind . -inum $inum -exec rm {} ;# ^^# Curly brackets are placeholder#+ for text output by "find." echo "File ""$1"" deleted!" exit 0

The find command also works without the -exec option.

#!/bin/bash# Find suid root files.# A strange suid file might indicate a security hole,#+ or even a system intrusion.directory="/usr/sbin" # Might also try /sbin, /bin, /usr/bin, /usr/local/bin, etc.permissions="+4000" # suid root (dangerous!)for file in $( find "$directory" -perm "$permissions" )do ls -ltF --author "$file" done

See Example 16-30, Example 3-4, and Example 11-9 for scripts using find. Its manpage provides more detail on this complex and powerful command.

xargs

A filter for feeding arguments to a command, and also a tool for assembling the commands themselves. It breaks a data stream into small enough chunks for filters and commands to process. Consider it as a powerful replacement for backquotes. In situations where command substitution fails with a too many arguments error, substituting xargs often works. [1] Normally, xargs reads from stdin or from a pipe, but it can also be given the output of a file.

The default command for xargs is echo. This means that input piped to xargs may have linefeeds and other whitespace characters stripped out.

bash$ ls -ltotal 0 -rw-rw-r-- 1 bozo bozo 0 Jan 29 23:58 file1 -rw-rw-r-- 1 bozo bozo 0 Jan 29 23:58 file2bash$ ls -l | xargstotal 0 -rw-rw-r-- 1 bozo bozo 0 Jan 29 23:58 file1 -rw-rw-r-- 1 bozo bozo 0 Jan...bash$ find ~/mail -type f | xargs grep "Linux"./misc:User-Agent: slrn/0.9.8.1 (Linux) ./sent-mail-jul-2005: hosted by the Linux Documentation Project. ./sent-mail-jul-2005: (Linux Documentation Project Site, rtf version) ./sent-mail-jul-2005: Subject: Criticism of Bozo's Windows/Linux article ./sent-mail-jul-2005: while mentioning that the Linux ext2/ext3 filesystem . . .

ls | xargs -p -l gzip gzips every file in current directory, one at a time, prompting before each operation.

Note that xargs processes the arguments passed to it sequentially, one at a time. bash$ find /usr/bin | xargs file/usr/bin: directory /usr/bin/foomatic-ppd-options: perl script text executable . . .

An interesting xargs option is -n NN, which limits to NN the number of arguments passed. ls | xargs -n 8 echo lists the files in the current directory in 8 columns.

Another useful option is -0, in combination with find -print0 or grep -lZ. This allows handling arguments containing whitespace or quotes. find / -type f -print0 | xargs -0 grep -liwZ GUI | xargs -0 rm -f grep -rliwZ GUI / | xargs -0 rm -f Either of the above will remove any file containing "GUI". (Thanks, S.C.) Or: cat /proc/"$pid"/"$OPTION" | xargs -0 echo# Formats output: ^^^^^^^^^^^^^^^# From Han Holl's fixup of "get-commandline.sh" #+ script in "/dev and /proc" chapter.

The -P option to xargs permits running processes in parallel. This speeds up execution in a machine with a multicore CPU. #!/bin/bashls *gif | xargs -t -n1 -P2 gif2png# Converts all the gif images in current directory to png.# Options:# =======# -t Print command to stderr.# -n1 At most 1 argument per command line.# -P2 Run up to 2 processes simultaneously.# Thank you, Roberto Polli, for the inspiration.

Example 16-5. Logfile: Using xargs to monitor system log

#!/bin/bash# Generates a log file in current directory# from the tail end of /var/log/messages.# Note: /var/log/messages must be world readable# if this script invoked by an ordinary user.# #root chmod 644 /var/log/messagesLINES=5( date; uname -a ) >>logfile# Time and machine nameecho ---------------------------------------------------------- >>logfiletail -n $LINES /var/log/messages | xargs | fmt -s >>logfileecho >>logfileecho >>logfileexit 0# Note:# ----# As Frank Wang points out,#+ unmatched quotes (either single or double quotes) in the source file#+ may give xargs indigestion.## He suggests the following substitution for line 15:# tail -n $LINES /var/log/messages | tr -d ""'" | xargs | fmt -s >>logfile# Exercise:# --------# Modify this script to track changes in /var/log/messages at intervals#+ of 20 minutes.# Hint: Use the "watch" command.

As in find, a curly bracket pair serves as a placeholder for replacement text.

Example 16-6. Copying files in current directory to another

#!/bin/bash# copydir.sh# Copy (verbose) all files in current directory ($PWD)#+ to directory specified on command-line.E_NOARGS=85if [ -z "$1" ] # Exit if no argument given.then echo "Usage: `basename $0` directory-to-copy-to" exit $E_NOARGSfi ls . | xargs -i -t cp ./{} $1# ^^ ^^ ^^# -t is "verbose" (output command-line to stderr) option.# -i is "replace strings" option.# {} is a placeholder for output text.# This is similar to the use of a curly-bracket pair in "find." ## List the files in current directory (ls .),#+ pass the output of "ls" as arguments to "xargs" (-i -t options),#+ then copy (cp) these arguments ({}) to new directory ($1). ## The net result is the exact equivalent of#+ cp * $1#+ unless any of the filenames has embedded "whitespace" characters.exit 0

Example 16-7. Killing processes by name

#!/bin/bash# kill-byname.sh: Killing processes by name.# Compare this script with kill-process.sh.# For instance,#+ try "./kill-byname.sh xterm" --#+ and watch all the xterms on your desktop disappear.# Warning:# -------# This is a fairly dangerous script.# Running it carelessly (especially as root)#+ can cause data loss and other undesirable effects.E_BADARGS=66if test -z "$1" # No command-line arg supplied?then echo "Usage: `basename $0` Process(es)_to_kill" exit $E_BADARGSfiPROCESS_NAME="$1" ps ax | grep "$PROCESS_NAME" | awk '{print $1}' | xargs -i kill {} 2&>/dev/null# ^^ ^^# ---------------------------------------------------------------# Notes:# -i is the "replace strings" option to xargs.# The curly brackets are the placeholder for the replacement.# 2&>/dev/null suppresses unwanted error messages.## Can grep "$PROCESS_NAME" be replaced by pidof "$PROCESS_NAME"?# ---------------------------------------------------------------exit $?# The "killall" command has the same effect as this script,#+ but using it is not quite as educational.

Example 16-8. Word frequency analysis using xargs

#!/bin/bash# wf2.sh: Crude word frequency analysis on a text file.# Uses 'xargs' to decompose lines of text into single words.# Compare this example to the "wf.sh" script later on.# Check for input file on command-line.ARGS=1E_BADARGS=85E_NOFILE=86if [ $# -ne "$ARGS" ]# Correct number of arguments passed to script?then echo "Usage: `basename $0` filename" exit $E_BADARGSfiif [ ! -f "$1" ] # Does file exist?then echo "File "$1" does not exist." exit $E_NOFILEfi#####################################################cat "$1" | xargs -n1 | # List the file, one word per line. tr A-Z a-z | # Shift characters to lowercase.sed -e 's/.//g' -e 's/,//g' -e 's/ //g' | # Filter out periods and commas, and#+ change space between words to linefeed,sort | uniq -c | sort -nr# Finally remove duplicates, prefix occurrence count#+ and sort numerically.###################################################### This does the same job as the "wf.sh" example,#+ but a bit more ponderously, and it runs more slowly (why?).exit $?

expr

All-purpose expression evaluator: Concatenates and evaluates the arguments according to the operation given (arguments must be separated by spaces). Operations may be arithmetic, comparison, string, or logical.

expr 3 + 5

returns 8

expr 5 % 3

returns 2

expr 1 / 0

returns the error message, expr: division by zero

Illegal arithmetic operations not allowed.

expr 5 * 3

returns 15

The multiplication operator must be escaped when used in an arithmetic expression with expr.

y=`expr $y + 1`

Increment a variable, with the same effect as let y=y+1 and y=$(($y+1)). This is an example of arithmetic expansion.

z=`expr substr$string $position $length`

Extract substring of $length characters, starting at $position.

Example 16-9. Using expr

#!/bin/bash# Demonstrating some of the uses of 'expr'# =======================================echo# Arithmetic Operators# ---------- ---------echo "Arithmetic Operators" echoa=`expr 5 + 3`echo "5 + 3 = $a" a=`expr $a + 1`echoecho "a + 1 = $a" echo "(incrementing a variable)" a=`expr 5 % 3`# moduloechoecho "5 mod 3 = $a" echoecho# Logical Operators# ------- ---------# Returns 1 if true, 0 if false,#+ opposite of normal Bash convention.echo "Logical Operators" echox=24y=25b=`expr $x = $y` # Test equality.echo "b = $b" # 0 ( $x -ne $y )echoa=3b=`expr $a > 10`echo 'b=`expr $a > 10`, therefore...'echo "If a > 10, b = 0 (false)" echo "b = $b" # 0 ( 3 ! -gt 10 )echob=`expr $a < 10`echo "If a < 10, b = 1 (true)" echo "b = $b" # 1 ( 3 -lt 10 )echo# Note escaping of operators.b=`expr $a <= 3`echo "If a <= 3, b = 1 (true)" echo "b = $b" # 1 ( 3 -le 3 )# There is also a ">=" operator (greater than or equal to).echoecho# String Operators# ------ ---------echo "String Operators" echoa=1234zipper43231echo "The string being operated upon is "$a"." # length: length of stringb=`expr length $a`echo "Length of "$a" is $b." # index: position of first character in substring# that matches a character in stringb=`expr index $a 23`echo "Numerical position of first "2" in "$a" is "$b"." # substr: extract substring, starting position & length specifiedb=`expr substr $a 2 6`echo "Substring of "$a", starting at position 2,and 6 chars long is "$b"." # The default behavior of the 'match' operations is to#+ search for the specified match at the BEGINNING of the string.## Using Regular Expressions ...b=`expr match "$a" '[0-9]*'` # Numerical count.echo Number of digits at the beginning of "$a" is $b.b=`expr match "$a" '([0-9]*)'` # Note that escaped parentheses# == == #+ trigger substring match.echo "The digits at the beginning of "$a" are "$b"." echoexit 0

The : (null) operator can substitute for match. For example, b=`expr $a : [0-9]*` is the exact equivalent of b=`expr match $a [0-9]*` in the above listing. #!/bin/bashechoecho "String operations using "expr $string : " construct" echo "===================================================" echoa=1234zipper5FLIPPER43231echo "The string being operated upon is "`expr "$a" : '(.*)'`"." # Escaped parentheses grouping operator. == ==# ***************************#+ Escaped parentheses#+ match a substring# ***************************# If no escaped parentheses ...#+ then 'expr' converts the string operand to an integer.echo "Length of "$a" is `expr "$a" : '.*'`." # Length of stringecho "Number of digits at the beginning of "$a" is `expr "$a" : '[0-9]*'`." # ------------------------------------------------------------------------- #echoecho "The digits at the beginning of "$a" are `expr "$a" : '([0-9]*)'`." # == ==echo "The first 7 characters of "$a" are `expr "$a" : '(.......)'`." # ===== == ==# Again, escaped parentheses force a substring match.#echo "The last 7 characters of "$a" are `expr "$a" : '.*(.......)'`." # ==== end of string operator ^^# (In fact, means skip over one or more of any characters until specified#+ substring found.)echoexit 0

Time/date and timing

date

Simply invoked, date prints the date and time to stdout. Where this command gets interesting is in its formatting and parsing options.

Example 16-10. Using date

#!/bin/bash# Exercising the 'date' commandecho "The number of days since the year's beginning is `date +%j`." # Needs a leading '+' to invoke formatting.# %j gives day of year.echo "The number of seconds elapsed since 01/01/1970 is `date +%s`." # %s yields number of seconds since "UNIX epoch" began,#+ but how is this useful?prefix=tempsuffix=$(date +%s) # The "+%s" option to 'date' is GNU-specific.filename=$prefix.$suffixecho "Temporary filename = $filename" # It's great for creating "unique and random" temp filenames,#+ even better than using $$.# Read the 'date' man page for more formatting options.exit 0

The -u option gives the UTC (Universal Coordinated Time).

bash$ dateFri Mar 29 21:07:39 MST 2002bash$ date -uSat Mar 30 04:07:42 UTC 2002

This option facilitates calculating the time between different dates.

Example 16-11. Date calculations

#!/bin/bash# date-calc.sh# Author: Nathan Coulter# Used in ABS Guide with permission (thanks!).MPHR=60 # Minutes per hour.HPD=24 # Hours per day.diff () { printf '%s' $(( $(date -u -d"$TARGET" +%s) - $(date -u -d"$CURRENT" +%s)))# %d = day of month.}CURRENT=$(date -u -d '2007-09-01 17:30:24' '+%F %T.%N %Z')TARGET=$(date -u -d'2007-12-25 12:30:00' '+%F %T.%N %Z')# %F = full date, %T = %H:%M:%S, %N = nanoseconds, %Z = time zone.printf 'In 2007, %s ' "$(date -d"$CURRENT + $(( $(diff) /$MPHR /$MPHR /$HPD / 2 )) days" '+%d %B')" # %B = name of month ^ halfwayprintf 'was halfway between %s ' "$(date -d"$CURRENT" '+%d %B')" printf 'and %s' "$(date -d"$TARGET" '+%d %B')" printf 'On %s at %s, there were' $(date -u -d"$CURRENT" +%F) $(date -u -d"$CURRENT" +%T)DAYS=$(( $(diff) / $MPHR / $MPHR / $HPD ))CURRENT=$(date -d"$CURRENT +$DAYS days" '+%F %T.%N %Z')HOURS=$(( $(diff) / $MPHR / $MPHR ))CURRENT=$(date -d"$CURRENT +$HOURS hours" '+%F %T.%N %Z')MINUTES=$(( $(diff) / $MPHR ))CURRENT=$(date -d"$CURRENT +$MINUTES minutes" '+%F %T.%N %Z')printf '%s days, %s hours, ' "$DAYS" "$HOURS" printf '%s minutes, and %s seconds ' "$MINUTES" "$(diff)" printf 'until Christmas Dinner!'# Exercise:# --------# Rewrite the diff () function to accept passed parameters,#+ rather than using global variables.

The date command has quite anumber of output options. Forexample %N gives the nanosecond portionof the current time. One interesting use for this is togenerate random integers.

date +%N | sed -e 's/000$//' -e 's/^0//' ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^# Strip off leading and trailing zeroes, if present.# Length of generated integer depends on#+ how many zeroes stripped off.# 115281032# 63408725# 394504284

There are many more options (try man date).

date +%j# Echoes day of the year (days elapsed since January 1).date +%k%M# Echoes hour and minute in 24-hour format, as a single digit string.# The 'TZ' parameter permits overriding the default time zone.date # Mon Mar 28 21:42:16 MST 2005TZ=EST date # Mon Mar 28 23:42:16 EST 2005# Thanks, Frank Kannemann and Pete Sjoberg, for the tip.SixDaysAgo=$(date --date='6 days ago')OneMonthAgo=$(date --date='1 month ago') # Four weeks back (not a month!)OneYearAgo=$(date --date='1 year ago')

See also Example 3-4 and Example A-43.

zdump

Time zone dump: echoes the time in a specified time zone.

bash$ zdump ESTEST Tue Sep 18 22:09:22 2001 EST

time

Outputs verbose timing statistics for executing a command.

time ls -l / gives something like this:

real 0m0.067s user 0m0.004s sys 0m0.005s

See also the very similar times command in the previous section.

As of version 2.0 of Bash, time became a shell reserved word, with slightly altered behavior in a pipeline.

touch

Utility for updating access/modification times of a file to current system time or other specified time, but also useful for creating a new file. The command touch zzz will create a new file of zero length, named zzz, assuming that zzz did not previously exist. Time-stamping empty files in this way is useful for storing date information, for example in keeping track of modification times on a project.

The touch command is equivalent to : >> newfile or >> newfile (for ordinary files).

Before doing a cp -u (copy/update), use touch to update the time stamp of files you don't wish overwritten. As an example, if the directory /home/bozo/tax_audit contains the files spreadsheet-051606.data, spreadsheet-051706.data, and spreadsheet-051806.data, then doing a touch spreadsheet*.data will protect these files from being overwritten by files with the same names during a cp -u /home/bozo/financial_info/spreadsheet*data /home/bozo/tax_audit.

at

The at job control command executes a given set of commands at a specified time. Superficially, it resembles cron, however, at is chiefly useful for one-time execution of a command set.

at 2pm January 15 prompts for a set of commands to execute at that time. These commands should be shell-script compatible, since, for all practical purposes, the user is typing in an executable shell script a line at a time.Input terminates with a Ctl-D.

Using either the -f option or input redirection (<), at reads a command list from a file. This file is an executable shell script, though it should, of course, be non-interactive. Particularly clever is including the run-parts command in the file to execute a different set of scripts.

bash$ at 2:30 am Friday < at-jobs.listjob 2 at 2000-10-27 02:30

batch

The batch job control command is similar to at, but it runs a command list when the system load drops below .8. Like at, it can read commands from a file with the -f option.

The concept of batch processing dates back to the era of mainframe computers. It means running a set of commands without user intervention.

cal

Prints a neatly formatted monthly calendar to stdout. Will do current year or a large range of past and future years.

sleep

This is the shell equivalent of a wait loop. It pauses for a specified number of seconds, doing nothing. It can be useful for timing or in processes running in the background, checking for a specific event every so often (polling), as in Example 32-6.

sleep 3 # Pauses 3 seconds.

The sleep command defaults to seconds, but minute, hours, or days may also be specified. sleep 3 h # Pauses 3 hours!

The watch command may be a better choice than sleep for running commands at timed intervals.

usleep

Microsleep (the u may be read as the Greek mu, or micro- prefix). This is the same as sleep, above, but "sleeps" in microsecond intervals. It can be used for fine-grained timing, or for polling an ongoing process at very frequent intervals.

usleep 30 # Pauses 30 microseconds.

This command is part of the Red Hat initscripts / rc-scripts package.

The usleep command does not provide particularly accurate timing, and is therefore unsuitable for critical timing loops.

hwclock, clock

The hwclock command accesses or adjusts the machine's hardware clock. Some options require root privileges. The /etc/rc.d/rc.sysinit startup file uses hwclock to set the system time from the hardware clock at bootup.

The clock command is a synonym for hwclock.