Deepbot Continue Randomuser Through Complex Command

16.2. Complex Commands

Commands for more advanced users

find

-exec COMMAND \;

Carries out COMMAND on each file that find matches. The command sequence terminates with ; (the ";" is escaped to make certain the shell passes it to find literally, without interpreting it as a special character).

                      bash$                                                                    find ~/ -name '*.txt'                                            /home/bozo/.kde/share/apps/karm/karmdata.txt  /home/bozo/misc/irmeyc.txt  /home/bozo/test-scripts/1.txt                    

If COMMAND contains {}, then find substitutes the full path name of the selected file for "{}".

find ~/ -name 'core*' -exec rm {} \; # Removes all core dump files from user's home directory.

find /home/bozo/projects -mtime -1 #                               ^   Note minus sign! #  Lists all files in /home/bozo/projects directory tree #+ that were modified within the last day (current_day - 1). # find /home/bozo/projects -mtime 1 #  Same as above, but modified *exactly* one day ago. # #  mtime = last modification time of the target file #  ctime = last status change time (via 'chmod' or otherwise) #  atime = last access time  DIR=/home/bozo/junk_files find "$DIR" -type f -atime +5 -exec rm {} \; #                          ^           ^^ #  Curly brackets are placeholder for the path name output by "find." # #  Deletes all files in "/home/bozo/junk_files" #+ that have not been accessed in *at least* 5 days (plus sign ... +5). # #  "-type filetype", where #  f = regular file #  d = directory #  l = symbolic link, etc. # #  (The 'find' manpage and info page have complete option listings.)

find /etc -exec grep '[0-9][0-9]*[.][0-9][0-9]*[.][0-9][0-9]*[.][0-9][0-9]*' {} \;  # Finds all IP addresses (xxx.xxx.xxx.xxx) in /etc directory files. # There a few extraneous hits. Can they be filtered out?  # Possibly by:  find /etc -type f -exec cat '{}' \; | tr -c '.[:digit:]' '\n' \ | grep '^[^.][^.]*\.[^.][^.]*\.[^.][^.]*\.[^.][^.]*$' # #  [:digit:] is one of the character classes #+ introduced with the POSIX 1003.2 standard.   # Thanks, St�phane Chazelas.                    

The -exec option to find should not be confused with the exec shell builtin.

Example 16-3. Badname, eliminate file names in current directory containing bad characters and whitespace.

#!/bin/bash # badname.sh # Delete filenames in current directory containing bad characters.  for filename in * do   badname=`echo "$filename" | sed -n /[\+\{\;\"\\\=\?~\(\)\<\>\&\*\|\$]/p` # badname=`echo "$filename" | sed -n '/[+{;"\=?~()<>&*|$]/p'`  also works. # Deletes files containing these nasties:     + { ; " \ = ? ~ ( ) < > & * | $ #   rm $badname 2>/dev/null #             ^^^^^^^^^^^ Error messages deep-sixed. done  # Now, take care of files containing all manner of whitespace. find . -name "* *" -exec rm -f {} \; # The path name of the file that _find_ finds replaces the "{}". # The '\' ensures that the ';' is interpreted literally, as end of command.  exit 0  #--------------------------------------------------------------------- # Commands below this line will not execute because of _exit_ command.  # An alternative to the above script: find . -name '*[+{;"\\=?~()<>&*|$ ]*' -maxdepth 0 \ -exec rm -f '{}' \; #  The "-maxdepth 0" option ensures that _find_ will not search #+ subdirectories below $PWD.  # (Thanks, S.C.)

Example 16-4. Deleting a file by its inode number

#!/bin/bash # idelete.sh: Deleting a file by its inode number.  #  This is useful when a filename starts with an illegal character, #+ such as ? or -.  ARGCOUNT=1                      # Filename arg must be passed to script. E_WRONGARGS=70 E_FILE_NOT_EXIST=71 E_CHANGED_MIND=72  if [ $# -ne "$ARGCOUNT" ] then   echo "Usage: `basename $0` filename"   exit $E_WRONGARGS fi    if [ ! -e "$1" ] then   echo "File \""$1"\" does not exist."   exit $E_FILE_NOT_EXIST fi    inum=`ls -i | grep "$1" | awk '{print $1}'` # inum = inode (index node) number of file # ----------------------------------------------------------------------- # Every file has an inode, a record that holds its physical address info. # -----------------------------------------------------------------------  echo; echo -n "Are you absolutely sure you want to delete \"$1\" (y/n)? " # The '-v' option to 'rm' also asks this. read answer case "$answer" in [nN]) echo "Changed your mind, huh?"       exit $E_CHANGED_MIND       ;; *)    echo "Deleting file \"$1\".";; esac  find . -inum $inum -exec rm {} \; #                           ^^ #        Curly brackets are placeholder #+       for text output by "find." echo "File "\"$1"\" deleted!"  exit 0

The find command also works without the -exec option.

#!/bin/bash #  Find suid root files. #  A strange suid file might indicate a security hole, #+ or even a system intrusion.  directory="/usr/sbin" # Might also try /sbin, /bin, /usr/bin, /usr/local/bin, etc. permissions="+4000"  # suid root (dangerous!)   for file in $( find "$directory" -perm "$permissions" ) do   ls -ltF --author "$file" done

See Example 16-30, Example 3-4, and Example 11-10 for scripts using find. Its manpage provides more detail on this complex and powerful command.

xargs

A filter for feeding arguments to a command, and also a tool for assembling the commands themselves. It breaks a data stream into small enough chunks for filters and commands to process. Consider it as a powerful replacement for backquotes. In situations where command substitution fails with a too many arguments error, substituting xargs often works. [1] Normally, xargs reads from stdin or from a pipe, but it can also be given the output of a file.

The default command for xargs is echo. This means that input piped to xargs may have linefeeds and other whitespace characters stripped out.

                      bash$                                                                    ls -l                                            total 0  -rw-rw-r--    1 bozo  bozo         0 Jan 29 23:58 file1  -rw-rw-r--    1 bozo  bozo         0 Jan 29 23:58 file2                      bash$                                                                    ls -l | xargs                                            total 0 -rw-rw-r-- 1 bozo bozo 0 Jan 29 23:58 file1 -rw-rw-r-- 1 bozo bozo 0 Jan...                      bash$                                                                    find ~/mail -type f | xargs grep "Linux"                                            ./misc:User-Agent: slrn/0.9.8.1 (Linux)  ./sent-mail-jul-2005: hosted by the Linux Documentation Project.  ./sent-mail-jul-2005: (Linux Documentation Project Site, rtf version)  ./sent-mail-jul-2005: Subject: Criticism of Bozo's Windows/Linux article  ./sent-mail-jul-2005: while mentioning that the Linux ext2/ext3 filesystem  . . .                    

ls | xargs -p -l gzip gzips every file in current directory, one at a time, prompting before each operation.

Note that xargs processes the arguments passed to it sequentially, one at a time. bash$ find /usr/bin | xargs file /usr/bin: directory /usr/bin/foomatic-ppd-options: perl script text executable . . .

An interesting xargs option is -n NN , which limits to NN the number of arguments passed. ls | xargs -n 8 echo lists the files in the current directory in 8 columns.

Another useful option is -0, in combination with find -print0 or grep -lZ . This allows handling arguments containing whitespace or quotes. find / -type f -print0 | xargs -0 grep -liwZ GUI | xargs -0 rm -f grep -rliwZ GUI / | xargs -0 rm -f Either of the above will remove any file containing "GUI". (Thanks, S.C.) Or: cat /proc/"$pid"/"$OPTION" | xargs -0 echo # Formats output: ^^^^^^^^^^^^^^^ # From Han Holl's fixup of "get-commandline.sh" #+ script in "/dev and /proc" chapter.

The -P option to xargs permits running processes in parallel. This speeds up execution in a machine with a multicore CPU. #!/bin/bash ls *gif | xargs -t -n1 -P2 gif2png # Converts all the gif images in current directory to png. # Options: # ======= # -t Print command to stderr. # -n1 At most 1 argument per command line. # -P2 Run up to 2 processes simultaneously. # Thank you, Roberto Polli, for the inspiration.

Example 16-5. Logfile: Using xargs to monitor system log

#!/bin/bash  # Generates a log file in current directory # from the tail end of /var/log/messages.  # Note: /var/log/messages must be world readable # if this script invoked by an ordinary user. #         #root chmod 644 /var/log/messages  LINES=5  ( date; uname -a ) >>logfile # Time and machine name echo ---------------------------------------------------------- >>logfile tail -n $LINES /var/log/messages | xargs | fmt -s >>logfile echo >>logfile echo >>logfile  exit 0  #  Note: #  ---- #  As Frank Wang points out, #+ unmatched quotes (either single or double quotes) in the source file #+ may give xargs indigestion. # #  He suggests the following substitution for line 15: #  tail -n $LINES /var/log/messages | tr -d "\"'" | xargs | fmt -s >>logfile    #  Exercise: #  -------- #  Modify this script to track changes in /var/log/messages at intervals #+ of 20 minutes. #  Hint: Use the "watch" command.                    

As in find, a curly bracket pair serves as a placeholder for replacement text.

Example 16-6. Copying files in current directory to another

#!/bin/bash # copydir.sh  #  Copy (verbose) all files in current directory ($PWD) #+ to directory specified on command-line.  E_NOARGS=85  if [ -z "$1" ]   # Exit if no argument given. then   echo "Usage: `basename $0` directory-to-copy-to"   exit $E_NOARGS fi    ls . | xargs -i -t cp ./{} $1 #            ^^ ^^      ^^ #  -t is "verbose" (output command-line to stderr) option. #  -i is "replace strings" option. #  {} is a placeholder for output text. #  This is similar to the use of a curly-bracket pair in "find." # #  List the files in current directory (ls .), #+ pass the output of "ls" as arguments to "xargs" (-i -t options), #+ then copy (cp) these arguments ({}) to new directory ($1).   # #  The net result is the exact equivalent of #+   cp * $1 #+ unless any of the filenames has embedded "whitespace" characters.  exit 0

Example 16-7. Killing processes by name

#!/bin/bash # kill-byname.sh: Killing processes by name. # Compare this script with kill-process.sh.  #  For instance, #+ try "./kill-byname.sh xterm" -- #+ and watch all the xterms on your desktop disappear.  #  Warning: #  ------- #  This is a fairly dangerous script. #  Running it carelessly (especially as root) #+ can cause data loss and other undesirable effects.  E_BADARGS=66  if test -z "$1"  # No command-line arg supplied? then   echo "Usage: `basename $0` Process(es)_to_kill"   exit $E_BADARGS fi   PROCESS_NAME="$1" ps ax | grep "$PROCESS_NAME" | awk '{print $1}' | xargs -i kill {} 2&>/dev/null #                                                       ^^      ^^  # --------------------------------------------------------------- # Notes: # -i is the "replace strings" option to xargs. # The curly brackets are the placeholder for the replacement. # 2&>/dev/null suppresses unwanted error messages. # # Can  grep "$PROCESS_NAME" be replaced by pidof "$PROCESS_NAME"? # ---------------------------------------------------------------  exit $?  #  The "killall" command has the same effect as this script, #+ but using it is not quite as educational.

Example 16-8. Word frequency analysis using xargs

#!/bin/bash # wf2.sh: Crude word frequency analysis on a text file.  # Uses 'xargs' to decompose lines of text into single words. # Compare this example to the "wf.sh" script later on.   # Check for input file on command-line. ARGS=1 E_BADARGS=85 E_NOFILE=86  if [ $# -ne "$ARGS" ] # Correct number of arguments passed to script? then   echo "Usage: `basename $0` filename"   exit $E_BADARGS fi  if [ ! -f "$1" ]       # Does file exist? then   echo "File \"$1\" does not exist."   exit $E_NOFILE fi    ##################################################### cat "$1" | xargs -n1 | \ #  List the file, one word per line.  tr A-Z a-z | \ #  Shift characters to lowercase. sed -e 's/\.//g'  -e 's/\,//g' -e 's/ /\ /g' | \ #  Filter out periods and commas, and #+ change space between words to linefeed, sort | uniq -c | sort -nr #  Finally remove duplicates, prefix occurrence count #+ and sort numerically. #####################################################  #  This does the same job as the "wf.sh" example, #+ but a bit more ponderously, and it runs more slowly (why?).  exit $?

expr

All-purpose expression evaluator: Concatenates and evaluates the arguments according to the operation given (arguments must be separated by spaces). Operations may be arithmetic, comparison, string, or logical.

expr 3 + 5

returns 8

expr 5 % 3

returns 2

expr 1 / 0

returns the error message, expr: division by zero

Illegal arithmetic operations not allowed.

expr 5 \* 3

returns 15

The multiplication operator must be escaped when used in an arithmetic expression with expr.

y=`expr $y + 1`

Increment a variable, with the same effect as let y=y+1 and y=$(($y+1)) . This is an example of arithmetic expansion.

z=`expr substr $string $position $length`

Extract substring of $length characters, starting at $position.

Example 16-9. Using expr

#!/bin/bash  # Demonstrating some of the uses of 'expr' # =======================================  echo  # Arithmetic Operators # ---------- ---------  echo "Arithmetic Operators" echo a=`expr 5 + 3` echo "5 + 3 = $a"  a=`expr $a + 1` echo echo "a + 1 = $a" echo "(incrementing a variable)"  a=`expr 5 % 3` # modulo echo echo "5 mod 3 = $a"  echo echo  # Logical Operators # ------- ---------  #  Returns 1 if true, 0 if false, #+ opposite of normal Bash convention.  echo "Logical Operators" echo  x=24 y=25 b=`expr $x = $y`         # Test equality. echo "b = $b"            # 0  ( $x -ne $y ) echo  a=3 b=`expr $a \> 10` echo 'b=`expr $a \> 10`, therefore...' echo "If a > 10, b = 0 (false)" echo "b = $b"            # 0  ( 3 ! -gt 10 ) echo  b=`expr $a \< 10` echo "If a < 10, b = 1 (true)" echo "b = $b"            # 1  ( 3 -lt 10 ) echo # Note escaping of operators.  b=`expr $a \<= 3` echo "If a <= 3, b = 1 (true)" echo "b = $b"            # 1  ( 3 -le 3 ) # There is also a "\>=" operator (greater than or equal to).   echo echo    # String Operators # ------ ---------  echo "String Operators" echo  a=1234zipper43231 echo "The string being operated upon is \"$a\"."  # length: length of string b=`expr length $a` echo "Length of \"$a\" is $b."  # index: position of first character in substring #        that matches a character in string b=`expr index $a 23` echo "Numerical position of first \"2\" in \"$a\" is \"$b\"."  # substr: extract substring, starting position & length specified b=`expr substr $a 2 6` echo "Substring of \"$a\", starting at position 2,\ and 6 chars long is \"$b\"."   #  The default behavior of the 'match' operations is to #+ search for the specified match at the BEGINNING of the string. # #       Using Regular Expressions ... b=`expr match "$a" '[0-9]*'`               #  Numerical count. echo Number of digits at the beginning of \"$a\" is $b. b=`expr match "$a" '\([0-9]*\)'`           #  Note that escaped parentheses #                   ==      ==             #+ trigger substring match. echo "The digits at the beginning of \"$a\" are \"$b\"."  echo  exit 0

The : (null) operator can substitute for match. For example, b=`expr $a : [0-9]*` is the exact equivalent of b=`expr match $a [0-9]*` in the above listing. #!/bin/bash echo echo "String operations using \"expr \$string : \" construct" echo "===================================================" echo a=1234zipper5FLIPPER43231 echo "The string being operated upon is \"`expr "$a" : '\(.*\)'`\"." # Escaped parentheses grouping operator. == == # *************************** #+ Escaped parentheses #+ match a substring # *************************** # If no escaped parentheses ... #+ then 'expr' converts the string operand to an integer. echo "Length of \"$a\" is `expr "$a" : '.*'`." # Length of string echo "Number of digits at the beginning of \"$a\" is `expr "$a" : '[0-9]*'`." # ------------------------------------------------------------------------- # echo echo "The digits at the beginning of \"$a\" are `expr "$a" : '\([0-9]*\)'`." # == == echo "The first 7 characters of \"$a\" are `expr "$a" : '\(.......\)'`." # ===== == == # Again, escaped parentheses force a substring match. # echo "The last 7 characters of \"$a\" are `expr "$a" : '.*\(.......\)'`." # ==== end of string operator ^^ # (In fact, means skip over one or more of any characters until specified #+ substring found.) echo exit 0

The above script illustrates how expr uses the escaped parentheses -- \( ... \) -- grouping operator in tandem with regular expression parsing to match a substring. Here is a another example, this time from "real life."

# Strip the whitespace from the beginning and end. LRFDATE=`expr "$LRFDATE" : '[[:space:]]*\(.*\)[[:space:]]*$'`  #  From Peter Knowles' "booklistgen.sh" script #+ for converting files to Sony Librie/PRS-50X format. #  (http://booklistgensh.peterknowles.com)

Perl, sed, and awk have far superior string parsing facilities. A short sed or awk "subroutine" within a script (see Section 36.2) is an attractive alternative to expr.

See Section 10.1 for more on using expr in string operations.

whitleythaid1981.blogspot.com

Source: https://tldp.org/LDP/abs/html/moreadv.html

Share this post