Social Engineering

Contents:
Cooperating with Command Interpreters
Cooperating with Other Processes
Cooperating with Strangers
Cooperating with Other Languages

Languages have different personalities. You can classify computer languages by how introverted or extroverted they are; for instance, Icon and Lisp are stay-at-home languages, while Tcl and the various shells are party animals. Self-sufficient languages prefer to compete with other languages, while social languages prefer to cooperate with other languages. As usual, Perl tries to do both.

So this chapter is about relationships. Until now we've looked inward at the competitive nature of Perl, but now we need to look outward and see the cooperative nature of Perl. If we really mean what we say about Perl being a glue language, then we can't just talk about glue; we have to talk about the various kinds of things you can glue together. A glob of glue by itself isn't very interesting.

Perl doesn't just glue together other computer languages. It also glues together command line interpreters, operating systems, processes, machines, devices, networks, databases, institutions, cultures, Web pages, GUIs, peers, servers, and clients, not to mention people like system administrators, users, and of course, hackers, both naughty and nice. In fact, Perl is rather competitive about being cooperative.

So this chapter is about Perl's relationship with everything in the world. Obviously, we can't talk about everything in the world, but we'll try.

Cooperating with Command Interpreters

It is fortunate that Perl grew up in the UNIX world - that means its invocation syntax works pretty well under the command interpreters of other operating systems too. Most command interpreters know how to deal with a list of words as arguments, and don't care if an argument starts with a minus sign. There are, of course, some sticky spots where you'll get fouled up if you move from one system to another. You can't use single quotes under MS-DOS as you do under UNIX, for instance. And on systems like VMS, some wrapper code has to jump through hoops to emulate UNIX I/O redirection. Once you get past those issues, however, Perl treats its switches and arguments much the same on any operating system.

Even when you don't have a command interpreter, per se, it's easy to execute a Perl script from another program, such as the inet daemon or a CGI server. Not only can such a server pass arguments in the ordinary way, but it can also pass in information via environment variables and (under UNIX at least) inherited file descriptors. Even more exotic argument-passing mechanisms may be encapsulated in a module that can be brought into the Perl script via a simple use directive.

Command Processing

Perl parses command-line switches in the standard fashion.[] That is, it expects any switches (words beginning with a minus) to come first on the command line. After that comes the name of the script (usually), followed by any additional arguments (often filenames) to be passed into the script. Some of these additional arguments may be switches, but if so, they must be processed by the script, since Perl gives up parsing switches as soon as it sees a non-switch, or the special "--" switch that terminates switch processing.

[1] Presuming you agree that UNIX is both standard and fashionable.

Perl gives you some flexibility in how you supply your program. For small, quick-and-dirty jobs, you can program Perl entirely from the command line. For larger, more permanent jobs, you can supply a Perl script as a separate file. Perl looks for the script to be specified in one of three ways:

1. Specified line by line via -e switches on the command line.
2. Contained in the file specified by the first filename on the command line. (Note that systems supporting the #! shebang notation invoke interpreters this way on your behalf.)
3. Passed in implicitly via standard input. This only works if there are no filename arguments; to pass arguments to a standard-input script you must explicitly specify a "-" for the script name. For example, under UNIX:

   echo "print 'Hello, world'" | perl -
   

   With methods 2 and 3, Perl starts parsing the input file from the beginning, unless you've specified a -x switch, in which case it scans for the first line starting with #! and containing the word "perl", and starts there instead. This is useful for running a script embedded in a larger message. (In this case you might indicate the end of the script using the __END__ token.)

Whether or not you use -x, the #! line is always examined for switches as the line is being parsed. Thus, if you're on a machine that only allows one argument with the #! line, or worse, doesn't even recognize the #! line as special, you still can get consistent switch behavior regardless of how Perl was invoked, even if -x was used to find the beginning of the script.

WARNING: Because many versions of UNIX silently chop off kernel interpretation of the #! line after 32 characters, some switches may be passed in on the command line, and some may not; you could even get a "-" without its letter, if you're not careful. You probably want to make sure that all your switches fall either before or after that 32-character boundary. Most switches don't actually care if they're processed redundantly, but getting a "-" instead of a complete switch could cause Perl to try to execute standard input instead of your script. And a partial -I switch could also cause odd results. Of course, if you're not on a UNIX system, you're guaranteed not to have this problem.

Parsing of the switches on the #! line starts wherever "perl" is mentioned in the line. The sequences "-*" and "- " are specifically ignored for the benefit of emacs users, so that, if you're so inclined, you can say:

#!/bin/sh -- # -*- perl -*- -p eval 'exec perl -S $0 ${1+"$@"}' if 0;

and Perl will see only the -p switch. The fancy "-*- perl -*-" gizmo tells emacs to start up in Perl mode; you don't need it if you don't use emacs. The -S mess is explained below.

If the #! line does not contain the word "perl", the program named after the #! is executed instead of the Perl interpreter. For example, suppose you have an ordinary Bourne shell script out there that says:

#!/bin/sh echo "I am a shell script"

If you feed that file to Perl, then Perl will run /bin/sh for you. This is slightly bizarre, but it helps people on machines that don't recognize #!, because - by setting their SHELL environmental variable - they can tell a program (such as a mailer) that their shell is /usr/bin/perl, and Perl will then dispatch the program to the correct interpreter for them, even though their kernel is too stupid to do so. Classify it as a strange form of cooperation.

But back to Perl scripts that are really Perl scripts. After locating your script, Perl compiles the entire script to an internal form. If any compilation errors arise, execution of the script is not attempted (unlike the typical shell script, which might run partway through before finding a syntax error). If the script is syntactically correct, it is executed. If the script runs off the end without hitting an exit or die operator, an implicit exit(0) is provided to indicate successful completion.

Switches

A single-character switch with no argument may be combined (bundled) with the following switch, if any.

#!/usr/bin/perl -spi.bak # same as -s -p -i.bak

Switches are also known as options, or flags. Perl recognizes these switches:

• --
• Terminates switch processing, even if the next argument starts with a minus. It has no other effect.
• -0[octnum]
• Specifies the record separator ($/) as an octal number. If octnum is not present, the null character is the separator. Other switches may precede or follow the octal number. For example, if you have a version of find(1) that can print filenames terminated by the null character, you can say this:

  find . -name '*.bak' -print0 | perl -n0e unlink
  

  The special value will cause Perl to slurp files in paragraph mode, equivalent to setting the $/ variable to "". The value will cause Perl to slurp files whole since there is no legal ASCII character with that value. This is equivalent to undefining the $/ variable.

• -a
• Turns on autosplit mode when used with a -n or -p. An implicit split command to the @F array is done as the first thing inside the implicit while loop produced by the -n or -p. So:

  perl -ane 'print pop(@F), "\n";'
  

  is equivalent to:

  while (<>) {
   @F = split(' ');
  print pop(@F), "\n";
  }
  

  A different field delimiter may be specified using -F.

• -c
• Causes Perl to check the syntax of the script and then exit without executing it. Actually, it will execute any BEGIN blocks and use directives, since these are considered to occur before the execution of your program. It also executes any END blocks, in case they need to clean up something that happened in a corresponding BEGIN block. The switch is more or less equivalent to having an exit(0) as the first statement in your program.
• -d
• Runs the script under the Perl debugger. See "The Perl Debugger" in Other Oddments.
• -d:foo
• Runs the script under the control of a debugging or tracing module installed in the Perl library as Devel::foo. For example, -d:DProf executes the script using the Devel::DProf profiler. See also the debugging section in .
• -Dnumber
• -Dlist
• Sets debugging flags. (This only works if debugging is compiled into your version of Perl via the -DDEBUGGING C compiler switch.) You may specify either a number that is the sum of the bits you want, or a list of letters. To watch how it executes your script, for instance, use -D14 or -Dslt. Another nice value is -D1024 or -Dx, which lists your compiled syntax tree. And -D512 or -Dr displays compiled regular expressions. The numeric value is available internally as the special variable $^D. Here are the assigned bit values:

  Bit	Letter	Meaning
  p	Tokenizing and parsing
  s	Stack snapshots
  l	Label stack processing
  t	Trace execution
  o	Object method Lookup
  c	String/numeric conversions
  P	Print preprocessor command for -P
  m	Memory allocation
  f	Format processing
  r	Regular expression processing
  x	Syntax tree dump
  u	Tainting checks
  L	Memory leaks (not supported any more)
  H	Hash dump -- usurps values()
  X	Scratchpad allocation
  D	Cleaning up

• -e commandline
• May be used to enter one or more lines of script. If -e is used, Perl will not look for a script filename in the argument list. The -e argument is treated as if it ends with a newline, so multiple -e commands may be given to build up a multi-line script. (Make sure to use semicolons where you would in a normal program.) Just because -e supplies a newline on each argument doesn't mean you have to use multiple -e switches - if your shell supports multi-line quoting, you may pass a multi-line script as one -e argument, just as awk(1) scripts are typically passed.
• -Fpattern
• Specifies the pattern to split on if -a is also in effect. The pattern may be surrounded by //, "" or '', otherwise it will be put in single quotes. (Remember that to pass quotes through a shell, you have to quote the quotes.)
• -h
• Prints a summary of Perl's command-line options.
• -i[extension]
• Specifies that files processed by the <> construct are to be edited in-place. It does this by renaming the input file, opening the output file by the original name, and selecting that output file as the default for print statements. The extension, if supplied, is added to the name of the old file to make a backup copy. If no extension is supplied, no backup is made. From the shell, saying:

  $ perl -p -i.bak -e "s/foo/bar/; ... "
  

  is the same as using the script:

  #!/usr/bin/perl -pi.bak s/foo/bar/;
  

  which is equivalent to:

  #!/usr/bin/perl while (<>) {
   if ($ARGV ne $oldargv) {
   rename($ARGV, $ARGV . '.bak'); open(ARGVOUT, ">$ARGV"); select(ARGVOUT); $oldargv = $ARGV;
  }
  s/foo/bar/;
  }
  continue {
   print; # this prints to original filename
  }
  select(STDOUT);
  

  except that the -i form doesn't need to compare $ARGV to $oldargv to know when the filename has changed. It does, however, use ARGVOUT for the selected filehandle. Note that STDOUT is restored as the default output filehandle after the loop. You can use eof without parentheses to locate the end of each input file, in case you want to append to each file, or reset line numbering (see the examples of eof in Functions).

• -Idirectory
• Directories specified by -I are prepended to @INC, which holds the search path for modules. -I also tells the C preprocessor where to search for include files. The C preprocessor is invoked with -P; by default it searches /usr/include and /usr/lib/perl. Unless you're going to be using the C preprocessor (and almost no one does any more), you're better off using the use lib directive within your script.
• -l[octnum]
• Enables automatic line-end processing. It has two effects: first, it automatically chomps the line terminator when used with -n or -p, and second, it sets $\ to the value of octnum so any print statements will have a line terminator of ASCII value octnum added back on. If octnum is omitted, sets $\ to the current value of $/, typically newline. So, to trim lines to 80 columns, say this:

  perl -lpe 'substr($_, 80) = ""'
  

  Note that the assignment $\ = $/ is done when the switch is processed, so the input record separator can be different from the output record separator if the -l switch is followed by a -0 switch:

  gnufind / -print0 | perl -ln0e 'print "found $_" if -p'
  

  This sets $\ to newline and later sets $/ to the null character. (Note that would have been interpreted as part of the -l switch had it followed the -l directly. That's why we bundled the -n switch between them.)

• -m[-]module
• -M[-]module
• -M[-]'module ...'
• -[mM][-]module=arg[arg]
• -mmodule
• Executes use module() before executing your script.
• -Mmodule
• Executes use module before executing your script. The command is formed by mere interpolation, so you can use quotes to add extra code after the module name, for example, -M'module qw(foo bar)'. If the first character after the -M or -m is a minus (-), then the use is replaced with no.

  A little built-in syntactic sugar means you can also say -mmodule=foo,bar or -Mmodule=foo,bar as a shortcut for -M'module qw(foo bar)'. This avoids the need to use quotes when importing symbols. The actual code generated by -Mmodule=foo,bar is:

  use module split(/,/, q{foo, bar})
  

  Note that the = form removes the distinction between -m and -M.

• -n
• Causes Perl to assume the following loop around your script, which makes it iterate over filename arguments rather as sed -n or awk do:

  LINE: while (<>) {
   ... # your script goes here }
  

  Note that the lines are not printed by default. See -p to have lines printed. Here is an efficient way to delete all files older than a week, assuming you're on UNIX:

  find . -mtime +7 -print | perl -nle unlink
  

  This is faster than using the -exec switch of find(1) because you don't have to start a process on every filename found. By an amazing coincidence, BEGIN and END blocks may be used to capture control before or after the implicit loop, just as in awk.

• -p
• Causes Perl to assume the following loop around your script, which makes it iterate over filename arguments rather as sed does:

  LINE: while (<>) {
   ... # your script goes here
  }
  continue {
   print;
  }
  

  Note that the lines are printed automatically. To suppress printing use the -n switch. A -p overrides a -n switch. By yet another amazing coincidence, BEGIN and END blocks may be used to capture control before or after the implicit loop, just as in awk.

• -P
• Causes your script to be run through the C preprocessor before compilation by Perl. (Since both comments and cpp(1) directives begin with the # character, you should avoid starting comments with any words recognized by the C preprocessor such as "if", "else" or "define".)
• -s
• Enables some rudimentary switch parsing for switches on the command line after the script name but before any filename arguments or "--" switch terminator. Any switch found there is removed from @ARGV, and a variable of the same name as the switch is set in the Perl script. No switch bundling is allowed, since multi-character switches are allowed. The following script prints "true" if and only if the script is invoked with a -xyz switch.

  #!/usr/bin/perl -s if ($xyz) {
   print "true\n";
  }
  

  If the switch in question is followed by an equals sign, the variable is set to whatever follows the equals sign in that argument. The following script prints "true" if and only if the script is invoked with a -xyz=abc switch.

  #!/usr/bin/perl -s if ($xyz eq 'abc') {
   print "true\n";
  }
  

• -S
• Makes Perl use the PATH environment variable to search for the script (unless the name of the script starts with a slash). Typically this is used to emulate #! startup on machines that don't support #!, in the following manner:

  #!/usr/bin/perl eval "exec /usr/bin/perl -S $0 $*" if $running_under_some_shell;
  

  The system ignores the first line and feeds the script to /bin/sh, which proceeds to try to execute the Perl script as a shell script. The shell executes the second line as a normal shell command, and thus starts up the Perl interpreter. On some systems $0 doesn't always contain the full pathname, so -S tells Perl to search for the script if necessary. After Perl locates the script, it parses the lines and ignores them because the variable $running_under_some_shell is never true. A better construct than $* would be ${1+"$@"}, which handles embedded spaces and such in the filenames, but doesn't work if the script is being interpreted by csh. In order to start up sh rather than csh, some systems have to replace the #! line with a line containing just a colon, which Perl will politely ignore. Other systems can't control that, and need a totally devious construct that will work under any of csh, sh, or perl, such as the following:

  eval '(exit $?0)' && eval 'exec /usr/bin/perl -S $0 ${1+"$@"}' & eval 'exec /usr/bin/perl -S $0 $argv:q' if 0;
  

  Yes, it's ugly, but so are the systems that work[] this way.

  [2] We use the term advisedly.

• -T
• Forces "taint" checks to be turned on so you can test them. Ordinarily these checks are done only when running setuid or setgid. It's a good idea to turn them on explicitly for programs run on another's behalf, such as CGI programs. See "Cooperating with Strangers" later in this chapter.
• -u
• Causes Perl to dump core after compiling your script. You can then take this core dump and turn it into an executable file by using the undump program (not supplied). This speeds startup at the expense of some disk space (which you can minimize by stripping the executable). If you want to execute a portion of your script before dumping, use Perl's dump operator instead. Note: availability of undump is platform specific; it may not be available for a specific port of Perl.
• -U
• Allows Perl to do unsafe operations. Currently the only "unsafe" operations are the unlinking of directories while running as superuser, and running setuid programs with fatal taint checks turned into warnings.
• -v
• Prints the version and patchlevel of your Perl executable.
• -V
• Prints a summary of the major Perl configuration values and the current value of @INC.
• -V:name
• Prints to STDOUT the value of the named configuration variable.
• -w
• Prints warnings about identifiers that are mentioned only once, and scalar variables that are used before being set. Also warns about redefined subroutines, and references to undefined filehandles or filehandles opened read-only that you are attempting to write on. Also warns you if you use a non-number as though it were a number, or if you use an array as though it were a scalar, or if your subroutines recurse more than 100 deep, and innumerable other things. See every entry labeled (W) in Diagnostic Messages.
• -xdirectory
• Tells Perl to extract a script that is embedded in a message. Leading garbage will be discarded until the first line that starts with #! and contains the string "perl". Any meaningful switches on that line after the word "perl" will be applied. If a directory name is specified, Perl will switch to that directory before running the script. The -x switch only controls the disposal of leading garbage. The script must be terminated with __END__ or __DATA__ if there is trailing garbage to be ignored. (The script can process any or all of the trailing garbage via the DATA filehandle if desired.)