Updated: July 20, 2004

The Portable Batch System (PBS) was written as a joint project between the Numerical Aerospace Simulation (NAS) Systems Division of NASA AMES Research Center and National Energy Research Supercomputer Center (NERSC) of Lawrence Livermore National Laboratory. It is a batch software processing system that provides a simple consistent mean of job submission to a flexible and configurable job scheduler. It is currently being employed across all of our HPC systems.

See the PBS Homepage for more details.

Although development (compiles, short runs, etc.) is allowed, there is a kernel limit of 15 cpu minutes on all interactive processes. Processes that exceed 15 minutes and are not submitted through the PBS system are subject to be immediately killed without prior notification.

CHPC Implementation Specifics

PBS works together with a scheduler to manage resources and optimize the use of available resources. At CHPC we use the Maui Scheduler on the Compaq Sierra Cluster and the ICE Box Linux Cluster. On the shared memory SGI systems, we use a simple fifo scheduler.

For details on platform specific implementation and examples scripts, please refer to the following links:

• Batch on the Opteron Cluster (arches)

Batch Environment Variables

These are the environment variables set by PBS for a job run in October 2000 by user jhc.

PBS_O_HOME=/home/chpc/jhc
PBS_O_LANG=en_US
PBS_O_LOGNAME=jhc
PBS_O_PATH=/uufs/sp/sys/bin:/uufs/sp/host/bin:/usr/local/bin:/usr/bin:/etc:/usr/sbin:/usr/ucb:/usr/bin/X11:/sbin:
/usr/ibmcxx/bin:/usr/local/chpc/bin:/uufs/inscc.utah.edu/sys/bin:/usr/ucb:/usr/totalview/bin:/usr/local/chpc/ncar
g/bin:/home/chpc/jhc/bin:.
PBS_O_MAIL=/usr/spool/mail/jhc
PBS_O_SHELL=/bin/ksh
PBS_O_TZ=MST7MDT
PBS_O_HOST=a68-e.chpc.utah.edu
PBS_O_WORKDIR=/home/chpc/jhc/intro_mpi/sp
PBS_O_QUEUE=sp
PBS_JOBNAME=sppbsjob
PBS_JOBID=5336.a68-e.chpc.utah.edu
PBS_QUEUE=sp
SHELL=/bin/csh
PBS_JOBCOOKIE=2720BE7965F1673E424DD59E5808621C
PBS_NODENUM=0
PBS_TASKNUM=1
PBS_MOMPORT=15003
PBS_NODEFILE=/uufs/sp/host/var/pbs/aux/5336.a68-e.chpc.utah.edu
MP_PROCS=4
MP_HOSTFILE=/uufs/sp/host/var/pbs/aux/5336.a68-e.chpc.utah.edu
PBS_JOB_KEY=1476944412
MP_MPI_NETWORK=0
PBS_ENVIRONMENT=PBS_BATCH
ENVIRONMENT=BATCH

PBS Batch Script Options

• -a date_time.  Declares the time after which the job is eligible for execution. The date_time element is in the form: [[[[CC]YY]MM]DD]hhmm[.S].
• -e path.  Defines the path to be used for the standard error stream of the batch job. The path is of the form: [hostname:]path_name.
• -h.  Specifies that a user hold will be applied to the job at submission time.
• -I.  Declares that the job is to be run "interactively". The job will be queued and scheduled as PBS batch job, but when executed the standard input, output, and error streams of the job will be connected through qsub to the terminal session in which qsub is running.
• -j join.  Declares if the standard error stream of the job will be merged with the standard ouput stream. The join argument is one of the following:
  • oe-  Directs the two streams as standard output.
  • eo-  Directs the two streams as standard error.
  • n-  Any two streams will be separate(Default).
• -l resource_list.  Defines the resources that are required by the job and establishes a limit on the amount of resources that can be consumed. Users will want to specify the walltime resource, and if they wish to run a parallel job, the ncpus resource.
• -m mail_options.  Conditions under which the server will send a mail message about the job. The options are:
  • n: No mail ever sent
  • a (default): When the job aborts
  • b: When the job begins
  • e: When the job ends
• -M user_list.  Declares the list of e-mail addresses to whom mail is sent. If unspecified it defaults to userid@host from where the job was submitted. You will most likely want to set this option.
• -N name.  Declares a name for the job.
• -o path.  Defines the path to be used for the standard output. [hostname:]path_name.
• -q destination.  The destination is the queue.
• -S path_list.  Declares the shell that interprets the job script. If not specified it will use the user's login shell.
• -v variable_list.  Expands the list of environment variables which are exported to the job. The variable list is a comma-separated list of strings of the form variable or variable=value.
• -V.  Declares that all environment variables in the qsub command's environment are to be exported to the batch job.

PBS User Commands

For any of the commands listed below you may do a "man command" for syntax and detailed information.

Frequently used PBS user commands:

• qsub. Submits a job to the PBS queuing system. Please see qsub Options below.
• qdel. Deletes a PBS job from the queue.
• qstat. Shows status of PBS batch jobs.
• xpbs. X interface for PBS users.

Less Frequently-Used PBS User Commands:

• qalter. Modifies the attributes of a job.
• qhold. Requests that the PBS server place a hold on a job.
• qmove. Removes a job from the queue in which it resides and places the job in another queue.
• qmsg. Sends a message to a PBS batch job. To send a message to a job is to write a message string into one or more of the job's output files.
• qorder. Exchanges the order of two PBS batch jobs within a queue.
• qrerun. Reruns a PBS batch job.
• qrls. Releases a hold on a PBS batch job.
• qselect. Lists the job identifier of those jobs which meet certain selection criteria.
• qsig. Requests that a signal be sent to the session leader of a batch job.