Sunday, February 26, 2017

Running Unix Shell Scripts from the PeopleSoft Process Scheduler


It is nearly 10 years since I first wrote about how to call Unix shell scripts from the Process Scheduler.  Although very little has changed, I have had a number of questions recently, so I thought it was time I checked the script and updated the posting.  I have used PeopleTools 8.54 in the preparation of this note.
The Process Scheduler is essentially just a mechanism for initiating processes on another server.  Mostly those are other PeopleSoft delivered executables.  The exception is the Application Engine Tuxedo Server process (PSAESRV) where the Process Scheduler submits a service request message that is picked up by one of the server processes that are already running.
NB: although the PSAESRV server is configured by default in the Process Scheduler domain, Oracle recommends that you should only use this when you have lots of very short-lived (runtime less than 30 seconds) application engine processes.  Typically, this only occurs in CRM.

(Update 2.7.2019) Dan Iverson of psadmin.io has also converted the psft.sh script to run on Windows in Powershell - see Process Scheduler and Powershell (https://psadmin.io/2019/07/02/process-scheduler-and-powershell/).  The code is available on github in the ps-poweshell repository.

Process Type Definition

First you need to create a new process type. I chose to call it ‘Shell Script’.  It runs a named shell wrapper script, psft.sh.  The wrapper script calls the script that is to be executed. Note that the command line in the process type definition includes the fully qualified path.


Wrapper Script

This is the wrapper script, psft.sh, that will be called by the process scheduler.
#!/bin/ksh
# (c) David Kurtz 2007
# Script:  psft.sh
#
# Syntax:  psft.sh DBNAME ACCESSID ACCESSPSWD PRCSINSTANCE command
# where
# DBNAME is the name of the PeopleSoft datbase with a corresponding TNS entry
# ACCESSID is the schema containing the PeopleSoft database
# ACCESSPSWD is the password to ACCESSID
# PRCSINSTANCE is the process instance number supplied by PeopleSoft
#
# Purpose: To start Standard UNIX Shell Script from Process Scheduler, and interface with the PeopleSoft Process Scheduler
# 07.09.2007 Initial Version
# 23.02.2017 Remove unnecessary logfiles section
#set -x 

if [ $# -lt 4 ]; then
  echo "Usage $0: <DBNAME> <ACCESSID> <ACCESSPSWD> <PRCSINSTANCE> <command>"
  exit 1
fi

CONNECT=$2/$3@$11
PRCSINSTANCE=$4
shift 4

#
# Function to set status of API aware process instance
#
function prcsapi2
{
if [ $# -lt 2 ]; then
  echo "Parameter Error in function $0"
  exit 1
fi

TIMESTAMPCOL=${1}
STATUS=${2}

if [ ${PRCSINSTANCE} -gt 0 ];then
  echo "Setting process request ${PRCSINSTANCE} to status ${STATUS}"
sqlplus -S /nolog <<!
set termout off echo off feedback off verify off
connect ${CONNECT}
UPDATE psprcsque
SET    runstatus = ${STATUS}
,      sessionidnum = $$3
,      lastupddttm = SYSTIMESTAMP
WHERE  prcsinstance = ${PRCSINSTANCE}
;
UPDATE psprcsrqst 
SET    runstatus = ${STATUS}
,      prcsrtncd = ${PRCSRTNCD}
,      continuejob = DECODE(${STATUS},2,1,7,1,9,1,0)4
,      ${TIMESTAMPCOL} = SYSTIMESTAMP
,      lastupddttm = SYSTIMESTAMP
WHERE  prcsinstance = ${PRCSINSTANCE}
;
COMMIT;
exit
!

  RET=$?
  if [ ! $RET ];then
    echo "SQL*Plus Error Return Code: $?"
  fi
fi
}

#
# Main Execution Starts Here
#

echo $0:$*
date
uname -a
echo "Current Directory: `pwd`"
echo "Process log files in: ${PSPRCSLOGDIR}"
PRCSRTNCD=0
prcsapi begindttm 75 
#set
#Run the command
$* 
PRCSRTNCD=$?6

if [ ${PRCSRTNCD} -ne 0 ]; then
  prcsapi enddttm 3 # failure
else
  prcsapi enddttm 9 # success
fi

date
Notes:
  1. The Oracle user ID, password and TNS name for the PeopleSoft database are supplied in the first three parameters to the wrapper script. The PeopleSoft Process Instance number is the 4th command line parameter. These parameters are then removed with the shift command leaving any other parameters that have been specified.
  2. Function prcsapi sets the status on the process request row and updates the appropriate timestamp columns in the Process Scheduler tables.
  3. PSPRCSQUE.SESSIONIDNUM holds the operating system process ID of the shell executing the wrapper script.
  4. When the process completes and an end of process status is set (either 9 for success, 3 for failure or 2 for delete) CONTINUEJOB is set to 1, otherwise it is set to 0.
  5. When the wrapper scripts start it sets the process status on the process request record to 7 indicate that it is processing.  This can be seen in the Process Monitor.
  6. The return code of the executed script is captured. Later it will be recorded on
    PSPRCSRQST.PRCSRTNCD. A non-zero return code indicates an error and the process status will be set to error.

Process Definition

Now I can create a Process Definition that will use the process type twrapper script to execute another command or script.
The first four parameters passed to the wrapper script are the name of the database, the access ID and password, and the process instance. A string of further parameters will be appended in the individual Process Definition that is the specific command and parameters to be executed.
It is important that this new process type is defined as being API aware.  That means the process interacts with the Process Scheduler by updating the process status.  You can see how the interaction should be done by looking at procedure Update-Process-Status in the delivered SQR library prcsapi.sqc. Otherwise, the Process Scheduler cannot determine their status.  Consequently, all API-unaware processes have a run status of Success to indicate that they were started successfully.

I have written a silly test script called i that I want to be executed by the Process Scheduler.  It just prints out the command line parameters as banner text to both standard output and a file called mybanner.log.  This script will be called by psft.sh
The Process Scheduler creates a working directory for each process request.  It sets the variable $PSPRCSLOGDIR to the fully qualified location of this directory. Note that mybanner.sh changes the current directory to the location of this variable so that it writes mybanner.log there, and thus it is picked up by the distribution agent and made available via the report repository.  You may wish to do this in your scripts.
Current working directory can be specified at Process Type or Process definition.  However, during my testing, I found that these settings had no effect.  The working directory of the script did not change, and the value was not found in any environmental variable.
#!/bin/ksh
#A silly script to test psft.sh
#(c) David Kurtz 2017
#banner function from http://stackoverflow.com/questions/652517/whats-the-deal-with-the-banner-command

if [ "$PSPRCSLOGDIR" ] ; then
  cd $PSPRCSLOGDIR
fi

(
while [ $# -gt 0 ]
 do
  /opt/oracle/psft/ptdb/custhome/banner $1
  shift
done
) | tee mybanner.log
exit $?
I can now create a Process Definition that uses the Shell Script process type that will execute mybanner.sh.  Note that this command line is appended to the existing command line specified in the Process Type definition

You can't quite see it in the screen shot, but the parameter list includes the process instance number:
/opt/oracle/psft/ptdb/custhome/mybanner.sh "Hello World" %%INSTANCE%%

Process Scheduler System Settings

During my testing, I found that it was necessary to specify output type settings for process type Other in the Process Scheduler System Settings; otherwise the output files were not posted to the report repository.


The newly defined Process can be run just as any other process is usually run. Any output from the script on the standard output channel is captured by the Process Scheduler and written to a log file that can then be viewed from the View Log/Trace facility within Process Monitor.
In this case the standard output was written to OTH_DMKTEST_.log, and I also get the mybanner.log that was written to $PSPRCSLOGDIR  in the list of available files.

mybanner.log
contains just the three words passed as parameters
H     H         ll      ll              
H     H          l       l              
H     H  eeee    l       l       oooo   
HHHHHHH e    e   l       l      o    o  
H     H eeeeee   l       l      o    o  
H     H e        l       l      o    o  
H     H  eeee   lll     lll      oooo   
                                        
W     W                 ll           d  
W     W                  l           d  
W     W  oooo   rr rr    l           d  
W     W o    o   rr  r   l       ddddd  
W  W  W o    o   r       l      d    d  
W W W W o    o   r       l      d    d  
 W   W   oooo   rr      lll      dddd d 
                                        
 33333   99999   99999  5555555  00000  
3     3 9     9 9     9 5       0    00 
      3 9     9 9     9 5       0   0 0 
  3333   999999  999999  55555  0  0  0 
      3       9       9       5 0 0   0 
3     3       9       9 5     5 00    0 
 33333   99999   99999   55555   00000
OTH_DMKTEST_39950.log contains the standard output of the entire command - including the additional messages emitted by psft.sh (in bold).
Note that the current directory is reported as being the location of the Process Scheduler Tuxedo domain. 
/opt/oracle/psft/ptdb/custhome/psft.sh:/opt/oracle/psft/ptdb/custhome/mybanner.sh Hello World 39950
Tue Sep  1 21:59:46 UTC 2015
Linux hcm.london.go-faster.co.uk 2.6.39-400.215.10.el5uek #1 SMP Tue Sep 9 22:51:46 PDT 2014 x86_64 x86_64 x86_64 GNU/Linux
Current Directory: /home/psadm2/psft/pt/8.54/appserv/prcs/PRCSDOM
Process log files in: /home/psadm2/psft/pt/8.54/appserv/prcs/PRCSDOM/log_output/OTH_DMKTEST_39950
Setting process request 39950 to status 7
H     H         ll      ll              
H     H          l       l              
H     H  eeee    l       l       oooo   
HHHHHHH e    e   l       l      o    o  
H     H eeeeee   l       l      o    o  
H     H e        l       l      o    o  
H     H  eeee   lll     lll      oooo   
                                        
W     W                 ll           d  
W     W                  l           d  
W     W  oooo   rr rr    l           d  
W     W o    o   rr  r   l       ddddd  
W  W  W o    o   r       l      d    d  
W W W W o    o   r       l      d    d  
 W   W   oooo   rr      lll      dddd d 
                                        
 33333   99999   99999  5555555  00000  
3     3 9     9 9     9 5       0    00 
      3 9     9 9     9 5       0   0 0 
  3333   999999  999999  55555  0  0  0 
      3       9       9       5 0 0   0 
3     3       9       9 5     5 00    0 
 33333   99999   99999   55555   00000  
                                        
Setting process request 39950 to status 9
Tue Sep  1 21:59:46 UTC 2015 
A more detailed version of this document can be found at http://www2.go-faster.co.uk/docs/process_scheduler_shell_scripts.pdf.

4 comments :

Mani S said...

Hi David,
Was wondering if you could help me with a peculiar issue.

I've been having performance issues with a SQL view containing a long field.

The SQL view has many other fields - and this particular field has 300 as the length. The underlying SQL text selects data from a DESCRLONG field. Now, if I use a SUBSTR to fetch just 300 characters from DESCRLONG, it takes too long for the query to complete execution. Also, a DESC on the SQL view shows the column as a CLOB. So, if I replace the corresponding SQL text with DBMS_LOB.SUBSTR(DESCRLONG,300), a DESC on the SQL view returns the column type as VARCHAR 4000.

The performance has slightly increased, but not much of a noticeable difference. I'm on PeopleTools 8.53 and Oracle 11g. Have you encountered similar issues before? Any pointers on how to improve performance for such views?

Dan Iverson said...

Hi David, I had one additional configuration step to get the shell scripts to run. In the server definition (PSUNX), I had to add "Shell Script" as one of the allowed process types to run on the scheduler. After that everything worked great.

Bart22 said...

Hi David - shouldn't the UPDATE psprcsrqst statement also contain the WHERE prcsinstance = ${PRCSINSTANCE} clause? I got strange results without it, where all my previous process instance statuses got updated while running a new one, and also I couldn't delete old PIs. This no longer happened after adding the WHERE prcsinstance = ${PRCSINSTANCE} clause to the UPDATE psprcsrqst statement.
Thank you for this - it has been very useful.

David Kurtz said...

Bart - Yes. That is an important omission that I have corrected in the blog. Not sure how it crept in, but thank you.