Accenture Enkitec Group E4 Webinar

Friday, May 26, 2017

Interview with PeopleSoft Administrator Podcast: Cost-Based Optimizer Statistics in PeopleSoft

I recently recorded another interview with Dan Iverson and Kyle Benson for the PeopleSoft Administrator Podcast, this time about management of Cost-Based Optimizer Statistics in PeopleSoft systems.

(19 May 2017) #81 - Database Statistics

You can listen to the podcast on psadmin.io, or subscribe with your favourite podcast player, or in iTunes.

Sunday, February 26, 2017

Running Unix Shell Scripts from the PeopleSoft Process Scheduler


It is nearly 10 years since I first wrote about how to call Unix shell scripts from the Process Scheduler.  Although very little has changed, I have had a number of questions recently, so I thought it was time I checked the script and updated the posting.  I have used PeopleTools 8.54 in the preparation of this note.
The Process Scheduler is essentially just a mechanism for initiating processes on another server.  Mostly those are other PeopleSoft delivered executables.  The exception is the Application Engine Tuxedo Server process (PSAESRV) where the Process Scheduler submits a service request message that is picked up by one of the server processes that are already running.
NB: although the PSAESRV server is configured by default in the Process Scheduler domain, Oracle recommend that you should only use this when you have lots of very short-lived (runtime less than 30 seconds) application engine processes.  Typically, this only occurs in CRM.

Process Type Definition

First you need to create a new process type. I chose to call it ‘Shell Script’.  It runs a named shell wrapper script, psft.sh.  The wrapper script calls the script that is to be executed. Note that the command line in the process type definition includes the fully qualified path.


Wrapper Script

This is the wrapper script, psft.sh, that will be called by the process scheduler.
#!/bin/ksh
# (c) David Kurtz 2007
# Script:  psft.sh
#
# Syntax:  psft.sh DBNAME ACCESSID ACCESSPSWD PRCSINSTANCE command
# where
# DBNAME is the name of the PeopleSoft datbase with a corresponding TNS entry
# ACCESSID is the schema containing the PeopleSoft database
# ACCESSPSWD is the password to ACCESSID
# PRCSINSTANCE is the process instance number supplied by PeopleSoft
#
# Purpose: To start Standard UNIX Shell Script from Process Scheduler, and interface with the PeopleSoft Process Scheduler
# 07.09.2007 Initial Version
# 23.02.2017 Remove unnecessary logfiles section
#set -x 

if [ $# -lt 4 ]; then
  echo "Usage $0: <DBNAME> <ACCESSID> <ACCESSPSWD> <PRCSINSTANCE> <command>"
  exit 1
fi

CONNECT=$2/$3@$11
PRCSINSTANCE=$4
shift 4

#
# Function to set status of API aware process instance
#
function prcsapi2
{
if [ $# -lt 2 ]; then
  echo "Parameter Error in function $0"
  exit 1
fi

TIMESTAMPCOL=${1}
STATUS=${2}

if [ ${PRCSINSTANCE} -gt 0 ];then
  echo "Setting process request ${PRCSINSTANCE} to status ${STATUS}"
sqlplus -S /nolog <<!
set termout off echo off feedback off verify off
connect ${CONNECT}
UPDATE psprcsque
SET    runstatus = ${STATUS}
,      sessionidnum = $$3
,      lastupddttm = SYSTIMESTAMP
WHERE  prcsinstance = ${PRCSINSTANCE}
;
UPDATE psprcsrqst 
SET    runstatus = ${STATUS}
,      prcsrtncd = ${PRCSRTNCD}
,      continuejob = DECODE(${STATUS},2,1,7,1,9,1,0)4
,      ${TIMESTAMPCOL} = SYSTIMESTAMP
,      lastupddttm = SYSTIMESTAMP
;
COMMIT;
exit
!

  RET=$?
  if [ ! $RET ];then
    echo "SQL*Plus Error Return Code: $?"
  fi
fi
}

#
# Main Execution Starts Here
#

echo $0:$*
date
uname -a
echo "Current Directory: `pwd`"
echo "Process log files in: ${PSPRCSLOGDIR}"
PRCSRTNCD=0
prcsapi begindttm 75 
#set
#Run the command
$* 
PRCSRTNCD=$?6

if [ ${PRCSRTNCD} -ne 0 ]; then
  prcsapi enddttm 3 # failure
else
  prcsapi enddttm 9 # success
fi

date
Notes:
  1. The Oracle user ID, password and TNS name for the PeopleSoft database are supplied in the first three parameters to the wrapper script. The PeopleSoft Process Instance number is the 4th command line parameter. These parameters are then removed with the shift command leaving any other parameters that have been specified.
  2. Function prcsapi sets the status on the process request row and updates the appropriate timestamp columns in the Process Scheduler tables.
  3. PSPRCSQUE.SESSIONIDNUM holds the operating system process ID of the shell executing the wrapper script.
  4. When the process completes and an end of process status is set (either 9 for success, 3 for failure or 2 for delete) CONTINUEJOB is set to 1, otherwise it is set to 0.
  5. When the wrapper scripts start it sets the process status on the process request record to 7 indicate that it is processing.  This can be seen in the Process Monitor.
  6. The return code of the executed script is captured. Later it will be recorded on
    PSPRCSRQST.PRCSRTNCD. A non-zero return code indicates an error and the process status will be set to error.

Process Definition

Now I can create a Process Definition that will use the process type twrapper script to execute another command or script.
The first four parameters passed to the wrapper script are the name of the database, the access ID and password, and the process instance. A string of further parameters will be appended in the individual Process Definition that is the specific command and parameters to be executed.
It is important that this new process type is defined as being API aware.  That means the process interacts with the Process Scheduler by updating the process status.  You can see how the interaction should be done by looking at procedure Update-Process-Status in the delivered SQR library prcsapi.sqc. Otherwise, the Process Scheduler cannot determine their status.  Consequently, all API-unaware processes have a run status of Success to indicate that they were started successfully.

I have written a silly test script called i that I want to be executed by the Process Scheduler.  It just prints out the command line parameters as banner text to both standard output and a file called mybanner.log.  This script will be called by psft.sh
The Process Scheduler creates a working directory for each process request.  It sets the variable $PSPRCSLOGDIR to the fully qualified location of this directory. Note that mybanner.sh changes the current directory to the location of this variable so that it writes mybanner.log there, and thus it is picked up by the distribution agent and made available via the report repository.  You may wish to do this in your scripts.
Current working directory can be specified at Process Type or Process definition.  However, during my testing, I found that these settings had no effect.  The working directory of the script did not change, and the value was not found in any environmental variable.
#!/bin/ksh
#A silly script to test psft.sh
#(c) David Kurtz 2017
#banner function from http://stackoverflow.com/questions/652517/whats-the-deal-with-the-banner-command

if [ "$PSPRCSLOGDIR" ] ; then
  cd $PSPRCSLOGDIR
fi

(
while [ $# -gt 0 ]
 do
  /opt/oracle/psft/ptdb/custhome/banner $1
  shift
done
) | tee mybanner.log
exit $?
I can now create a Process Definition that uses the Shell Script process type that will execute mybanner.sh.  Note that this command line is appended to the existing command line specified in the Process Type definition

You can't quite see it in the screen shot, but the parameter list includes the process instance number:
/opt/oracle/psft/ptdb/custhome/mybanner.sh "Hello World" %%INSTANCE%%

Process Scheduler System Settings

During my testing, I found that it was necessary to specify output type settings for process type Other in the Process Scheduler System Settings; otherwise the output files were not posted to the report repository.


The newly defined Process can be run just as any other process is usually run. Any output from the script on the standard output channel is captured by the Process Scheduler and written to a log file that can then be viewed from the View Log/Trace facility within Process Monitor.
In this case the standard output was written to OTH_DMKTEST_.log, and I also get the mybanner.log that was written to $PSPRCSLOGDIR  in the list of available files.

mybanner.log
contains just the three words passed as parameters
H     H         ll      ll              
H     H          l       l              
H     H  eeee    l       l       oooo   
HHHHHHH e    e   l       l      o    o  
H     H eeeeee   l       l      o    o  
H     H e        l       l      o    o  
H     H  eeee   lll     lll      oooo   
                                        
W     W                 ll           d  
W     W                  l           d  
W     W  oooo   rr rr    l           d  
W     W o    o   rr  r   l       ddddd  
W  W  W o    o   r       l      d    d  
W W W W o    o   r       l      d    d  
 W   W   oooo   rr      lll      dddd d 
                                        
 33333   99999   99999  5555555  00000  
3     3 9     9 9     9 5       0    00 
      3 9     9 9     9 5       0   0 0 
  3333   999999  999999  55555  0  0  0 
      3       9       9       5 0 0   0 
3     3       9       9 5     5 00    0 
 33333   99999   99999   55555   00000
OTH_DMKTEST_39950.log contains the standard output of the entire command - including the additional messages emitted by psft.sh (in bold).
Note that the current directory is reported as being the location of the Process Scheduler Tuxedo domain. 
/opt/oracle/psft/ptdb/custhome/psft.sh:/opt/oracle/psft/ptdb/custhome/mybanner.sh Hello World 39950
Tue Sep  1 21:59:46 UTC 2015
Linux hcm.london.go-faster.co.uk 2.6.39-400.215.10.el5uek #1 SMP Tue Sep 9 22:51:46 PDT 2014 x86_64 x86_64 x86_64 GNU/Linux
Current Directory: /home/psadm2/psft/pt/8.54/appserv/prcs/PRCSDOM
Process log files in: /home/psadm2/psft/pt/8.54/appserv/prcs/PRCSDOM/log_output/OTH_DMKTEST_39950
Setting process request 39950 to status 7
H     H         ll      ll              
H     H          l       l              
H     H  eeee    l       l       oooo   
HHHHHHH e    e   l       l      o    o  
H     H eeeeee   l       l      o    o  
H     H e        l       l      o    o  
H     H  eeee   lll     lll      oooo   
                                        
W     W                 ll           d  
W     W                  l           d  
W     W  oooo   rr rr    l           d  
W     W o    o   rr  r   l       ddddd  
W  W  W o    o   r       l      d    d  
W W W W o    o   r       l      d    d  
 W   W   oooo   rr      lll      dddd d 
                                        
 33333   99999   99999  5555555  00000  
3     3 9     9 9     9 5       0    00 
      3 9     9 9     9 5       0   0 0 
  3333   999999  999999  55555  0  0  0 
      3       9       9       5 0 0   0 
3     3       9       9 5     5 00    0 
 33333   99999   99999   55555   00000  
                                        
Setting process request 39950 to status 9
Tue Sep  1 21:59:46 UTC 2015 
A more detailed version of this document can be found at http://www.go-faster.co.uk/docs/process_scheduler_shell_scripts.pdf.

Tuesday, January 17, 2017

Removing Redundant Indexes in PeopleSoft

This is the second of a two-part article that discusses how to identify and remove redundant indexes from a PeopleSoft system. 
This article describes a script on my website (psredundantfix.sql) that uses a similar query to that described in the previous article to identify redundant indexes from the metadata in the PeopleTools tables. It uses an anonymous block of PL/SQL so nothing is installed in the database. For each redundant index identified it:
  • Adds the record and redundant index into an Application Designer project called REDUNDANT INDEXES. The record must also be added, otherwise Application will not generate a DDL script to remove the index.
  • Unsets the platform flag on the redundant index if the superset index is active on the same platform. Thus Application Designer will generate a script to drop redundant indexes when the project is built. The redundant index definition remains in the record definition in case there is a need to revert the change.
  • If the superset index is active on Oracle
    • Creates and collects extended optimizer statistics on the combination of columns in the redundant index. If error ORA-00054 occurs, the exception will be ignored and the script will continue.
    • Makes the index invisible. Note that indexes are not dropped. That can be done later by building an alter script for the project. It would be normal to test the consequences of making the indexes invisible for a while, and drop them later. 

Sample Output

Project REDUNDANT INDEXES already exists
…
Record PSPROJECTITEM, Redundant Index C already in project
Creating Extended Statistics on PSPROJECTITEM for OBJECTVALUE4,OBJECTTYPE
Making index PSCPSPROJECTITEM invisible
ORA-00054: resource busy and acquire with NOWAIT specified or timeout expired
…
Record PSTREENODE, Redundant Index C already in project
…
Record TL_RPTD_TIME, Redundant Index C added to project
Making index PSCTL_RPTD_TIME invisible
Record TL_TA_BATCH, Redundant Index _ added to project
Making index PS_TL_TA_BATCH invisible
Making index PS_TL_TA_BATCH1 invisible
…
Making index PS_TL_TA_BATCH48 invisible
…
Record WRK_XREF_CALC, Redundant Index A already in project
Creating Extended Statistics on PS_WRK_XREF_CALC for PROCESS_INSTANCE,SEQ_NBR
Creating Extended Statistics on PS_WRK_XREF_CALC1 for PROCESS_INSTANCE,SEQ_NBR
…
Creating Extended Statistics on PS_WRK_XREF_CALC6 for PROCESS_INSTANCE,SEQ_NBR
Making index PSAWRK_XREF_CALC invisible
Making index PSAWRK_XREF_CALC1 invisible
…
Making index PSAWRK_XREF_CALC6 invisible
Note:
  • The script commits any changes it makes. There is also a rollback command at the top to prevent it accidentally committing something else.
  • Having run it once, should you immediately run the script again, nothing will be found because any redundant indexes have been marked as disabled on Oracle
  • Should you want to revert changes, mark all indexes in the REDUNDANT INDEXES project as active on Oracle.
UPDATE psindexdefn
SET    platform_ora = 1
,      activeflag = 1
WHERE  (recname, indexid) IN (
  SELECT objectvalue1, objectvalue2
  FROM   psprojectitem
  WHERE  objecttype = 1
  AND    projectname = 'REDUNDANT INDEXES')
AND    platform_ora = 0;
COMMIT;
We can see in Application Designer that index PSAWRK_XREF_CALC is a subset of PS_WRK_XREF_CALC because both start with PROCESS_INSTANCE and SEQ_NBR.


Index A was marked inactive on all platforms because the superset index _ is active on all platforms.  


PeopleSoft delivers some indexes for some platforms only. For example, PSETREENODE is only active on Informix. Therefore the script only deactivates the Informix platform flag on PSCTREENODE, and the platform flag changes to 'Some'. The index is still added to the project, but no attempt is made to create extended statistics or to make it invisible, and the index continues to be built on Oracle.

The comment applied to index C indicates that only the Informix flag was changed.



The final step, when you are satisfied that making the indexes invisible has no unacceptable consequences, is to drop the redundant indexes using the script generated by Application Designer
…
DROP INDEX PSCPSPROJECTITEM
/
DROP INDEX PSEPSPROJECTITEM
/
…
DROP INDEX PSAWRK_XREF_CALC
/
DROP INDEX PSAWRK_XREF_CALC1
/
…
DROP INDEX PSAWRK_XREF_CALC6
/