Friday, June 15, 2007

Unix Process Limits for PeopleSoft

I have been having some trouble with long running Component Interfaces/Application Engine processes. The amount of memory allocated to the process continues to grow throughout the life of the process, and eventually the process hits the limit imposed by Unix. This leads to two modes of error.

You will get an error message like this in the Application Engine log file.

SQL error. Stmt #: Error Position: Return: 8007 - Insufficient memory available

The process might continue to run, or it might fail (I am not sure what makes the difference here), but if it terminates you get will get a message like this also in the Application Engine log.

<Unix PID>: 1181303384: PSPAL::Abort: Unrecoverable signal received
<Unix PID>: 1181303384: PSPAL::Abort: Location: /vob/peopletools/src/pspal/exception_sigaction.cpp:553: PSPAL::SigactionSignalHandler::UnrecoverableSignalHandler(int, siginfo_t *, void *)
<Unix PID>: 1181303384: PSPAL::Abort: Generating process state report to <PS HOME>appserv/prcs/<Process Scheduler Name>/LOGS/psae.<Unix PID>/process_state.txt
<Unix PID>: 1181303384: PSPAL::Abort: Recursive call to Abort detected. Terminating process now.

A core file is also produced.

Application Engine seems to continue to allocate memory but not relinquish it again. I have found support notes on Customer Connection that recommend use of the CollectGarbage() function in PeopleCode in order to 'remove any unreachable application objects created by Application Classes'. However, my tests suggest that this function does not cause any memory to be released by the process.

Another suggestion from Customer Connection was to raise the Unix system limit for the soft data segment with ulimit -d. This has resolved the memory errors. The command ulimit -d can be user to set soft limits in the Unix shell. However, they cannot be set higher than hard limits. Hence it is necessary to set unlimited parameters in /etc/security/limits (this file can only be read or updated by the superuser: root).

It is common to disable the limits for Oracle, as shown in the example below. I think the same should be done for PeopleSoft accounts. Machine-wide parameter settings the set in the default section. Individual parameters can then be overridden for individual users. The following example is taken from AIX 5.3. The sizes are multiples of 512 byte blocks. A value of -1 means that the parameter is unlimited.

fsize = 2097151
core = 2097151
cpu = -1
data = 262144
rss = 65536
stack = 65536
nofiles = 4000
fsize = -1
data = 4194303
rss = 4194303
nofiles = 4000

fsize = -1
data = -1
stack = -1

data = -1

Changes to the limits take effect when the user next logs onto Unix. After changing the limits you need to log out and back into your Unix account and restart the process scheduler


Armand said...

Thanks it's help me so much ...

Anne said...

Thanks for this post.

I'm getting the "PSPAL: :Abort: Unrecoverable Signal ..." error, and this is the *ONLY* relevant reference I've hit upon in all of Googledom.

In my case there is a third-party app (Novell Identity Manager driver for pSoft) running against a Component Interface (no App Engine involved), but the error stack in the log is essentially identical to what you're seeing, so hopefully it's not too far-fetched to extrapolate that the cause may be similar as well.