WebLogic Fails to Listen on SSL Port after Installing a WildCard Certificate or a SHA2 Certificate. Logs Message: "Cannot convert identity certificate

This error can occur for two reasons:

Reason#1: Using a non-compliant certificate such as wildcard
------------------------------------------------------------------------
Older WebLogic versions, such as WebLogic 8.1 and 9.2, do not support wildcard certificates as per the RFC3280 specifications.   Although even though the wild card certificates are not supported on older WebLogic versions, they often function without issues.   However, we do know that there are issues with wildcards on WebLogic 8.1SP6 and we occasionally see issues on other WebLogic versions as well. In addition, there can be problems with using wildcard certificates on a PeopleSoft Gateway server as noted in bug 11607478.

Reason#2: Using a SHA2 certificate
------------------------------------------------
SHA2 is a newer hash algorithm and is supported by WebLogic starting with version 10.3.3.  However, if you use SHA2, you must enable JSSE SSL (details in 'Solution' section) or else you will get the above error

************************************************************************************

If you are using a Wildcard Certificate:
-----------------------------------------------
Install a certificate with no wildcard characters in the common name (CN)

If you are using a SHA 2 Certificate (ie Signature Algorithm starts with 'SHA2')
------------------------------------------------------------------------------------------
1. If you are using WebLogic 10.3.2 or lower, you must first upgrade to WebLogic 10.3.3 or higher (ideally, you should upgrade to WebLogic 10.3.6 as there are some SHA2 bug fixes that are included in WebLogic 10.3.6. Refer to document 1389918.1 for instructions on upgrading WebLogic.   If you are unable to upgrade, then you will need to switch to a SHA1 certificate

2. If you are using WebLogic 10.3.3 or higher, then use the following steps to enable JSSE SSL which trusts stronger certificates such as SHA2.
    a. Log in to your WebLogic admin console
    b. From left menu, choose Environment -> Servers -> PIA
    c. Click the 'Configuration' tab and 'SSL' subtab
    d. Go to bottom of page and click the 'Advanced' hyperlink
    e. Click the 'Lock & Edit' button on top left menu
    f. Go to bottom of page and check "Use JSSE SSL"
    g. Click "Save"
    h. Click 'Activate Changes'

WebLogic Fails to Start with Error "PersistentStoreException" or "Couldn't obtain an exclusive lock to the embedded LDAP data files directory"

This problem occurs if you kill the WebLogic process via kill -9 on Unix or via CTL+C from the PIA command prompt on a Windows system.

Make sure that no other java process is using the same port that this server is trying to acquire. If there is another process using this port, you will need to either change the port WebLogic uses or the port for the process that is already running. This can be done through netstat or on Unix, by listing all of the running java processes with "ps -ef|grep java".

OPTION #1:
============
To resolve the error, remove the ldap directory from:
    For PeopleTools 8.44 to 8.48: \webserv\\\
    For PeopleTools 8.49+: \webserv\\servers\\data\

There is file called EmbeddedLDAP.lok that is created under the ldapfiles folder (within the ldap directory). If this file is present, the server cannot boot and you will see the exceptions above.

This file is normally unlocked on shutdown, but if the previous shutdown occurred in an abnormal way, such as a crash, then this file may not be unlocked properly and the server will generate those exceptions.

OPTION #2:
===========
Remove the *.DAT and EmbeddedLDAP.lok files.

1. Make sure the server is stopped.
2. Go to the following directory
     For PeopleTools 8.44 to 8.48: \webserv\\\ldap\ldapfiles
     For PeopleTools 8.49+: \webserv\\servers\\data\ldap\ldapfiles
3. Remove "EmbeddedLDAP.lok"
4. Remove "*.DAT"
5. Restart WebLogic.

Crystal Reports 2008 - Client - Error "Missing or invalid version of SQL library PSORA32 (200.0). Invalid userid or password

The issue is caused by the following setup: PSORA32.dll present in "/bin/client/winx86" cannot be accessed from the PATH environment variable

With Peopletools 8.53, PeopleTools application and batch servers are now 64 bit applications. But some batch server components are still 32 bit. So you need to have both 32 bit and 64 bit Oracle client for batch server to work correctly.

So, as both 64 bit and 32 bit Oracle client on Scheduler machine, please make sure that both Clients locations populated correctly in "Add to Path" parameter in PSPRCS.cfg fle.

------------------------------------------------------------------------------------------------------------------

To implement the solution, please execute the following steps:

1. Please ensure both 32-bit and 64-bit connectivity has been nstalled on the batch server

2.. Go to Control Panel\All Control Panel Items\System > Advance > Environmental Variables 

3. Add the /bin/client/winx86" as the first entry in PATH variable  as the first entry in PATH variable. Make sure that the 32-bit connectivity is listed first, then the 64-bit listed second. If the PATH in the environment variables of the local windows machine is set up incorrectly with 64-bit listed first, then application designer would find the 64-bit connectivity first and attempt to use it, causing an error.   The servers that require 64-bit will not always use the first connectivity encountered in the PATH

4. Reboot the Process Scheduler domain. In some instances, the machine would need a reboot for the solution to take effect.

5. Migrate the solution as appropriate to other environments.

6. In some instances, a reboot of the machine may be required but not necessary.
 

Crystal Will Not Run With The Runtime SP3 Version - Crystal Runtime Initialization Failed

The issue was caused by extra Crystal DLLs existing in the problem environment's %pshome%\setup\crystalbin, which were not delivered with the PeopleTools installation.  It is not clear how these files got into the %pshome%\setup\crystalbin, possibly as a result of a prior copy operation.

1) Remove the extra files in %pshome%\setup\crystalbin which should not be present from the PT installation.
For PT8.51+ only the following should be included under %pshome%\setup\crystalbin:

(don't remove these from their delivered location, they need to be able to be copied by the psodbccrinst.exe program)

~\System\libpng10.dll
~\u2lbar.dll
~\u2lps.dll
~\u25store.dll
~\u25total.dll

NOTE:  To support the PStoWords function, psodbccrinst.exe also copies from %pshome%\bin\client\winx86: 
(don't remove these from their delivered location, they need to be able to be copied by the psodbccrinst.exe program)

~\u25pstowords.dll
~\icudt34.dll
~\psicuin34.dll
~\psicuuc34.dll


2) Uninstall all Crystal and the Crystal Runtime from the process scheduler
3) Disable all antivirus for the duration of the Crystal and Crystal Runtime install
4) Install  Crystal Reports 2008 runtim and client, if it will be used.
5) Rerun the psodbccrinst.exe and confirm that only the above files were copied by viewing the psodbccrinst.log.
6) Reenable anitvirus
7) Restart the process scheduler
8) Retest the issue.
9) Migrate as appropriate

PeopleSoft Virtualization

http://www.youtube.com/watch?v=WNGIUfop8Eo&list=PL4okLhHVjSg5nP9So9CtQVMeA_GS_PqAG

PeopleSoft - Oracle - SAP: Application Engine programs only stuck at queued

PeopleSoft - Oracle - SAP: Application Engine programs only stuck at queued

Generation copy book /remote call and .lis file

For Microsoft Windows, cblmake.bat, which lives under %PS_HOME%\src\cbl\win32. Add this location to your PATH environment variable so you can call the command from anywhere you want. Because there are so many dependencies, I would copy everything under %PS_HOME%\src\cbl\base to a temp location like C:\Temp, then issue the following command to compile a single COBOL in error call mode

C:\Temp>cblmake.bat GLPJEDIT

(Note the drop of .cbl extension).

If you run cblmake.bat without any parameters it will display all available compile options. 

Once compiled, copy the appropriate INT/GNT/EXE to your %PS_HOME%\CBLBIN<%Type%> directory.

Compiling Single COBOL program

To compile a single COBOL program, run the command with the COBOL program name excluding the .cbl extension.

For example, for a PeopleSoft PeopleTools COBOL program PTPDBTST.CBL, or a PeopleSoft
Application COBOL program GPPDPRUN.CBL, run:

pscbl.mak PTPDBTST
pscbl.mak GPPDPRUN


For more information, you may refer 'Installing and Compiling COBOL on UNIX Chapter 12B' in PeopleTools 8.52 install guide 

How to identify who is locking the Oracle account

With AUDIT_TRAIL

The first and preferred solution is with Oracle standard auditing feature. Start by setting initialization parameter AUDIT_TRAIL to db and restart your Oracle database as it is static parameter.
Then activate network auditing with (as SYS):

SQL> AUDIT network BY ACCESS;   AUDIT succeeded.

With below query you get everything needed:

SELECT * FROM dba_audit_session ORDER BY sessionid DESC;

Returncode column contains Oracle error code and so different of 0 if logon/logoff issue. The invalid password is the error we are chasing:

[oracle@server1 ~]$ oerr ora 1017 01017, 00000, "invalid username/password; logon denied" // *Cause: // *Action:

So if you find 1017 values in this column then we have found what we were looking for. For example with my test case where I intentionally specify a wrong password for my account:

SQL> SELECT username,userhost,returncode FROM dba_audit_session WHERE username='YJAQUIER' ORDER BY sessionid DESC;   USERNAME USERHOST RETURNCODE ------------------------------ -------------------- ---------- YJAQUIER server1 1017 YJAQUIER GVADT30596 0 YJAQUIER server1 0 YJAQUIER server1 0 . . .

And if you insist, as explained, you get:

SQL> SELECT username, account_status,lock_date, PROFILE FROM dba_users WHERE username='YJAQUIER';   USERNAME ACCOUNT_STATUS LOCK_DATE PROFILE ------------------------------ -------------------------------- -------------------- ------------------------------ YJAQUIER LOCKED(TIMED) 23-nov-2012 10:30:37 DEFAULT

If you set AUDIT_TRAIL to db behave the size of SYS.AUD$ table as a small list of audits are already implemented by default:

SQL> SET lines 200 SQL> SET pages 200 SQL> SELECT * FROM DBA_STMT_AUDIT_OPTS;   USER_NAME PROXY_NAME AUDIT_OPTION SUCCESS FAILURE ------------------------------ ------------------------------ ---------------------------------------- ---------- ---------- ALTER SYSTEM BY ACCESS BY ACCESS SYSTEM AUDIT BY ACCESS BY ACCESS CREATE SESSION BY ACCESS BY ACCESS CREATE USER BY ACCESS BY ACCESS ALTER USER BY ACCESS BY ACCESS DROP USER BY ACCESS BY ACCESS PUBLIC SYNONYM BY ACCESS BY ACCESS DATABASE LINK BY ACCESS BY ACCESS ROLE BY ACCESS BY ACCESS PROFILE BY ACCESS BY ACCESS CREATE ANY TABLE BY ACCESS BY ACCESS ALTER ANY TABLE BY ACCESS BY ACCESS DROP ANY TABLE BY ACCESS BY ACCESS CREATE PUBLIC DATABASE LINK BY ACCESS BY ACCESS GRANT ANY ROLE BY ACCESS BY ACCESS SYSTEM GRANT BY ACCESS BY ACCESS ALTER DATABASE BY ACCESS BY ACCESS CREATE ANY PROCEDURE BY ACCESS BY ACCESS ALTER ANY PROCEDURE BY ACCESS BY ACCESS DROP ANY PROCEDURE BY ACCESS BY ACCESS ALTER PROFILE BY ACCESS BY ACCESS DROP PROFILE BY ACCESS BY ACCESS GRANT ANY PRIVILEGE BY ACCESS BY ACCESS CREATE ANY LIBRARY BY ACCESS BY ACCESS EXEMPT ACCESS POLICY BY ACCESS BY ACCESS GRANT ANY OBJECT PRIVILEGE BY ACCESS BY ACCESS CREATE ANY JOB BY ACCESS BY ACCESS CREATE EXTERNAL JOB BY ACCESS BY ACCESS

So you must put in place a purging policy for this table.

Without AUDIT_TRAIL

The only drawback of the previous solution is that you have to restart the database. And maybe two times because after problem solved you would like to deactivate auditing. This is most probably not reliable solution on a production database so I have been looking for a better solution with no database reboot.
I initially thought of the AFTER LOGON trigger but you need to be logged-in and the BEFORE LOGON does not exits. Then at same documentation place I found the AFTER SERVERERROR trigger and decided to give it a try.
First I created a dummy table to log server error (columns inherited from dba_audit_session dictionary table):

CREATE TABLE sys.logon_trigger ( USERNAME VARCHAR2(30), USERHOST VARCHAR2(128), TIMESTAMP DATE );

Second I created below trigger:

CREATE OR REPLACE TRIGGER sys.logon_trigger AFTER SERVERERROR ON DATABASE BEGIN IF (IS_SERVERERROR(1017)) THEN INSERT INTO logon_trigger VALUES(SYS_CONTEXT('USERENV', 'AUTHENTICATED_IDENTITY'), SYS_CONTEXT('USERENV', 'HOST'), SYSDATE); COMMIT; END IF; END; /

Then third simulated a wrong password access with my account and issued:

SQL> ALTER SESSION SET nls_date_format='dd-mon-yyyy hh24:mi:ss';   SESSION altered.   SQL> SET lines 200 SQL> col USERHOST FOR a30 SQL> SELECT * FROM sys.logon_trigger ORDER BY TIMESTAMP DESC;   USERNAME USERHOST TIMESTAMP ------------------------------ ------------------------------ -------------------- yjaquier ST\GVADT30596 23-nov-2012 11:05:56

Tablespace not displaying in Application designer to save records

Below SQL will help to resolve the Tablespace not displaying in Application designer to save records

INSERT INTO PSTBLSPCCAT
SELECT DISTINCT DDLSPACENAME, DBNAME ,'R',' ', '' FROM PSRECTBLSPC A
WHERE TEMPTBLINST = 'N'
AND NOT EXISTS (SELECT 'X' FROM PSTBLSPCCAT
WHERE DDLSPACENAME = A.DDLSPACENAME
AND DBNAME = A.DBNAME);

JSL/WSL (jolt/Work Listener) failed to start with error Could not establish listening address on network

These error messages indicate that the port number that the Tuxedo process is attempting to start listening on is either in use, invalid, or the hostname/IP address specified for that address is invalid.

The Application Server will spawn new processes when it boots up based on the Min and Max Handlers for the Workstation Listener and Jolt Listener. Make sure you have your ports spread out between the WSL and JSL to accommodate the Max Handlers value for the Workstation Listener. For example if you are using port 7000 for WSL and have a Max Handlers value of 5, ports 7001 - 7005 will be used as well. You will want to set the JSL to no less then 7006

For example appserver use port 9618 and spaned 5 more in use and it fails when rebooting.Find the spawned process in use using netstat -lpn and get the PID - kil the PID and start the appserver

netstat -lpn | grep 961*

tcp        0      0 10.137.4.25:9619            0.0.0.0:*                   LISTEN      16347/JSH          
tcp        0      0 10.137.4.25:9620            0.0.0.0:*                   LISTEN      16348/JSH          
tcp        0      0 10.137.4.25:9621            0.0.0.0:*                   LISTEN      16349/JSH          
tcp        0      0 10.137.4.25:9622            0.0.0.0:*                   LISTEN      16350/JSH          
tcp        0      0 10.137.4.25:9623            0.0.0.0:*                   LISTEN      16351/JSH


[PS_HOME/appserv]$ kill -9 16347 16348 16349 16350 16351

Query to find user security access by navigation

select a.oprid as OPRID, a.oprdefndesc as NAME, b.rolename as ROLENAME, c.descr254 as NAVIGATION, c.descr150_mixed as DESCRIPTION
from psoprdefn a, psroleuser b, psroleclass d, psauthitem e, ps_authitem_cmp c
where a.oprid = b.roleuser
  and b.rolename = d.rolename
  and d.classid = e.classid
  and e.menuname = c.menuname
  and e.barname = c.barname
  and e.baritemname = c.baritemname
  and e.pnlitemname = c.pnlitemname
  and a.oprclass like 'PP_UK%'
  and a.acctlock = 0
order by a.oprid, b.rolename;

Maintaining Security in PeopleSoft Upgarde

In order to preserve security through your upgrade passes, you will have 3 options.

1) Low Risk - Re-enter your security changes into one of the target databases. Once this is complete, those security tables can be exported out and imported into any other upgraded databases at the same tools release using security migration reDocument:610138.1, included below.

2) Low Risk - Re-do the Initial Upgrade again. This will get the security as it is when you take the copy of production. Many customers chose this option because they are most comfortable with it. Once the initial upgrade is completed, follow the steps outlined in reDocument:610138.1 (Included below) to migrate security to your other upgraded databases.

3) High Risk -
Attention! This workaround has not gone through our formal quality regression test cycle. We strongly recommend that you thoroughly test this workaround in a development environment before applying it to your production environment.

Be sure to document this change as this workaround may be detected during your next upgrade. Because this workaround has not yet gone through our formal quality regression test cycle, this workaround will have risk.

You should only follow this option if you only want the latest operator security in production migrated to your upgraded database. Do the Initial Upgrade again, up through the end of Chapter 2 (Updating PeopleTools). You will need to definitely run the Relnnn.sql scripts, copy the projects and do the alters - but to make sure you get everything correct, it would be best to run all steps in Chapter 2. Once completed your tools release will be at the same level as your fully upgraded database that you want operator security imported into. DO NOT FOLLOW Document:610138.1 as you will LOSE ALL security delivered by the new application release you just upgraded to. To migrate your operator security, select the appropriate Data Mover script for your PeopleTools release to export and import operator security:

For PT 8.4x -

******Export******
USEREXPORT.dms

This script looks as follows:

-- USERS
EXPORT PSOPRDEFN;
EXPORT PSOPRALIAS;
EXPORT PSROLEUSER;
EXPORT PSUSERATTR;
EXPORT PSUSEREMAIL;
EXPORT PSUSERPRSNLOPTN;
EXPORT PS_ROLEXLATOPR;
EXPORT PS_RTE_CNTL_RUSER;

******Import******
USERIMPORT.dms

This script looks as follows:

UPDATE PSLOCK SET VERSION = VERSION + 1 WHERE OBJECTTYPENAME = 'UPM';

REPLACE_DATA *;

UPDATE PSVERSION SET VERSION = VERSION + 1 WHERE OBJECTTYPENAME = 'SYS';
UPDATE PSVERSION SET VERSION = VERSION + 1 WHERE OBJECTTYPENAME = 'UPM';

UPDATE PSOPRDEFN SET VERSION = (SELECT VERSION FROM PSVERSION WHERE OBJECTTYPENAME = 'UPM');

Steps for PopleTools 8.5X PIA and Weblogic Installation with JRockit

1) To download the latest WebLogic maintenance pack (10.3.6) for PeopleTools 8.52, do the following:


a. Go to the following URL for the “Oracle Software Delivery Cloud site (aka eDelivery): http://edelivery.oracle.com/

b. Follow instructions for filling in personal info and confirm acceptance of terms, etc. Then click 'Continue' button

c. On the "Media Pack Search" page, click on the first ("Product Pack") drop down list, and choose "Oracle Fusion Middleware".

d. In the second, "Platform" drop down list, chose Linux x86-64

e. After making your choices, click the "Go" button.

f. Click the 'Oracle Fusion Middleware 11g Media Pack for Linux x86-64' hyperlink

g. Then click the 'Download' button next to 'Oracle WebLogic Server 11gR1 (10.3.6) Generic and Coherence' (part# is V29856-01)

h. After the file is downloaded, unzip it.



2) To download the latest JRockit:

a. Go to My Oracle Support through http://support.oracle.com and log in

b. Once logged in, click the "Patches & Updates" tab.

d. Select "Product or Family (Advanced Search)". (on top right corner of page)

e. Enter the following info:

Product is: "Oracle JRockit"

Release is "Oracle JRockit 28.2.5"

Platform is: "Linux x86-64"

f. Click "Search" to continue. You will see a list of the JRockit patches available.

Please select "JDK160 ORACLE JROCKIT R28.2.5" (Make sure you get JDK160, and NOT JDK150!).

g. Click "Download", and save the patch to your local drive. The file name will be in *.zip format. For example: p14261101_2825_Linux-x86-64.zip



3) To install WebLogic 10.3.6 and the new JRockit:

a) Unzip the JRockit zip file (from step 3) to the directory where you want it installed. (I'd suggest you install it in the same directory where you are installing WebLogic
c) Now install WebLogic 10.3.6. Follow the instructions in the PeopleTools 8.52 Installation guide

(see section: "Installing Oracle WebLogic on Linux or UNIX)



4) Now try to re-install the WebLogic PIA

Linux kernal parameters for PeopleTools 8.52

Below Linux kernal parameter has to be set to for PeopleTools 8.52 installtion on Linux else the the process with fail to start with less allocated memory


Add the following lines to /etc/sysctl.conf


# Added for PS
kernel.sem = 256 32000 128 192


# Added for PS
kernel.msgmni = 2048



# Added for PS
net.ipv4.ip_local_port_range = 1024 65000



to activate:

/sbin/sysctl -p



to retain settings after booting add the following line to /etc/rc.local

/sbin/sysctl -p