Issues and Resolution during Oracle AIA 2.4 installation on Weblogic 9.2 MP3 and Oracle SOA 10.1.3.4 MLR 9

This year January I was assigned to a project where I was supposed to implement Oracle SOA suite on top of Weblogic 9.2.I have already posted the steps to do so in my earlier posts. Another challenge was to implement Oracle AIA 2.4 on top of this setup ie on Weblogic 9.2 and SOA 10.1.3.4.

I call it a challenge because as always Oracle document looks neat but when you actually try to implement AIA 2.4 on Weblogic as per the Oracle document expect to hit numerous roadblocks or issues. I had to work a lot with Oracle support and finally was able to install AIA 2.4 on Weblogic 9.2 (single as well as clustered environment).


In the below post I will be talking about some of the issues faced by us during Oracle AIA 2.4 installation on Weblogic 9.2 and their resolution.

Issue 1:


While starting the manual install of AIA 2.4 on Weblogic 9.2 and SOA 10.1.3.4 the following error was obtained:


jdbc:oracle:thin:@mytestdbserver.soumya.com:1521:DVEIP
[trycatch] Caught exception: java.sql.SQLException: ORA-01017: invalid username/password; logon denied
[sql] Executing commands
[sql] Failed to execute: create user aia identified by aia
[sql] java.sql.SQLException: ORA-28003: password verification for the specified password failed
[sql] ORA-20001: Password same as or similar to user

Cause:
The Database security team has implemented the security policies, due to which AIA install script was not able to create the aia user with the same password ie aia .This was verified by logging into the databse as sys user and then trying to create aia user with same password.Look below

SQL> create user aia identified by aia;
create user aia identified by aia
*
ERROR at line 1:
ORA-28003: password verification for the specified password failed

Resolution:
Work with the DBA team and disable such security restrictions. They can enable the restriction after AIA installation.

Issue 2:



While trying to install AIA Foundation Pack 2.4 on top of SOA+Weblogic 9.2 MP3 the FPWLInstall script was failing with the below error.

ant --noconfig -buildfile FPWLInstall.xml


CreateDatasources:
CreateCommonDatasources:
createConnectionPool:
echo creating ConnectionPool AIAConnectionPool


BUILD FAILED
/u07_00/aia_home/Infrastructure/install/wlscripts/FPWLInstall.xml:218: The following error occurred while executing this line:
/u07_00/aia_home/Infrastructure/install/wlscripts/FPWLInstall.xml:225: The following error occurred while executing this line:
/u07_00/aia_home/Infrastructure/install/wlscripts/FPWLConfiguration.xml:107: The following error occurred while executing this line:
/u07_00/aia_home/Infrastructure/install/wlscripts/FPWLConfiguration.xml:64: The following error occurred while executing this line:
/u07_00/aia_home/Infrastructure/install/wlscripts/FPWLCommonConfig.xml:187: The following error occurred while executing this line:
/u07_00/aia_home/Infrastructure/install/wlscripts/FPWLCommonConfig.xml:122: Unable to create mbean: Could not create provider JDBCConnectionPool


Cause:
The proxy value was set incorrectly in the arguments of the SOA Server.


Resolution:
To fix that, please do the following:

a. Log in to WebLogic Administration console http://:port/console.
b. Click Lock and Edit.
c. Navigate to Environments-Servers-OracleSOAServer(Managed server where SOA is hosted)-Server Start tab.
d. In case you have proxy server, please change the value of proxyhost to the correct URL.
e. In case you don't have proxy server(as in my case), just incluse -Dhttp.proxySet=false in the arguments and remove the proxyhost parameter.
f.Restart the managed server and restart installing AIA.


Sample:

Xbootclasspath/a:/u01/apps/oracle/product/soacore/integration/esb/lib/esbprotocol.jar -XX:MaxPermSize=512m -Dhttp.nonProxyHosts=*.oracle.com*.oraclecorp.com localhost 127.0.0.1
10.177.255.135 sraghura-pc sraghura-pc.idc.oracle.com -Dhttp.proxyPort=80 -Dhttp.proxySet=false -Doracle.home=/u01/apps/oracle/product/soacore -Dorabpel.home=/u01/apps/oracle/product/soacore/bpel -Djavax.xml.parsers.DocumentBuilderFactory=oracle.xml.jaxp.JXDocumentBuilderFactory -Djavax.xml.parsers.SAXParserFactory=oracle.xml.jaxp.JXSAXParserFactory -Djavax.xml.transform.TransformerFactory=oracle.xml.jaxp.JXSAXTransformerFactory -Djavax.xml.soap.MessageFactory=oracle.j2ee.ws.saaj.soap.MessageFactoryImpl -Djavax.xml.soap.SOAPFactory=oracle.j2ee.ws.saaj.soap.SOAPFactoryImpl -Djavax.xml.soap.SOAPConnectionFactory=oracle.j2ee.ws.saaj.client.p2p.HttpSOAPConnectionFactory -Djavax.xml.soap.SOAPElementFactory=oracle.j2ee.ws.saaj.soap.SOAPFactoryImpl -Dbpel.platform=weblogic_8 -Doraesb.home=/u01/apps/oracle/product/soacore/integration/esb -Dprimary_oc4j=false -DsoaPlatform=weblogic_8 -Dhttp.webdir.enable=false -Daq.jms.enabled=true -DTrackingEnabled=Y -Dweblogic.rjvm.enableRouterFix=true -Daia.home=/u01/apps/oracle/product/AIA_24 -Djavax.xml.rpc.ServiceFactory=oracle.j2ee.ws.client.ServiceFactoryImpl

Issue 3:


AIA 2.4 installation on Weblogic 9.2 fails with the following error:

[echo] --------------------------------------------------------------
[echo]
Deploying bpel process AIAErrorTaskAdministrationProcess on eipsoad1.biogenidec.com, port 8892
[echo] --------------------------------------------------------------
[echo]
[deployProcess] Deploying process


/u01/apps/oracle/product/AIA_24/Infrastructure/ErrorHandling/AIAErrorTaskAdministrationProcess/output/bpel_AIAErrorTaskAdministrationProcess_1.0.jar

BUILD FAILED
/u01/apps/oracle/product/AIA_24/Infrastructure/install/wlscripts/FPWLInstall.xml:218: The following error occurred while executing this line:
/u01/apps/oracle/product/AIA_24/Infrastructure/install/wlscripts/FPWLInstall.xml:226: The following error occurred while executing this line:
/u01/apps/oracle/product/AIA_24/Infrastructure/install/wlscripts/FPWLDeployment.xml:17: The following error occurred while executing this line:
/u01/apps/oracle/product/AIA_24/Infrastructure/install/wlscripts/FPWLDeployment.xml:29: The following error occurred while executing this line:
/u01/apps/oracle/product/AIA_24/Infrastructure/install/wlscripts/FPWLErrorHandlingInstall.xml:138: The following error occurred while executing this line:


/u01/apps/oracle/product/AIA_24/Infrastructure/ErrorHandling/AIAErrorTaskAdministrationProcess/build.xml:204: A problem occured while
connecting to server "eipsoad1.biogenidec.com" using port "8892": bpel_aiaerrortaskadministrationprocess_1.0.jar failed to deploy. Exception
message is: weblogic.transaction.internal.TimedOutException: Transaction timed out after 29 seconds
BEA1-003F9A96E4CE965ECCFA at com.oracle.bpel.client.util.ExceptionUtils.handleServerException(ExceptionUtils.java:94)


Cause and Solution:


786431.1 WebLogic Server 9.2 JTA Timeout values for OSM. The Note provide steps to increase the JTA default timeout.


Quite often the Java Transaction API (JTA) is set to the default timeout value for many OSM development/testing installations. The default value is 30 seconds and when there are bottlenecks in the system, many OSM java transactions cannot finish within 30 seconds. As a result, the user will see errors


Ideally the root cause of the bottleneck should be identified and resolved accordingly. However sometimes it is necessary to increase the JTA value, for example :


a) the nature of transaction will take longer than 30 seconds
b) it takes time to fix system bottleneck and a quick fix is needed.


1. Go to the BEA console
2. Expand Services under "Domain Structure"
3. Click JTA
4. Click "Lock & Edit"
5. Change "Timeout Seconds:" from value 30 to 300


This change will increase JTA timeout from 30 seconds to 300 seconds, if after this change you don't see the Transaction time out anymore, this means JTA is set to an adequate value ( you may consider lowering the JTA timeout to a smaller value).


300 is an arbitrary number, System Administrators can choose a number between 30 to 600. Please be aware that large JTA timeout values might make system stall when there are many hanging threads. The goal is find optimal value for JTA timeout so that the user will see no / very few JTA timeout exceptions in a day.

Restart FPWLInstall script to install AIA 2.4

Issue 4:




The FPWLInstall execution fails with the following error:

DeployESBSuitcase:
[echo]
[echo] --------------------------------------------------------------
[echo]
Registering ESB service
/u01/apps/oracle/product/AIA_24/Infrastructure/install/systems/AIASystemFoundation on eipsoad1.biogenidec.com, port 8892
[echo] --------------------------------------------------------------
[echo]
[deployESBSuitcase] Deployment Attempt Response :
[deployESBSuitcase]
[deployESBSuitcase]
[deployESBSuitcase] An unhandled exception has been thrown in the ESB system. The exception reported is: "oracle.tip.esb.console.exception.ConsoleTransactionException: File repository not initialized. May be ESB bootstrap failed
[deployESBSuitcase] Please review ESB prameters for their correctness.


or

DeployESBSuitcase:
[echo]
[echo] --------------------------------------------------------------
[echo]
Registering ESB service
$AIA_HOME/Infrastructure/install/systems/AIASystemFoundation on , port
[echo] --------------------------------------------------------------
[echo]
[deployESBSuitcase] Deployment Attempt Response :
[deployESBSuitcase]
[deployESBSuitcase]
[deployESBSuitcase] Designtime cache has not been initialized
[deployESBSuitcase] Please look in logs for following signs of failure. Fix them and restart. (a)
Database access errors (b) ESB Bootstrap errors (c) OC4J class load errors (d)
Product installation errors (e) Export ESB params and verify if host and port parameters are correct. Please contact Oracle Support if unable to fix the issue.
[deployESBSuitcase]
[deployESBSuitcase]
[deployESBSuitcase]

Cause:


The customer is facing the issue documented in an internal documentation bug. This has been found to be due to the fact that esbprotocol.jar is not found in the Weblogic Bootstrap classpath. When the esbprotocol.jar is not loaded by the Bootstrap classpath, the Slide initialization fails to complete as it cannot resolve the "esb:///" artifacts.
Fix:
1. Find out the jdk used by WebLogic, to do this run the startWeblogic.sh that is located under /project_user/domains/
2. Copy the esbprotocol.jar that is under the JAVA_HOME found on the previous step,
location:\jre\lib\ext directory.






Issue 5


Error faced during AIA 2.4 installation on weblogic 9.2

(Refer to FPWLInstall10282009-013503.log the Dev server AIA install log path:)

DeployESBSuitcase:


[echo]
[echo] --------------------------------------------------------------
[echo]
Registering ESB service /u01/apps/oracle/product/AIA_24/Infrastructure/CAVS/CAVSESB on eipsoad1.biogenidec.com, port 8892
[echo] --------------------------------------------------------------
[echo]
[deployESBSuitcase] Deployment Attempt Response :
[deployESBSuitcase]
[deployESBSuitcase] Entity Deployment Failed
[deployESBSuitcase]
[deployESBSuitcase]
[deployESBSuitcase] WSDL "http://eipsoad1.biogenidec.com:8892/AIAValidationSystemServlet/asyncrequestrecipient?wsdl" of service "AIASystem.ValidationSystem.AsyncRequestRecipient" is invalid or unaccessible. Following error is thrown by WSDL Parser : WSDL Parsing
Failed: Response: '404: Not Found' for url: 'http://eipsoad1.biogenidec.com:8892/AIAValidationSystemServlet/asyncrequestrecipient?wsdl'
[deployESBSuitcase] Ensure that the WSDL is accessible and valid. If the problem persists, Contact Oracle Support.
[deployESBSuitcase]
[deployESBSuitcase]
[deployESBSuitcase]


Cause:


The issue is caused because the AIAValidationSystemServlet was not deployed to the weblogic server. This servlet is part of the AIAApplication. The log file showed the following error to:


[wldeploy] Deployment Plan: null
[wldeploy] App root: /tmp/oracle/./config/deployments/AIAApplication
[wldeploy] App config: /tmp/oracle/./config/deployments/AIAApplication/plan
[wldeploy] Deployment Options:
{isRetireGracefully=true,isGracefulProductionToAdmin=false,isGracefulIgnoreSessions=false,rmiGracePeriod=-1,retireTimeoutSecs=-1,undeployAllVersions=false,archiveVersion=null,planVersion=null,isLibrary=false,libSpecVersion=null,libImplVersion=null,stageMode=null,clusterTimeout=3600000,
altDD=null,altWlsDD=null,name=AIAApplication,securityModel=null,securityValidationEnabled=false,versionIdentifier=null,isTestMode=false,forceUn
deployTimeout=0,defaultSubmoduleTargets=true,timeout=0}


[wldeploy] [BasicOperation.execute():421] : Initiating deploy operation for app, AIAApplication, on targets:
[wldeploy] [BasicOperation.execute():423] : OracleSOAServer
[wldeploy] Task 11 initiated: [Deployer:149026]deploy application AIAApplication on OracleSOAServer.
[wldeploy] Task 11 failed: [Deployer:149026]deploy application AIAApplication on OracleSOAServer.
[wldeploy] Target state: deploy failed on Server OracleSOAServer
[wldeploy] oracle/binding/DataControl


This shows that the AIADeployment application deployment failed. The weblogic server log file shows the error ClassNotFoundError: oracle/binding/DataControl. This class is part of the ADF installation. The cause of this issue is an improper installation of ADF.




Fix:
Make sure the ADF installation is performed properly and that the ADF jar files are added to the CLASSPATH of the server were you deploy the AIAApplication.The best way to add the ADF jar files using the WebLogic Console. Follow the steps below to do so:


1) Connect to the middleware server
2) Set your corrunt directory to /user_projects/
cd /user_projects//

3) Convert the file setupadf.sh from Windows format to Unix format by running the command below:


Copy setupadf.sh to setupadfTmp.sh
dos2unix setupadf.sh setupadfTmp.sh
4) Replace relative paths to the ADF jar files with absolute ones, you could use the bash shell command below:
baseDir=$(echo `pwd`
sed 's/\(^.*\)\/user_projects.*/\1/g'
sed 's/\//\\\//g');
sed "s/..\/..\/../$baseDir/g" setupadfTmp.sh > setupadfUnix.sh
5) Source the file setupadfUnix.sh that is located on the directory /user_projects//setupadfUnix.sh
. setupadfUnix.sh

6) Print the value for the CLASSPATH environment variable
i.e. echo $CLASSPATH
7) Connect and login to the WebLogic console:
i.e. Open a browser and navigate to: http://:/console


8) Expand the -> Environment -> Servers node on the Domain Structure section of the Web Logic Home page
9) Click on the Server where you are installing aia (i.e. OracleSoaServer)
10) Click on the Configuration -> Server Start tab
11) Click on the Lock & Edit Button that is located on the Change Center section of the console
12) Append the output of the command you executed in step 2 to the CLASSPATH environment variable.
13) Click on the Release Configuration button located on the Change Center section of the console.


For any troubleshooting please refer to document id 822831.1