Applies to:
Oracle Weblogic Server - Version: 10.3 to 10.3.3Information in this document applies to any platform.
Symptoms
A stateless session bean (SLSB) is deployed in WebLogic 10.3.0. The SLSB only has one bean instance in the pool (max-beans-in-free-pool=1and initial-beans-in-free-pool=1).After several hours and with over 100000 incoming requests the bean instance goes into waiting state.
Since there is only one bean instance in the pool, this effectively hangs all incoming calls.
The thread dump shows the bean instance goes into waiting state:
"[ACTIVE] ExecuteThread: '13' for queue: 'weblogic.kernel.Default (self-tuning)'" daemon prio=10 tid=0149e600 nid=48 lwp_id=5603464 waiting on condition [6c240000]
java.lang.Thread.State: TIMED_WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <89861118> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:198)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2054)
at weblogic.ejb.container.pool.StatelessSessionPool.waitForBean(StatelessSessionPool.java:269)
at weblogic.ejb.container.pool.StatelessSessionPool.getBean(StatelessSessionPool.java:111)
at weblogic.ejb.container.manager.StatelessManager.preInvoke(StatelessManager.java:148)
at weblogic.ejb.container.internal.BaseRemoteObject.preInvoke(BaseRemoteObject.java:227)
at weblogic.ejb.container.internal.StatelessRemoteObject.preInvoke(StatelessRemoteObject.java:52)
at com.tetrapak.ppm.external.xml.bean.UMGMessageLogFacade_n73y0z_EOImpl.isMachineValid(UMGMessageLogFacade_n73y0z_EOImpl.java:261)
at com.tetrapak.ppm.external.xml.bean.UMGMessageLogFacade_n73y0z_EOImpl_WLSkel.invoke(Unknown Source)
at weblogic.rmi.internal.BasicServerRef.invoke(BasicServerRef.java:589)
at weblogic.rmi.cluster.ClusterableServerRef.invoke(ClusterableServerRef.java:230)
at weblogic.rmi.internal.BasicServerRef$1.run(BasicServerRef.java:477)
at weblogic.security.acl.internal.AuthenticatedSubject.doAs(AuthenticatedSubject.java:363)
at weblogic.security.service.SecurityManager.runAs(Unknown Source)
at weblogic.rmi.internal.BasicServerRef.handleRequest(BasicServerRef.java:473)
at weblogic.rmi.internal.wls.WLSExecuteRequest.run(WLSExecuteRequest.java:118)
at weblogic.work.ExecuteThread.execute(ExecuteThread.java:201)
at weblogic.work.ExecuteThread.run(ExecuteThread.java:173)
java.lang.Thread.State: TIMED_WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <89861118> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:198)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2054)
at weblogic.ejb.container.pool.StatelessSessionPool.waitForBean(StatelessSessionPool.java:269)
at weblogic.ejb.container.pool.StatelessSessionPool.getBean(StatelessSessionPool.java:111)
at weblogic.ejb.container.manager.StatelessManager.preInvoke(StatelessManager.java:148)
at weblogic.ejb.container.internal.BaseRemoteObject.preInvoke(BaseRemoteObject.java:227)
at weblogic.ejb.container.internal.StatelessRemoteObject.preInvoke(StatelessRemoteObject.java:52)
at com.tetrapak.ppm.external.xml.bean.UMGMessageLogFacade_n73y0z_EOImpl.isMachineValid(UMGMessageLogFacade_n73y0z_EOImpl.java:261)
at com.tetrapak.ppm.external.xml.bean.UMGMessageLogFacade_n73y0z_EOImpl_WLSkel.invoke(Unknown Source)
at weblogic.rmi.internal.BasicServerRef.invoke(BasicServerRef.java:589)
at weblogic.rmi.cluster.ClusterableServerRef.invoke(ClusterableServerRef.java:230)
at weblogic.rmi.internal.BasicServerRef$1.run(BasicServerRef.java:477)
at weblogic.security.acl.internal.AuthenticatedSubject.doAs(AuthenticatedSubject.java:363)
at weblogic.security.service.SecurityManager.runAs(Unknown Source)
at weblogic.rmi.internal.BasicServerRef.handleRequest(BasicServerRef.java:473)
at weblogic.rmi.internal.wls.WLSExecuteRequest.run(WLSExecuteRequest.java:118)
at weblogic.work.ExecuteThread.execute(ExecuteThread.java:201)
at weblogic.work.ExecuteThread.run(ExecuteThread.java:173)
Changes
None.Cause
This is a known issue.java.util.concurrent.locks.Condition.await(txTimeOutMS, TimeUnit.MILLISECONDS) is called by all threads except one thread (Thread A) which has got the bean from the pool.
When Thread A releases the bean, it acquires lock and call Condition.signal() which should wake up one waiting thread. But, none of thread actually comes out of waiting state unless one of the waiting thread times out and throw RuntimeException.
Solution
To implement the solution, please execute the following steps:1. Download and install the patch using Smart Update
Pass ID: QE7V
Passcode: T5QP1CXU
2. Ensure that you have taken a backup of your system before applying the recommended patch.
3. Apply the patch in a test environment.
4. Retest the issue.
5. Migrate the solution as appropriate to other environments.
Grate post,for more info keep it update
ReplyDeleteOracle SOA Online Training