Faster Weblogic Server Startup on Linux VM

Problem

Weblogic server running on a Linux Virtual Machine is stuck while startup. For instance a Oracle SOA 12.2.1 weblogic manage server may take upto 12 minutes to startup on a Linux VM. A Weblogic 12C manage Server running nothing may take upto 4 minutes to start. The virtual machines are all equipped with enough CPUs and memory. 

Applies To 

Weblogic 12.1.x,12.2.x
RHEL X86,X86_64 Virtual Machines

Cause

Linux has two devices to provide random data at any time: /dev/random and /dev/urandom. Both ways should be secure enough to use them in generating PGP keys, ssh challenges, and other applications where secure random numbers are required. Starting on kernel 2.6, default entropy is 4096 bits and problem arises when the entropy available on the system is minimum (around 100 bits or less).

How to verify if you are encountering this issue?

1. Check the default system entropy

$ cat /proc/sys/kernel/random/poolsize 
4096

2. Check the available entropy.

$ cat /proc/sys/kernel/random/entropy_avail 
160

3. On previous example, entropy is too low.

Monitor the current entropy of the system by using the following command:


$ for i in $(seq 500); do cat /proc/sys/kernel/random/entropy_avail ; sleep 5; done

4. Start a WebLogic server instance. You should see that entropy decreases or stalls (use script in step 3)

Solution

1. Temporary Solution (Use for testing purpose)

Start the WLS Server with below startup arguments.

-Djava.security.egd=file:/dev/./urandom

Override the JAVA_OPTIONS environment variable before starting WebLogic Server via shell scripts.

export JAVA_OPTIONS="${JAVA_OPTIONS} -Djava.security.egd=file:/dev/./urandom"

Start the Weblogic Server and note the timings!


2. Permanent Solution (Use if Step 1 works)

If the above solution works it is time to setup the fix permanently in the env. The fix can be applied in the JAVA_HOME the Weblogic server refers to.

i.   Edit the Java Security Properties file ($JAVA_HOME/jre/lib/security/java.security)

ii.  The securerandom.source property specifies the source of seed data for secure random.

Change

securerandom.source=file:/dev/random

To

securerandom.source=file:/dev/urandom

iii.  Save changes and start the WebLogic Server instances.

Observation

We could see the startup tiings for Weblogic server improved dramatically. For instance the SOA Manage Server now took only less than 3 minutes as against 12 minutes before the fix was applied! The bare Weblogic manage servers took less tha 20 seconds to start !

References

How to Diagnose a Linux Entropy Issue on WebLogic Server Instances (Doc ID 1574979.1)

Starting OHS 12c via startComponent takes several minutes on a Linux Virtual Machine - SOLVED

I recently started working with Oracle HTTP server on a Linux VM. On trying to start the OHS using the startComponent script I was surprised to observe the startup duration was over 8 minutes! The same took seconds in 11g version of the same software. After few research on metalink I could solve the issue. I am providing the solution below.

Issue Applies To

The issue described and solution offered applied to below Oracle FMW Components.

Oracle HTTP Server 12.2.1 Installed on RHEL 6 Virtual Machine. The same issue/solution applies to OHS 12.1. and later.

Issue Description

Starting from OHS 12C, the OHSservices are monitored/managed by Node Manager. Once the Node manager is started ,the OHS service is started using startComponent script.While trying to do the script gets stuck  for minutes (8 Minutes in my case!)

$DOMAIN_HOME/bin/startComponent.sh ohs1

    Starting system Component ohs1 ...

    Initializing WebLogic Scripting Tool (WLST) ...

    Welcome to WebLogic Server Administration Scripting Shell

    Type help() for help on available commands

    Reading domain from
     Here it just sits for several minutes 
     
    Connecting to Node Manager ...
    Successfully Connected to Node Manager.
    Starting server ohs1 ...

Cause 

The problem is due to random number generation (entropy) on the Linux VM. More on the entropy issues can be found in Metalink note NOTE:1574979.1.The vm was running out of entropy. After changing where Java got its random numbers from, the startup time came down manifold!

Solution 

1) Stop OHS
stopComponent.sh ohs1

2) Backup and edit java.security
$ORACLE_HOME/oracle_common/jdk/jre/lib/security/java.security

3) Change securerandom.source
From:
securerandom.source=file:/dev/urandom
(In 12.2.1 this is securerandom.source=file:/dev/random)
To:
securerandom.source=file:/dev/./urandom

4) Start OHS
startComponent.sh ohs1

Observation

Hurray! The startup time for OHS now takes close to 25 seconds compared to 8 minutes earlier!

References

NOTE:2006106.1 - Starting OHS 12c via startComponent takes several minutes on a Virtual Machine

NOTE:1574979.1 - How to Diagnose a Linux Entropy Issue on WebLogic Server Instances