Understanding Out of Memory (OOM) in Weblogic

As Fusion or Weblogic  Admin one is supposed to encounter Out of Memory (OOM )issues now and then. Why talk about Fusion or Weblogic admins this issue can be encountered on multiple products like the Oracle Applications EBS, Hyperion, OTM, Oracle Application server and many other products .What seems common across all these products is the use of Application server.  All the above mentioned products use some or the other application server over which the applications are deployed.

For example the latest Oracle SOA Suite 11g uses Weblogic 10.3 Application Server while EBS Release 12 uses Oracle Application Server 10g. The point I am trying to make is the Out of Memory issue is not product specific but is related to Application Servers running Oracle-Sun JVMs. Application servers like Oracle Weblogic, IBM Websphere and Oracle Application Server facilitate running Java Virtual Machines on which the Java Applications can be deployed and executed. Out of memory issue is related to filled up space issues in these JVMs which makes it generic across all the above mentioned application servers.

Before understanding OOM let’s first understand the memory used by the JVM. The Java Virtual Machine (JVM) has the following types of memory:

 Heap memory is the runtime data area from which memory for all class instances and arrays is allocated

Non-heap memory includes the method area and memory required for the internal processing or optimization of the JVM. It stores per-class structures such as a runtime constant pool, field and method data, and the code for methods and constructors.

Native memory is the virtual memory managed by the operating system.

When the above mentioned memory is insufficient for a deployed application, a java.lang.OutOfMemoryError is thrown. There will be different error messages for OutOfMemoryErrors in each type of memory. They are listed below:

-          Heap memory error. When an application creates a new object but the heap does not have sufficient space and cannot be expanded further, an OutOfMemoryError will be thrown with the following error message:

java.lang.OutOfMemoryError: Java heap space

-          Non-heap memory error:When the permanent generation is full, the application will fail to load a class or to allocate an interned string, and an OutOfMemoryError will be thrown with the following error message:

java.lang.OutOfMemoryError: PermGen space

-          Native memory error: The Java Native Interface (JNI) code or the native library of an application and the JVM implementation allocate memory from the native heap. An OutOfMemoryError will be thrown when an allocation in the native heap fails. For example, the following error message indicates insufficient swap space, which could be caused by a configuration issue in the operating system or by another process in the system that is consuming much of the memory:

java.lang.OutOfMemoryError: request bytes for .

Out of swap space?

After having understood the OOM issue the question is why would the code encounter an OOM at all. This could be for more than one reason.

An insufficient memory problem could be due

1.       Either to a problem with the configuration ie the application really needs that much memory

2.       Or to a performance problem in the application that requires you to profile and optimize to reduce the memory use. Configuring memory settings and profiling an application to reduce the memory use are beyond the scope of this post and I would cover them in my forth coming posts.

The issues are mentioned below:

a.       Memory Leaks

The JVM is responsible for automatic memory management, which reclaims the unused memory for the application. However, if an application keeps a reference to an object that it no longer needs, the object cannot be garbage collected and will occupy space in the heap until the object is removed. Such unintentional object retention is referred to as a memory leak. If the application leaks large amounts of memory, it will eventually run out of memory, and an OutOfMemoryError will be thrown. In addition, garbage collection may take place more frequently as the application attempts to free up space, thus causing the application to slow down.

b.      Finalizers

Another possible cause of an OutOfMemoryError is the excessive use of finalizers. The java.lang.Object class has a protected method called finalize. A class can override this finalize method to dispose of system resources or to perform cleanup before an object of that class is reclaimed by garbage collection. The finalize method that can be invoked for an object is called a finalizer of that object. There is no guarantee when a finalizer will be run or that it will be run at all. An object that has a finalizer will not be garbage collected until its finalizer is run. Thus, objects that are pending for finalization will retain memory even though the objects are no longer referenced by the application, and this could lead to a problem similar to a memory leak.

c.       Deadlocks

A deadlock occurs when two or more threads are each waiting for another to release a lock. The Java programming language uses monitors to synchronize threads. Each object is associated with a monitor, which can also be referred as an object monitor. If a thread invokes a synchronized method on an object, that object is locked. Another thread invoking a synchronized method on the same object will block until the lock is released. Besides the built-in synchronization support, the java.util.concurrent.locks package that was introduced in J2SE 5.0 provides a framework for locking and waiting for conditions. Deadlocks can involve object monitors as well as java.util.concurrent locks.

Typically, a deadlock causes the application or part of the application to become unresponsive. For example, if a thread responsible for the graphical user interface (GUI) update is deadlocked, the GUI application freezes and does not respond to any user action.

d.      Looping Threads

Looping threads can also cause an application to hang. When one or more threads are executing in an  infinite loop, that loop may consume all available CPU cycles and cause the rest of the application to be unresponsive.

e.      High Lock Contention

Synchronization is heavily used in multithreaded applications to ensure mutually exclusive access to a shared resource or to coordinate and complete tasks among multiple threads. For example, an application uses an object monitor to synchronize updates on a data structure. When two threads attempt to update the data structure at the same time, only one thread is able to acquire the object monitor and proceed to update the data structure. Meanwhile, the other thread blocks as it waits to enter the synchronized block until the first thread finishes its update and releases the object monitor. Contended synchronization impacts application performance and scalability.

Hope the above article helps you to understand the basics of OOM issues in Application Servers. OOM issues could be a night mare for any admin given the cause is memory leak. But we will discuss methods to resolve such scenarios soon. Also I plan to post on ways to alter JVM settings on Weblogic server.

 Below are URLs to guides/documents that has been used by me for compilation of above article:

Monitoring and Managing Java SE 6 Platform Applications (http://java.sun.com/developer/technicalArticles/J2SE/monitoring/)