2012-11-23

General

Debugging Class Loading with AOP

In this article I’ll briefly discuss how Aspect Oriented Programming (AOP) using AspectJ can be used to debug class loading, in particular in instances where a system have multiple class loaders. I’ll assume a basic understanding of Java class loading and AOP.

Class loading in Java is often misunderstood and derided. I’ll not spend my time here defending it, but rather try to find a way to gain more information from multiple class loaders.

[alert type=”success”]The solution is not perfect, but should be used as an example or as a jumping-off point. I’m writing this because I like the problem domain, not because I necessarily want to solve all your problems…  :)[/alert]

 

You’ll find complete source code for this article in the resources at the end.

The Problem

In any non-trivial class loader hierarchy, there’s always a danger of mixing up the class files on deployment. Code deployed on an application server or similar systems will usually share classes. Using the diagram to the right we can easily envision “artifact 1” and “artifact 2” sharing some classes, but if the packaging is not exactly right, or if we have static dependencies, we may be hit with class loading difficulties

We should also remember that the called class loader will usually ask its parent class loader first, and only when the parent class loader returns null, look to its own classes. Using the diagram above as an example, if the class loader for “artifact 1” is asked for a class it will first ask the class loader “deploy” witch in its turn will ask “system”, so the actual lookup chain will be in this order: system > deploy > artifact 1.

ClassNotFoundException

This exception means that a class file could not be found when one of the following three methods were called:

ClassLoader.loadClass(String)
ClassLoader.findSystemClass(String)
Class.forName(String)

Given the string name the class loader will usually do a normal recursive search through the parents as described above and if the class file is not found, raise the exception.

NoClassDefFoundError

This is a linkage error when a class is implicitly loaded but cannot be found in runtime. In other words, the executing class was compiled with access to the missing class, but when the code is executing the class is missing. In that sense this error is usually caused by a class not found exception.

One common source of these errors is when a parent class loader attempts to load a class which depends implicitly on classes from a child class loader. This is often the case with static factory methods where an API defines a static method like this:

public static Foo loadFoo(String fooClass);

If we assume that this factory is loaded by our “deploy” class loader but we have dependencies in any of the artifacts, there are two plausible scenarios:

  • We call the factory and the class we supply as the argument is located in one of the artifact class loaders we will get a class not found because a parent class loader will not search, or even be aware of, its child class loaders.
  • We call the factory and the class we supply as the argument is indeed found either in “deploy” or its parent “system”, but that class in its turn depends on a class found only  in one of the artifact class loaders we will get a class def not found error.

Given this little crash course, why is this such a huge problem? The answer is often that we do not get enough information with the errors. It seems to me there are some basic things we would like to know when any of these problems occur:

  • If a class is not found by a class loader, what classes does it actually contain? Or rather, from where does it read its classes?
  • If a class is not found by a class loader, is it perhaps loaded by another class loader but one which it does not have access to?

This information can only be supplied by the class loaders themselves. Which is fine if you write your own server and can add debugging code as you please, but not so interesting if you use an application server of any sort.

Enter AspectJ

Now then, could we use AspectJ to help us out? After all, it is perfectly reasonable to define “points” in the executing code we can use to gather the information we need.

[alert type=”success”]I’m using AspectJ for this article, but it should be possible to write it for other AOP frameworks as long as they support runtime weaving.[/alert]

 

Let’s see then, in order to answer our two main questions above we need to:

  • Gather data when a class is loaded anywhere in the system, to
    • Maintain a map between class names and their loaders.
    • Keep a set of known resources for each class loader.
  • Intercept our two exceptions and print some debugging.

Pointcut: Loading a Class

Looking at the class loaders, it turns out that there’s a very handy pointcut we can make: the call to load a class in the class loader.

pointcut source(String s, ClassLoader l) : 
        target(ClassLoader+) && 
        target(l) &&
        args(s) &&
        call(Class loadClass(String));

This pointcut has two arguments, the name of the class being asked for, and the class loader which is called. It intercepts the call to “load class” in ClassLoader or any of its descendants (that’s what the ‘+’ means).

[alert type=”success”]I’m limiting myself to one method call (“load class”) here, feel free to extend it if need be.[/alert]

 

Gather Data

Using the pointcut above we can now solve our first problem, maintain our maps of debugging data. Well start with defining the advice

after(String s, ClassLoader l) returning (Class cl) : source(s, l) {
    // do something here...
}

Some things to note here, this advice will only be called upon normal execution, because it is defined as “returning”, so we don’t have to worry about exceptions. When the advice executes we will also have access to three context items, the called class loader, the name of the class and the loaded class.

I’ll not add all code here, you find the source code of this article in the resources at the end, but what we need to do in the above advice is to map the class name to the class loader, and to create or update a set of resources associated with the class loader to include the code source of the loaded class.

All classes loaded by a secure class loader should have a “code source”. This represents the location the class was loaded from, in other words a file path, a JAR archive etc. It is available in the protection domain given the class, so you can extract the exact code source URL like this:

CodeSource src = .getProtectionDomain().getCodeSource();
URL url = (src == null ? null : src.getLocation());

So in effect the advice will look like this:

after(String s, ClassLoader l) returning (Class cl) : source(s, l) {
    /*
     * map class name to class loader
     */
    registerClassLoader(cl, l);
    /*
     * find the class code source and add it to a 
     * set mapped to the class loader
     */
    registerSources(cl, l); 
}

Debugging: Catch the ClassNotFoundException

Now, let’s catch the class not found errors. We’ll do this using the same pointcut as above but with another advice:

after(String s, ClassLoader l) throwing (ClassNotFoundException e) : source(s, l) {
    // do something here...
}

This advice will only run when a class not found is raised by the call identified by the pointcut. We could catch the exception before it is handled (using the “handler” directive) but since we want to know the class loader anyway, this method is a bit more effective.

Printing the debug information here will be rather straightforward (again, the complete sources can be found at the end of this article):

after(String s, ClassLoader l) throwing (ClassNotFoundException e) : source(s, l) {
    /*
     * get the code sources for the given class loader
     */
    LoaderSources fail = getSourcesFor(l);
    /*
     * check if can find the class in any other class loader
     */
    ClassLoader real = classLoaders.get(s);
    if(real != null) {
        /*
         * get the resources for the class loader we found
         * and print debug information
         */
        LoaderSources found = getSourcesFor(real);
        DebugPrinter.printClassNotFound(s, fail, found);
    } else {
        DebugPrinter.printClassNotFound(s, fail);
    }
}

Pointcut: Trap the NoClassDefFoundError

The no class def found error is a runtime error making it hard to catch. In this article we’ll simplify things a bit by using the special point “handler” which can be used for advice execution just before an exception is handled by a catch clause.

pointcut noDef(NoClassDefFoundError e) : 
        args(e) && 
        handler(NoClassDefFoundError);

This pointcut simply adds a handler for the error and saves the error as an argument.

Debugging: Handle the NoClassDefFoundError

We’ll use a very simple advice form here:

before(NoClassDefFoundError e) : noDef(e) {
    // do something here...
}

As a no class def found is implicit, and hence can be thrown from method executions as well as calls, the weaving would be rather extensive, so we’ll limit ourselves by using the handler. The handler mechanism has one downside though, it’s hard to get to the class loader from which the problem originated. What we’ll do here is to use the “this” object available from the pointcut which will in effect be the object which contains the catch clause.

[alert type=”success”]By using the handler you will most likely have to use an additional step when you get a no class def found error, when you get the error add a catch for it as high in the call stack you can within your own code, which will make the “this” object in the advice to be much more relevant[/alert]

 

before(NoClassDefFoundError e) : noDef(e) {
    /*
     * the class name we're looking for is the 
     * message of the exception with path separators
     * instead of dots, so...
     */
    String name = e.getMessage();
    if(name != null) {
        name = name.replace(File.separatorChar, '.');
        /*
         * use the current object to try to find a relevant class
         * loader, the higher in the stack the exception is caught, the
         * more chance it is this class loader will be of use
         */
        ClassLoader fail = attemptToFindFail(thisJoinPoint.getThis());
        ClassLoader real = classLoaders.get(name);

        // print debugging here
    }
}

The Rest is Plumbing

Really, just add you own or download the sources. For your reference this is an example (somewhat abbreviated) output I got when I tested this is in a Tomcat 6 installation:

>>>ClassNotFoundException
Class 'se.xec.commons.path.StringPath' was not found in the called class loader. 
However, it was found in another class loader in the system.
 ---
Failed to find class in ClassLoader: WebappClassLoader
 - with parent class loader: org.apache.catalina.loader.StandardClassLoader@1a85d38
 - with registered code sources:
        file:/test/webapps/tester/WEB-INF/classes/test/Tester.class
 ---
Class found in ClassLoader: WebappClassLoader
 - with parent class loader: org.apache.catalina.loader.StandardClassLoader@1a85d38
 - with registered code sources:
        file:/test/webapps/finder/WEB-INF/classes/test/Tester.class
        file:/test/webapps/finder/WEB-INF/lib/xec.se-commons-1.4.4.jar
<<<

For this test I used two applications. One “finder” application that works as expected and loads the “StringPath” class from a JAR in its library folder, and one “tester” application that cannot find the string path object. As you can see the debug output answers our initial questions, what code source does the class loader know of, and where can the class be found if it exists in the system.

Some Gotcha's

  • Remember that in Java classes are loaded lazily, which means that if a class exists in a code location it does not automatically mean the class is loaded and hence may not be found even with our nice test harness.
  • Most application servers have their own variations of class loaders, so you may have to modify my suggestions on a case by case basis. For example, in order for this code to work on Tomcat 6, you need to trap the “find class” method instead of “load class” in the initial pointcut.
  • If your no class def found errors contains a stack trace, you may be able to traverse the stack frames within the debugging code - instead of adding an additional catch clause - to make the code more general.
  • Some class loader implementations uses class not found exceptions internally when a class is looked up in the parent chain, you may wish to add additional filtering to the debug code to clear up the output.
  • The aspect should only be instantiated once, so make sure it is declared with “issingleton”.

The End

This should have given you an idea of what AOP could help you with in regards to debugging class loading. I'll leave the details of using and configuring the load time weaver for a rainy day.

Lars J. Nilsson is a founder and Executive Vice President of Cubeia Ltd. He's an autodidact programmer, former opera singer, and lunch time poet. You can contact him at lars.j.nilsson in Cubeia's domain.

Resources

The source code for this article
http://www.cubeia.com/docs/articles/cl-debugging/debugging-src.jar

Demystifying class loading problems, Part 2
http://www.ibm.com/developerworks/java/library/j-dclp2.html

AspectJ documentation
http://www.eclipse.org/aspectj/doc/released/devguide/index.html