Good Objects Breaking Bad

The purpose of this blog post is to look at two typical problems that may arise when different versions of the same class may be loaded by the JVM. Abstractly speaking, I’m going to examine the following two scenarios in detail:

1. Two versions of the same class, but originating from different class loaders, come in contact with each other.
2. There are two versions of the same class available on the class path. Only one gets loaded, but it might be hard to impossible to tell which.

As the last scenario is far more common, I’m going to discuss it first. It often leads to seemingly invalid NoSuchMethodErrors. To understand why that happens, here is how this issue can be reproduced easily:

1. Create a library, Lib1.jar, that contains exactly this class:

   package com.wordpress.mlangc;
   
   public class GoodObject {
      
   }
   

2. Create another library, Lib2.jar, that contains an extended version of GoodObject:

   package com.wordpress.mlangc;
   
   public class GoodObject {
       public int getExcitingNewFeature() {
           return 42;
       }
   }
   

3. Create a third library, Lib3.jar, that depends on Lib2.jar and contains this nifty peace of code:

   package com.wordpress.mlangc;
   
   public class BetterObject {
       private GoodObject goodObject = new GoodObject();
       
       public int doSomethingAwfullyExcitingAndNew() {
           return goodObject.getExcitingNewFeature();
       }
   }
   

4. Now add the libraries you’ve just created, Lib1.jar, Lib2.jar and Lib3.jar, to your class path and run the following:

   package com.wordpress.mlangc;
   
   public class B0rkage {
       public static void main(String[] args) {
           System.out.println("Feel free to try this at home: " 
               + new BetterObject().doSomethingAwfullyExcitingAndNew());
       }
   }
   

   You should see an error message like

   Exception in thread "main" java.lang.NoSuchMethodError: com.wordpress.mlangc.GoodObject.getExcitingNewFeature()I
   	at com.wordpress.mlangc.BetterObject.doSomethingAwfullyExcitingAndNew(BetterObject.java:7)
   

   If not, make sure that you’ve added the libraries to the class path in the “correct” order. Indeed, if Lib2.jar is loaded before Lib1.jar, like here

   java -cp "${deps}/Lib2.jar:${deps}/Lib1.jar:${deps}/Lib3.jar:." com.wordpress.mlangc.B0rkage
   

   the program works just fine. Note that your IDE should allow you to specify the order in which JARs are loaded.

As it is my feeling that example above is rather self explanatory, I won’t elaborate on it in detail. What matters is simply what version GoodObject is found first, and this in turn depends on the order in which your JARs are scanned. While you can easily control the order in this example, things might be far more tricky in a real application. Also note that the order in which JARs in WEB-INF/lib are scanned is undefined as far as I know.

The second issue, namely that two versions of the same class originating from different class loaders come into contact with each other, is less common, but might lead to extremely surprising behavior. I going to demonstrate this on two examples. Both are going to result in a ClassCastException with the seemingly nonsensical error message

java.lang.ClassCastException: com.wordpress.mlangc.GoodObject cannot be cast to com.wordpress.mlangc.GoodObject

The first example uses basic Java only. Compared to the second example it’s extremely artificial (you would normally never do such a thing intentionally or unintentionally), but it shows the issue at hand very clearly. It goes like this:

1. Create a library, GoodLib.jar containing exactly this class:

   package com.wordpress.mlangc;
   
   public class GoodObject {
      
   }
   

2. Execute

   public final class WhenGoodObjectsGoBad {
   
       public static void main(String[] args) {
           System.out.println("Best served cold: " + goodObjectFromOtherClassLoader());
       }
       
       private static GoodObject goodObjectFromOtherClassLoader() {
           try(URLClassLoader cl = new URLClassLoader(new URL[] { new URL(pathToJarWithGoodObject()) }, null)) {
               Class<?> goodObjectClass = cl.loadClass(GoodObject.class.getName());
               return (GoodObject) goodObjectClass.newInstance();
           } catch (IOException | ClassNotFoundException | InstantiationException | IllegalAccessException e) {
               throw new RuntimeException(e);
           }
       }
       
       private static String pathToJarWithGoodObject() {
           String relPath = "/" + GoodObject.class.getName().replace(".", "/") + ".class";
           String absPath = GoodObject.class.getResource(relPath).toExternalForm();
           String jarPath = absPath.substring(0, absPath.length() - relPath.length() + 1);
           return jarPath;
       }
   }
   

   with GoodLib.jar in your class path. You should see an error message like

   Exception in thread "main" java.lang.ClassCastException: com.wordpress.mlangc.GoodObject cannot be cast to com.wordpress.mlangc.GoodObject
   	at com.wordpress.mlangc.WhenGoodObjectsGoBad.goodObjectFromOtherClassLoader(WhenGoodObjectsGoBad.java:16)
   	at com.wordpress.mlangc.WhenGoodObjectsGoBad.main(WhenGoodObjectsGoBad.java:10)
   

Don’t spent too much attention to the code in pathToJarWithGoodObject(). What matters is that we find the path to the JAR that contains GoodObject in a portable manner. This path is then used to construct a second class loader in line 8. Note that I explicitly pass null for the parent class loader. The new class loader is employed to load GoodObject once again, which directly leads to the ClassCastException mentioned above. If you take a look at the relevant parts of the The Java® Virtual Machine Specification it shouldn’t be that surprising that the code above calls for serious trouble. Here is a quote from chapter 5.3. Creation and Loading:

At run time, a class or interface is determined not by its name alone, but by a pair: its binary name (§4.2.1) and its defining class loader.

The cast in line 10 fails, because it attempts to cast a GoodObject originating from a foreign class loader to a GoodObject tied to the class loader that loaded our main class. The error message would be clearer if it included the involved class loaders, but with the necessary background information it’s pretty easy to interpret it the way it is.

Before closing this discussion I want to show you another, more realistic scenario that I’ve actually seen in the wild, where two identical classes are loaded by different class loaders. It involves a custom Tomcat Valve and a simple web application:

1. Create a Java library, GoodValve.jar, containing

   package com.wordpress.mlangc;
   
   import java.io.IOException;
   
   import javax.servlet.ServletException;
   
   import org.apache.catalina.connector.Request;
   import org.apache.catalina.connector.Response;
   import org.apache.catalina.valves.ValveBase;
   
   public class GoodValve extends ValveBase {
       @Override
       public void invoke(Request request, Response response) throws IOException, ServletException {
           request.setAttribute("goodObject", new GoodObject());
           getNext().invoke(request, response);
       }
   }
   

   as well as the GoodObject class from above. You need some Tomcat APIs to compile this code. If you are using Maven, adding the following dependency should do the trick:

   <dependency>
     <groupId>org.apache.tomcat</groupId>
     <artifactId>tomcat-catalina</artifactId>
     <version>7.0.40</version>
     <scope>provided</scope>
   </dependency>
   

2. Put the generated JAR file in ${catalina.home}/lib and add the following line to the appropriate Host section in your ${catalina.home}/conf/server.xml:

   <Valve className="com.wordpress.mlangc.GoodValve"/>
   

3. Create a WAR file that contains the following servlet

   package com.wordpress.mlangc;
   
   import java.io.IOException;
   
   import javax.servlet.ServletException;
   import javax.servlet.annotation.WebServlet;
   import javax.servlet.http.HttpServlet;
   import javax.servlet.http.HttpServletRequest;
   import javax.servlet.http.HttpServletResponse;
   
   @WebServlet("/*")
   public class GoodServlet extends HttpServlet {
       
       @Override
       protected void doGet(HttpServletRequest req, HttpServletResponse resp) throws ServletException, IOException {
           doPost(req, resp);
       }
       
       @Override
       protected void doPost(HttpServletRequest req, HttpServletResponse resp) throws ServletException, IOException {
           resp.setContentType("text/plain;charset=utf-8");
           try {
               GoodObject goodObject = (GoodObject) req.getAttribute("goodObject");
               resp.getWriter().print("Mmmh, let's take a look: " + goodObject);
           } catch(ClassCastException e) {
               e.printStackTrace(resp.getWriter());
               resp.setStatus(HttpServletResponse.SC_INTERNAL_SERVER_ERROR);
           }
       }
   }
   

   and of course another copy of our GoodObject class.

4. Deploy the WAR file you’ve just created into the Tomcat installation you’ve modified before, start the server and try to open a page from the deployed WAR file. Voilà, your browser should now tell you that

   java.lang.ClassCastException: com.wordpress.mlangc.GoodObject cannot be cast to com.wordpress.mlangc.GoodObject
   

Again, the key to this example is that the same class is loaded by different class loaders. The GoodObject in GoodValve is loaded from GoodValve.jar by the Tomcat Common Class Loader while the GoodObject from GoodServlet is loaded by the Webapp Class Loader from the WAR file. Take a look at Tomcat 7 Class Loader HOW-TO if you want to know the gory details.