Troubleshooting

This section describes the various techniques that are available to troubelshoot InfraRED for both the Load Time Weaving(LTW, as detailed in the Installation page) and the Compile Time Weaving(CTW, as detailed in the Ant Task page).

Version Information

The InfraRED version information gets displayed on the console when the instrumented application gets deployed. This is a necessary (though not sufficient) indicator to the successful setting up of the various server paths. This message is applicable to both the aop weaving mechanisms,viz, load time weaving as detailed in the Installation page and Ant Task page A sample version information message is given below :


*********************************************
* InfraRED version XX-VERSION
*
* Configured logging system from file:/D:/infrared-home/infrared-log4j.xml
*********************************************
            

Sun 1.4 JVM Troubleshooting : Load Time Weaving Information

If the instrumented application makes use of the Sun 1.4 JVM and the Load Time aop Weaving mechanism, then :

  1. For AspectJ 1.5 the following message should be displayed on the console during server startup.
    AspectWerkz - INFO - Pre-processor net.sf.infrared.weaverhook.ClassPreProcessorAdaptor loaded and initialized
    						
  2. For AspectWerkz the message on console during server startup is:
    AspectWerkz - INFO - Pre-processor org.codehaus.aspectwerkz.transform.AspectWerkzPreProcessor loaded and initialized
    						
These messages indicate that the enhanced.jar(which needs to be generated if load time weaving for Sun 1.4 JVM is required) and the weaver are correcly set up.

Identifying applications integrated with InfraRED

If the changes to the application as mentioned in Installation page were applied correctly, and if the system property infrared.print.applications is set to true (using -D JVM flag), you should see the following message on the console(System.out) when the application startsup.

[InfraRED] Created MonitorFacade for your-app, instance your-host-name, with config MonitorConfig[file:/D:/your-app-home/WEB-INF/classes/infrared-agent.properties]
            

The second line above indicates that InfraRED has successfully identified your-app application and would be monitoring it. The configuration file for this application is at D:/your-app-home/WEB-INF/classes/infrared-agent.properties. The first line is always printed and should not be taken as a sign of your application being sucessfully identifed by InfraRED.

Logging

InfraRED uses log4j for logging. It ships with a modified log4j version and does not interfere with the normal logging of the application.

To enable logging in InfraRED,

  1. you should have the file infrared-log4j.xml in the server's classpath and
  2. set the system property infrared.debug to true.

The format of the infrared-log4j.xml file is same as the regular log4j.xml file, except that all references to classes in package org.apache.log4j should be replaced with net.sf.infrared.org.apache.log4j. A sample infrared-log4j.xml is given below:


<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE log4j:configuration SYSTEM "log4j.dtd">
<log4j:configuration debug="false">
    <appender name="FILE" class="net.sf.infrared.org.apache.log4j.FileAppender">
      <param name="file" value="./infrared.log"/>
      <layout class="net.sf.infrared.org.apache.log4j.PatternLayout">
        <param name="ConversionPattern" value="[%c] - %m%n"/>
      </layout>
    </appender>

    <appender name="CONSOLE" class="net.sf.infrared.org.apache.log4j.ConsoleAppender">
        <layout class="net.sf.infrared.org.apache.log4j.PatternLayout">
            <param name="ConversionPattern" value="[%t %c] - %m%n"/>
        </layout>
    </appender>

    <category name="net.sf.infrared.agent.transport" additivity="false">
        <priority value="DEBUG"/>
        <appender-ref ref="FILE"/>
    </category>

    <category name="net.sf.infrared.agent.health" additivity="false">
        <priority value="DEBUG"/>
        <appender-ref ref="CONSOLE"/>
    </category>

    <root>
        <priority value="FATAL"/>
        <appender-ref ref="FILE"/>
    </root>
</log4j:configuration>

            

Notice how all packages start with net.sf.infrared.

If the system property infrared.debug (specified typically using -D JVM flag) is turned off (i.e its value is "false"), InfraRED will not log any debug messages even if the log4j configuration specifies debug level for the category. If the system property is turned on (i.e. its value is "true"), InfraRED will log debug message iff the category has log level DEBUG or more verbose. The default value of this system property is false.

Two log categories are particularly important

  • net.sf.infrared.agent.transport.impl.SocketWriter is used to log if InfraRED is able to collect data in the application and send it across the GUI. If some data is collected and send to the GUI, you would be seeing logs like:
    [net.sf.infrared.agent.transport.impl.SocketWriter] - SocketWriter (address = localhost/127.0.0.1, port = 7777 connected) - Wrote stats
                        
  • net.sf.infrared.agent.health is used to print information on how much data is collected. Sometimes you would think that InfraRED is all set up correctly, but no data shows up on the GUI. In those cases you should enable this log level - it helps identify whether data is not showing up because of all executions getting pruned. A sample log entry would look like:
    [net.sf.infrared.agent.health] - Agent collected stats for operation that started at Tue Dec 13 07:35:53 GMT 2005 and ended at Tue Dec 13 07:35:53 GMT 2005. Tracked 10 and ignored 1
                        
    Some statistics would be ignored because they are below the prune threshold. The concept of prune threshold is explained in the configuration page.

In general, InfraRED generates lots of logs in DEBUG mode, so it should be only be enabled when you run into issues.

Bug# 1423202: Casts to driver-specific interfaces fail when using InfraRED

To capture JDBC statistics, InfraRED it wraps driver-provided implementations of java.sql interfaces with its own (in the package net.sf.infrared.aspects.jdbc.p6spy).

This is fine in most cases, because most often the client code would use only the java.sql interfaces. But when using some drivers, one might need to cast to driver-specific interfaces. For instance, when inserting LOBs using Oracle drivers, one might need to cast to Oracle-specific interfaces.

As a solution, we provide a newer technique to collect JDBC statistics.

For AspectJ: Edit the aop.xml in the infrared-agent-all-servlet-XX.jar, infrared-agent-all-weblogic-XX.jar or infrared-agent-all-jboss-XX.jar (whichever you are using), comment out the aspect net.sf.infrared.aspects.aj.JdbcAspect, and uncomment the aspect net.sf.infrared.aspects.aj.NonWrappingJdbcAspect.

For AspectWerkz: Edit the aop.xml in the infrared-agent-all-servlet-XX.jar, infrared-agent-all-weblogic-XX.jar or infrared-agent-all-jboss-XX.jar (whichever you are using; we assume that you would have already replaced the aop.xml as mentioned in the installation page), comment out the system infrared.system.jdbc, and uncomment the system infrared.system.jdbc.nowrap.

As of the now, the feature where InfraRED records the PreparedStatement query parameter bindings is not available when JDBC is set up with this mode.

Linkage Error on Tomcat and JBoss

The instrumented application being monitored by Infrared throws a java.lang.Linkage error when both the instrumented application and the Infrared web application are deployed on the same server instance of Tomcat or JBoss. There are several ways to overcome this problem :

  1. Deploy the instrumented application and the Infrared web application in two separate server instances.
  2. If you are deploying the webapps in the same server instance, you could do so by deleting two of the classes packaged in the infrared-web-all-XX.war - viz ExecutionTimer.class and ExecutionContext.class. These classes are packaged within the infrared-collector-all-XX.jar inside the infrared-web-all-XX.war.
  3. For JBoss server, this issue can be resolved by setting the UseJbossWebLoader property of the JBOSS_HOME/server/default/deploy/jbossweb-tomcat55.sar/META-INF/jboss-service.xml file to true.

Out of Memory Error (OME)

  • OMEs - Load Time Weaving: The infrared integration detailed in the installation page weaves infrared code into the application when the application classes get loaded in the VM. This technique, commonly known as the load time weaving in the aspect world, is found to consume a lot of memory. Hence for large applications, the load time weaving technique of integrating infrared can pave way to OMEs. In those cases it is advisable to make use of the Compile Time Weaving technique as detailed in the Ant task page.
  • OMEs - Compile Time Weaving: The OMEs while integrating applications using Compile Time Weaving technique is largely due to insufficient Perm Generation space in the VM. Perm Generation is where the JVM stores class files, string literals etc. The Sun JVM has a default perm generation size maximum of 64MB. If a large application takes up a major share of this space, the perm generation runs out when the classes are instrumented for InfraRED. In those cases it is advisable to try increasing the max perm size limit for the VM. The JVM flag to set this is -XX:MaxPermSize
    eg: -XX:MaxPermSize=128m