IBM Real Time Application Execution Optimizer for Java

1. How does the tool work?

IBM Real Time Application Execution Optimizer for Java is a command-line tool that loads classes using the load options, then it analyzes or modifies the classes, then it writes the classes to disk along with any requested analysis files.

Back to top

2. What options are available to me?

You can use -help to get the following list of options:

IBM Real Time Application Execution Optimizer for Java
usage: java AppExecOptimizer [[-version|-help] | [basic options] [load options] [analysis options] [jar generation options]]

• -help: print out this message
• -version: print out program version

basic options:

• -log xxx: write to the named log file
• -macro xxx xxx: define a macro with the given name and value
• @xxx: read named file for additional options

load options:

• -icp xxx: append named jar/dir to internal class path
• -icpAll xxx: append jars in named dir to internal cls path
• -cp xxx: append named jar/dir to external class path
• -cpAll xxx: append jars in named dir to external cls path
• -jre xxx: append named jdk/jre to external class path
• -jarExtension xxx: recognize files with named extension as jars

• -loadClass xxx: load named class(es) from class path
• -loadResource xxx: load named resource(s) from class path

• -load xxx: load named jar/dir/class/resource on internal cp
• -loadAll xxx: load jars in named dir on internal class path

• -unresolvedRefFile xxx: create named file listing unresolved references

options to load and mark points of entry:

• -entryClass xxx: class(es)
• -entryWholeClass xxx: class(es) and members within
• -entryLibraryClass xxx: class(es) and non-private members within
• -entryAccessibleClass xxx: class(es) and public/protected members within
• -entryField xxx: field(s) and their declaring class(es)
• -entryMethod xxx: method(s) and their declaring class(es)
• -mainMethod xxx: class(es) and main methods within
• -entryMethodEx xxx: method(s) independent of declaring class(es)
• -entryExtLibraryClass xxx: class(es), non-prvt members and overriding meths
• -entryExtAccClass xxx: class(es), pub/prot members and overriding meths

path analysis options:

• -split: split jars according to thread type
• -errorFile xxx: create named file listing errors
• -checkExternal: report errors in external classes
• -followExternal: follow path analysis into external classes
• -entryPointFile xxx: create named file listing points of entry

escape analysis options:

• -escapeReport xxx: create named file listing objects not escaping
• -escapeMethod xxx: analyze object escape from named method(s)
• -escapeDepth xxx: constrain escape analysis to given call depth

jar generation options:

• -output xxx: write to named jar file or directory
• -removeDebugInfo: remove debug attributes from classes
• -removeSourceInfo: remove source file info from classes
• -removeAnnotations: remove annotations from classes
• -removeAttribute xxx: remove named attribute(s) from classes
• -removeStackMaps: remove stackmap table attributes from code
• -addStackMaps: add stackmap table attributes to code
• -excludeResource xxx: exclude the named resource(s) from archives
• -excludeClass xxx: exclude the named class(es) from archives
• -createAutoLoaders: create auto load class within each archive

Back to top

3. What is a macro?

If you use "-macro name value", this will cause any appearance of "{{name}}" on the command line or in options files to be replaced with "value".

Back to top

4. What is in the log file?

Actions performed by the tool are logged to the log file.

Back to top

5. How does the class path work?

The tool uses a class path in the exact same way as a java application, except that the class path is divided between external and internal class path locations. Class path locations are either directories or java archives, although some options cause the tool to identify numerous class path locations at once (icpAll, cpAll). The class path is specified in order to tell the tool where the class files used by the application or library may be found. Some options (-icp, -icpAll, -cp, -cpAll, -jre) specify locations on the class path, others (-loadFile, -loadAll) specify locations and additionally cause all classes found at these locations to be loaded.

Back to top

6. How do I tell the tool about java archives that are not ".jar" or ".zip" files?

Use the -jarExtension option to tell japt that a given file extension (such as ".war") of a file on the class path is in fact an archive.

Back to top

7. How do I cause classes to be loaded?

You have several choices:

1. specify a class path and then specify classes to load (which loads only those classes that are needed)
2. explicitly load all classes found within archives or directories (which automatically adds to the internal class path)
3. a combination of both

Back to top

8. When loading a class, what additional classes are loaded?

Classes loaded include all classes required by the loaded classes at runtime. The tool will continue loading classes as it attempts to resolve all class references. This includes:

• classes accessed by new, instanceof, checkcast, constant string, constant class, method invoke and field access instructions
• super classes and super interfaces
• classes loaded by enclosing methods
• classes loaded by annotations
• classes loaded by inner class and outer classes
• classes referenced by method signatures and field signatures
• exception classes caught within methods
• exception classes in method declarations
• the element class types of array classes
• classes referenced in the local variable tables of local variable types for debugging

Back to top

9. How do I load all classes found within an archive or directory?

You load all classes found within a specific location by using -load or -loadAll.

1. -load will load a class file, all class files within an archive, or all class files within a directory
2. -loadAll will load all archives found within a directory

Each class file or class path location (archive or directory) specified this way is considered a part of the internal class path, and all classes loaded from these locations will be internal classes. Classes referenced from these locations, but not found within them,will be loaded from the full class path, and will be internal or external classes depending upon where they are found.

You may optionally wish to specify the rest of the class path to find classes that are referenced and not found within the archives or directories to be loaded, such as JRE classes.

Back to top

10. How do I load classes, methods or fields by name?

Once you have specified a class path, you can load classes and resources that can be found somewhere on the class path using either:

1. -loadClass or -loadResource
2. Any one of the following options that not only loads the specified item(s) but also marks the item(s) as entry points into the app or library:
• -entryClass
• -entryWholeClass
• -entryLibraryClass
• -entryAccessibleClass
• -entryAccessibleClass
• -entryField
• -entryMethod
• -mainMethod
• -entryMethodEx
• -entryExtLibraryClass
• -entryExtAccClass

The items wil be loaded, and additionally all items that are referenced by these items or their declaring classes will be loaded from the class path.

Back to top

11. Why would I load classes, methods or fields by name?

This will make the tool load only the classes referenced by the application. It allows you to be selective about what is loaded, and it will not load classes that are unused. It allows you to load classes in a way that is similar to the way classes are loaded by a virtual machine.

Back to top

12. How do I cause resources to be loaded?

You can specify the full path name of the resource relative to the class path with -loadResource. You can load a resource explicitly with -load. Additionally, archives loaded with -load or -loadAll will have their included resources loaded.

Back to top

13. What is an entry point?

An entry point is any method, field or class in the loaded classes that serve as a point of entry into the application. For instance, the main method is an entry point into the application when the application is started. If an application is accessed dynamically with java reflection, java serialization, or accessed from non-Java code with JNI (Java Native Interface), then those points of access are also entry points. For a library, any number of non-private fields or methods might be identified as entry points.

Back to top

14. Do I need to mark the points of entry?

The points of entry are used to start the path analysis. Without them, the path analysis does not know how the app is started or the library is accessed. Without them, the path analysis does not know where to start.

Back to top

15. What is the difference between the external and internal class path, and external and internal classes?

Each class path location is identified as either internal or external. A class that is loaded from an external location is considered an external class, while a class that is loaded from an internal location is considered an internal class. Generally, but not exclusively, an internal class is considered part of the application or library being deployed or analyzed, while an external class is a supporting class. For instance, you would typically specify the location of your application with the internal class path, and specify the java runtime environment classes with the external class path. When the classes are written to disk, only the internal classes are written.

Back to top

16. Is it necessary to have the classes from a Java Runtime Environment (JRE) on the class path?

No, it is not necessary, but it is desirable. You can use the option -jre. The tool does not contain an implementation of Java SE or the Real-Time Specification for Java. However, it does contain a minimal set of proxy classes which enable path analysis when a JVM is not on the class path. If there is an implementation of RTSJ on the class path, then the class path implementation takes precedence over the proxy classes contained within the tool.

Back to top

17. How do I specify multiple classes, methods, fields or resources at once?

Any option used to specify a class, field, method or resource will accept wildcard characters. The available wildcards are:

*: any sequence of characters
^: any sequence of characters that does not include either of the separators ‘/’ or ‘.’

For example:

• to mark as points of entry all classes in java.lang and subpackages:
  -entryClass java.lang.*
• to mark as points of entry all classes in java.lang: -entryClass java.lang.^
• to mark as points of entry the methods in java.lang.Object: -entryMethod java.lang.Object.^
• to mark as points of entry all methods: -entryMethod *
• to mark as points of entry all methods in the default package: -entryMethod ^.^
• to mark as points of entry all classes and their public, protected and package methods and fields in java.lang and subpackages:
  -entryLibraryClass java.lang.*
• to limit escape analysis to setter and gettermethods:
  -escapeMethod *.set^ -escapeMethod *.get^

Back to top

18. What is an unresolved reference?

An unresolved reference is a reference to a class not found on the class path, or a reference to a field or method not found within the loaded classes.

Back to top

19. What is the difference between -entryMethod and -entryMethodEx?

If a method or field is an entry point, then generally the declaring class is also an entry point, but not necessarily. In general, the method or field can only be identified by also specifying the declaring class. However, it is possible for a method to be an entry point while the class declaring the method is not. If a method overrides a method in an external class, thent the method can be accessed directly through a virtual invocation, while the identity of the class declaring the method is irrelevant. -includeMethodEx can be used to identify the method as an entry point independently of the declaring class.

Back to top

20. Can a private method or field be an entry point?

Yes, particularly if the method or field is accessed directly with JNI, the java reflection API, or java serialization.

Back to top

21. What is listed in the entry point file?

The file will list methods, fields and classes which serve as points of entry into the internal classes from the external classes.

This may include entry points not explicitly specified on the command line, these will be points of entry found by the path analysis, such as virtual invocations accessing internal methods from external code.

If -followExternal is not specified, the list will be limited to the entry points listed on the command line.

If -followExternal is specified, external paths will be followed and any additional entry points found by crossing from external classes back into internal classes through virtual or interface method invocations will be added to the entry point file.

Back to top

22. What is a path analysis?

Some options, namely the options -split, -errorFile, and -entryPointFile, will initiate a path analysis of the analyzed application.

The path analysis will follow java methods and instantiate objects and to approximate the flow of execution at runtime. The path analysis is by default limited to the internal classes, but it can be extended to all classes loaded by using -followExternal, including the JVM classes, although this may require a considerable amount of memory and time.

Back to top

23. Should the path analysis follow paths into external classes with the -followExternal option?

Following external paths may sometimes require a large amount of run-time memory, due to the large number of possible paths that can be taken by an application. If an application has no callbacks from external classes to internal classes, external paths need not be followed for a proper analysis of internal classes. However, if callbacks exist, then the analysis may miss some paths into the internal classes if -followExternal is not specified.

Back to top

24. How are loaded classes and resources categorized according to thread type?

All loaded classes are divided into the following groups with the -split option, using the path analysis:

• accessible to NoHeapRealtimeThread threads
• accessible to RealtimeThread threads
• accessible to regular java threads
• inaccessible to any thread

Resources are separated into their own group.

Back to top

25. How are the classes verified for runtime errors?

The -errorFile option will cause errors found by the analysis to be reported within the specified file.

The tool will report potential occurrences of the runtime exceptions MemoryAccessError, IllegalAssignmentError and IllegalThreadStateException. Also reported are any exceptions that can be thrown as a result of the class loading, verification and resolution process. This includes AbstractMethodError, ClassCircularityError, ClassFormatError, IllegalAccessError, IncompatibleClassChangeError, InstantiationError, LinkageError, NoClassDefFoundError, NoSuchFieldError, NoSuchMethodError, UnsupportedClassVersionError, and VerifyError.

Back to top

26. How does the tool locate potential runtime errors?

The path analysis maintains context information that allows the tool to detect the possible occurence of runtime errors such as MemoryAccessError, IllegalAssignmentError and IllegalThreadStateException. The tool also searches for errors in class structure, performing a full verification, when classes are loaded.

Back to top

27. What are the limitations of the path analysis?

When reporting the potential errors MemoryAccessError, IllegalAssignmentError and IllegalThreadStateException , the tool may report false positives, and it may also miss some potential errors.

The static analysis does not attempt to evaluate boolean conditions that are evaluated at runtime, and so it will follow paths within methods that might not be followed at runtime, resulting in errors reported that might not be reachable in runtime conditions. Additionally, the usage of JNI and reflection (java.lang.Class and java.lang.reflection) cannot be followed by the analysis and may result in unreported errors. Unresolved references can also cause unreported errors.

Class initialization poses a challenge for path analysis. This is because it is not possible to tell during static analysis which thread triggers the initialization of a class. It is possible that initialization could be triggered by a regular thread, an RT, or an NHRT. If class initializers try to make use of scoped memory then problems can result if the initializing thread is a regular thread. If initializers try to make use of the heap, then problems can arise if the intializing thread is an NHRT.

For the purpose of this analysis, the initializing thread is assumed to be a regular thread running in immortal memory. This includes classes that are loaded by the run-time verifier; it is assumed that such classes are loaded by a regular thread running in immortal memory.

In fact, virtual machines which provide aggressive loading will ensure that the initializing thread is a regular thread, by loading when the application starts, so that loading is never performed by user threads.

Finalization poses another challenge for analysis. This is because it is not possible to tell during static analysis which thread triggers the finalization of an object.

There is no confusion in the case of heap objects, which are typically finalized by a regular thread, while immortal objects are not finalized at all.

Back to top

28. How do I use the auto-load classes?

When you specify -createAutoLoaders, the tool will create an auto-load class for each archive and tell you their names. When you call the "load()" method in each class in your application, it will load all the other classes in the same archive. However, all you need to do is initialize the class and "load()" will be called automatically, so you can simply pass each auto-loader class name to an invocation of java.lang.Class.forName(java.lang.String).

Back to top

29. What is escape analysis?

Escape analysis determines if an object can outlive a method invocation that created it. When a method is invoked, it will create objects, and subinvocations will create objects, and some of these objects may not be reachable when the original invocation completes, which means that these objects did not escape the invocation. Objects can escape in numerous ways, such as into static fields, as uncaught exception objects that were thrown, as returned objects, through method arguments, into other threads, into invocations that could not be followed such as native method invocations, into a field of the receiving object of the invocation, or as an object reachable (through fields or array elements) from another object that escapes.

Back to top

30. How can the escape analysis be used?

Escape analysis can assist with determining the memory requirements of executing a method. It will tell you the number of object instantiation locations in the code and whether the objects instantiated at each location will persist. It can also help to determine whether a method is safe to be executed in a scope, since objects escaping may cause IllegalAssignmentErrors.

Back to top

31. How should the escape analysis results be interpreted?

"Can potentially escape" means that the analysis has found a path through which an object may escape. This does not necessarily mean that the object will escape, this may mean that there is a route of escape (even though that route might not be chosen at runtime, or possibly cannot be chosen at runtime).

"Cannot escape" means that there is simply no route of escape possible, and the object will never escape no matter how the application is executed or how often.

Back to top

32. What is the effect of limiting the depth of escape analysis?

The context analysis will not enter method invocations if the depth-limit would be exceeded. This will not result in incorrect reports of objects not escaping, rather it would results in more objects being reported as potentially escaping, since objects that can pass into such depth-limited method invocations would be deemed escaping objects.

Limiting the depth can be useful to avoid long analyses that take a second or more. Most escape analyses take on the order of microseconds. Limiting the depth limit to a value of 11 or more does not significantly affect the results of the escape analyses.

Back to top

33. How are classes written to disk?

When writing classes to disk, the user may specify either a directory or a jar file with -output.

When specifying a directory, the classes will be written to individual archives with names corresponding to the archives from which the classes originated.

When specifying an archive, all classes will be written to the same file.

If the "split" option has been specified, the written archives will each be split into as many as five separate archives, whose names have been altered by applying a suffix to the file name: "nhrt", "rt", "t", "inaccessible", and "resources", for each of the categories listed above.

Back to top

34. What are the consequences of removing debug info from classes?

When using a debugger, the local variables and line numbers of source files are not available to the debugger. Also, while running the application outside of a debugger, the stack traces of exceptions will not show source code line numbers. A class file remains valid if these items are removed. Removing this information does not change program behaviour.

Back to top

35. What are the consequences of removing source info or annotation info from classes?

Annotations are available through the java reflection library and are used for deploying java applications in specific environments. Source info is generally available through the java reflection library. Source info tells the java virtual machine information about items in the class file such as:

• inner and outer class relationships or the enclosing methods of local classes,
• which items are synthetic items (compiler-generated classes, methods and fields)
• the types of items that are generic (generic class names, generic field types, generic method signatures)
• deprecated items

A class file remains valid if these items are removed. For most java programs, removing this information does not change program behaviour.

Back to top

37. What are stackmaps?

Stackmaps are structures introduced in Java 5 to speed up class file verification. They represent the operand stack during execution of a java method.

For java 5 virtual machines stackamps are optionally found in the class file. Java 6 virtual machines require them.

Back to top

38. What attributes can be removed from class files?

Any class file generator can create proprietary attributes that can be added and removed. Additionally, there are numerous standard attributes defined by the Java Virtual Machine Specification. Most can be removed under various circumstances.

Stackmaps:
StackMapTable: Java SE stack maps
StackMap: CLDC stack maps

Source info attributes:
EnclosingMethod: Identifies the enclosing method of local classes
Signature: Identifies generic signatures for classes, field types and method signatures
Deprecated: Identifies deprecated items
Synthetic: Identifies a compiler-generated item
InnerClasses: Identifies inner and outer class relationships (including nested and anonymous)

Annotation attributes:
AnnotationDefault
RuntimeVisibleAnnotations
RuntimeInvisibleAnnotations
RuntimeVisibleParameterAnnotations
RuntimeInvisibleParameterAnnotations

Debug:
SourceFile
SourceDebugExtension
LineNumberTable
LocalVariableTable
LocalVariableTypeTable

Attributes that will result in an invalid class file when removed:
Code
ConstantValue
Exceptions

Back to top

39. What does an example command line look like?

1. Command line:
   java AppExecOptimizer -log log.txt -load <an additional application jar file> -icp <application class path directory > -mainMethod <application class> -jre <the java run-time environment> -output <an output directory> -split -entryPointFile entryPoints.txt -unresolvedRefFile unresolved.txt escapeReport escaped.txt -escapeDepth 5 -escapeMethod "a.package.*" -followExternal

   Load an application on the internal class path, loading all classes within a jar file and also specifying a supporting class path directory. Indicate the entry point as the main class and add the supporting JRE to the external class path. Produce the unresolved reference, entry points and escape analysis files. Restrict the escape analysis to all classes in the package "a.package", and limit the escape analysis stack depth to five frames. Write the classes to jar files inside the indicated output directory, and split the jar files according to thread accessibility.

2. Command line:
   java AppExecOptimizer -load <a class file> -output <a jar file> -removeDebugInfo -addStackMaps

   Load a class file, strip the debug info, add stackmaps, and write it to a jar file.

3. Command line:
   java AppExecOptimizer -entryLibraryClass "com.x.*" -icp <app jars>-cp <supporting jars> -entryPointFile entryPoints.txt -followExternal

   Mark all public and protected methods in the package com.x of a of an app as entry points, and determine what additional entry points exist from the supporting jars into the application.

4. Command line:
   java AppExecOptimizer -mainMethod a.package.Cls -icp <app class path> -split -errorFile errorFile.txt -output output.jar

   Split an app according to thread accessibility, where the app's main method is in the class "a.pacakge.Cls", and check the app for any run-time or verification errors. Write the app to the output jar that has been split into several jars according to thread accessibility.

   Back to top