Class Benchmark

public class Benchmark
extends GWTTestCase

A type of GWTTestCase which specifically records performance results. Benchmarks have additional functionality above and beyond GWT's JUnit support for standard TestCases.

In a single JUnit run, the results of all executed benchmarks are collected and stored in an XML report viewable with the benchmarkViewer.
GWT automatically removes jitter from your benchmark methods by running them for a minimum period of time (150ms). GWT also optionally limits your benchmark execution to a maximum period of time (1000ms).
GWT supports "begin" and "end" test methods that separate setup and teardown costs from the actual work being benchmarked. Simply name your functions "begin[TestMethodName]" and "end[TestMethodName]" and they will be executed before and after every execution of your test method. The timings of these setup methods are not included in the test results.
GWT supports test methods that have parameters. GWT will execute each benchmark method multiple times in order to exhaustively test all the possible combinations of parameter values. For each parameter that your test method accepts, it should document it with the annotation, @gwt.benchmark.param.
The syntax for gwt.benchmark.param is <param name> = <Iterable>. For example,
```
 @gwt.benchmark.param where = java.util.Arrays.asList(
   new Position[] { Position.BEGIN, Position.END, Position.VARIED } )
 @gwt.benchmark.param size -limit = insertRemoveRange
 public void testArrayListRemoves(Position where, Integer size) { ... }
 
```
In this example, the annotated function is executed with all the possible permutations of Position = (BEGIN, END, and VARIED) and insertRemoveRange = IntRange( 64, Integer.MAX_VALUE, "*", 2 ).

This particular example also demonstrates how GWT can automatically limit the number of executions of your test. Your final parameter (in this example, size) can optionally be decorated with -limit to indicate to GWT that it should stop executing additional permutations of the test when the execution time becomes too long (over 1000ms). So, in this example, for each value of Position, testArrayListRemoves will be executed for increasing values of size (beginning with 64 and increasing in steps of 2), until either it reaches Integer.MAX_VALUE or the execution time for the last permutation is > 1000ms.

Benchmarks support the following annotations on each test method in order to decorate each test with additional information useful for reporting.

@gwt.benchmark.category - The class name of the Category the benchmark belongs to. This property may also be set at the Benchmark class level.

Please note that Benchmarks do not currently support asynchronous testing mode. Calling GWTTestCase.delayTestFinish(int) or GWTTestCase.finishTest() will result in an UnsupportedOperationException.

Examples of benchmarking in action

A simple benchmark example

AllocBenchmark is a simple example of a basic benchmark that doesn't take advantage of most of benchmarking's advanced features.

public class AllocBenchmark extends Benchmark {

  /**
   * @gwt.benchmark.name Allocation Benchmarks
   * @gwt.benchmark.description A series of benchmarks that tests the impact of
   * different kinds of allocations.
   *
   */
  class AllocCategory implements Category {
  }

  private static final int numAllocs = 1000;

  public String getModuleName() {
    return "com.google.gwt.examples.Benchmarks";
  }

  /**
   * Allocates java.lang.Object in a for loop 1,000 times.
   *
   * The current version of the compiler lifts the declaration of obj outside
   * of this loop and also does constant folding of numAllocs.
   * Also, this loop allocs the GWT JS mirror for java.lang.Object
   * <em>NOT</em> an empty JS object, for example.
   *
   */
  public void testJavaObjectAlloc() {
    for ( int i = 0; i < numAllocs; ++i ) {
      Object obj = new Object();
    }
  }

  /**
   * Compares GWT mirror allocations of java.lang.Object to an empty JS object.
   */
  public native void testJsniObjectAlloc1() /*-{
    for (var i = 0; i < @com.google.gwt.examples.benchmarks.AllocBenchmark::numAllocs; ++i ) {
      var obj = {}; // An empty JS object alloc
    }
  }-*/;

  /**
   * Like version 1, but also folds the constant being used in the iteration.
   */
  public native void testJsniObjectAlloc2() /*-{
    for (var i = 0; i < 1000; ++i ) {
      var obj = {}; // An empty JS object alloc
    }
  }-*/;

  /**
   * Like version 2, but hoists the variable declaration from the loop.
   */
  public native void testJsniObjectAlloc3() /*-{
    var obj;
    for (var i = 0; i < 1000; ++i ) {
      obj = {}; // An empty JS object alloc
    }
  }-*/;

  /**
   * Like version 3, but counts down (and in a slightly different range).
   */
  public native void testJsniObjectAlloc4() /*-{
    var obj;
    for (var i = 1000; i > 0; --i ) {
      obj = {}; // An empty JS object alloc
    }
  }-*/;
}

An advanced benchmark example

ArrayListAndVectorBenchmark is a more sophisticated example of benchmarking. It demonstrates the use of "begin" and "end" test methods, parameterized test methods, and automatic test execution limits.

public class ArrayListAndVectorBenchmark extends Benchmark {

  /**
   * Many profiled widgets have position dependent insert/remove code.
   * <code>Position</code> is a helper class meant to capture the positional
   * information for these sorts of operations.
   */
  protected static class Position {

    public static final Position BEGIN = new Position("at the beginning");
    public static final Position END = new Position("at the end");
    public static final Position NONE = new Position("no location specified");
    public static final Position VARIED = new Position("in varied locations");

    public static final Range positions = new Range() {
      public Iterator iterator() {
        return Arrays.asList( new Position[] {BEGIN, END, NONE, VARIED } ).iterator();
      }
    };

    public static final Range positions2 = new Range() {
      public Iterator iterator() {
        return Arrays.asList( new Position[] {BEGIN, END, VARIED } ).iterator();
      }
    };

    private String label;

    /**
     * Constructor for <code>Position</code>.
     */
    public Position(String label) {
      this.label = label;
    }

    public String toString() {
      return " " + label;
    }
  }

  private static final int PRIME = 3001;

  final IntRange insertRemoveRange = new IntRange(64, Integer.MAX_VALUE,
      Operator.MULTIPLY, 2);

  final IntRange baseRange = new IntRange(512, Integer.MAX_VALUE,
      Operator.MULTIPLY, 2);

  ArrayList list;
  Vector vector;
  int index = 0;

  public String getModuleName() {
    return "com.google.gwt.examples.Benchmarks";
  }

  /**
   * Appends <code>size</code> items to an empty ArrayList.
   * @gwt.benchmark.param size -limit = baseRange
   */
  public void testArrayListAdds( Integer size ) {
    int num = size.intValue();
    for (int i = 0; i < num; i++) {
      list.add("hello");
    }
  }

  // Required for JUnit
  public void testArrayListAdds() {
  }

  /**
   * Performs <code>size</code> gets on an ArrayList of size, <code>size</code>.
   * @gwt.benchmark.param size -limit = baseRange
   */
  public void testArrayListGets( Integer size ) {
    int num = size.intValue();
    for (int i = 0; i < num; i++) {
      list.get(i);
    }
  }

  // Required for JUnit
  public void testArrayListGets() {
  }

  /**
   * Performs <code>size</code> inserts at position, <code>where</code>, on an
   * empty ArrayList.
   * @gwt.benchmark.param where = Position.positions
   * @gwt.benchmark.param size -limit = insertRemoveRange
   */
  public void testArrayListInserts( Position where, Integer size ) {
    insertIntoCollection(size, where, list);
  }

  // Required for JUnit
  public void testArrayListInserts() {
  }

  /**
   * Performs <code>size</code> removes at position, <code>where</code>, on an
   * ArrayList of size, <code>size</code>.
   * @gwt.benchmark.param where = Position.positions2
   * @gwt.benchmark.param size -limit = insertRemoveRange
   */
  public void testArrayListRemoves(Position where, Integer size) {
    removeFromCollection(size, where, list);
  }

  // Required for JUnit
  public void testArrayListRemoves() {
  }

  /**
   * Appends <code>size</code> items to an empty Vector.
   * @gwt.benchmark.param size -limit = baseRange
   */
  public void testVectorAdds( Integer size ) {
    int num = size.intValue();
    for (int i = 0; i < num; i++) {
      vector.add("hello");
    }
  }

  // Required for JUnit
  public void testVectorAdds() {
  }

  /**
   * Performs <code>size</code> gets on a Vector of size, <code>size</code>.
   * @gwt.benchmark.param size -limit = baseRange
   */
  public void testVectorGets( Integer size ) {
    int num = size.intValue();
    for (int i = 0; i < num; i++) {
      vector.get(i);
    }
  }

  // Required for JUnit
  public void testVectorGets() {
  }

  /**
   * Performs <code>size</code> inserts at position, <code>where</code>, on an
   * empty Vector.
   * @gwt.benchmark.param where = Position.positions
   * @gwt.benchmark.param size -limit = insertRemoveRange
   */
  public void testVectorInserts(Position where, Integer size) {
    insertIntoCollection( size, where, vector );
  }

  // Required for JUnit
  public void testVectorInserts() {
  }

  /**
   * Performs <code>size</code> removes at position, <code>where</code>, on a
   * Vector of size, <code>size</code>.
   * @gwt.benchmark.param where = Position.positions2
   * @gwt.benchmark.param size -limit = insertRemoveRange
   */
  public void testVectorRemoves( Position where, Integer size ) {
    removeFromCollection( size, where, vector );
  }

  // Required for JUnit
  public void testVectorRemoves() {
  }

  void beginArrayListAdds( Integer size ) {
    list = new ArrayList();
  }

  void beginArrayListGets( Integer size ) {
    createArrayList( size );
  }

  void beginArrayListInserts(Position where, Integer size) {
    list = new ArrayList();
    index = 0;
  }

  void beginArrayListRemoves(Position where, Integer size) {
    beginArrayListInserts(where, size);
    testArrayListInserts(where, size);
  }

  void beginVectorAdds(Integer size) {
    vector = new Vector();
  }

  void beginVectorGets( Integer size ) {
    createVector( size );
  }

  void beginVectorInserts(Position where, Integer size) {
    vector = new Vector(); index = 0;
  }

  void beginVectorRemoves(Position where, Integer size) {
    beginVectorInserts(where,size); testVectorInserts(where,size);
  }

  private void createArrayList( Integer size ) {
    beginArrayListAdds( size );
    testArrayListAdds( size );
  }

  private void createVector( Integer size ) {
    beginVectorAdds( size );
    testVectorAdds( size );
  }

  private void insertIntoCollection(Integer size, Position where, List v) {
    int num = size.intValue();
    for (int i = 0; i < num; i++) {
      if (where == Position.NONE ) {
        v.add("hello");
      } else if (where == Position.BEGIN) {
        v.add(0, "hello");
      } else if (where == Position.END) {
        v.add(v.size(), "hello");
      } else if (where == Position.VARIED) {
        v.add(index, "hello");
        index += PRIME;
        index %= v.size();
      }
    }
  }

  private int removeFromCollection(Integer size, Position where, List v) {
    int num = size.intValue();
    for (int i = 0; i < num; i++) {
      if (where == Position.NONE) {
        throw new RuntimeException("cannot remove with no position");
      } else if (where == Position.BEGIN) {
        v.remove(0);
      } else if (where == Position.END) {
        v.remove(v.size() - 1);
      } else if (where == Position.VARIED) {
        v.remove(index);
        index += PRIME;
        int currentSize = v.size();
        if ( currentSize > 0 ) {
          index %= v.size();
        }
      }
    }
    return index;
  }
}

Fields

REPORT_PATH

The name of the system property that specifies the location where benchmark reports are both written to and read from.

Field Detail

REPORT_PATH

public static final String REPORT_PATH

The name of the system property that specifies the location where benchmark reports are both written to and read from. Its value is com.google.gwt.junit.reportPath. If this system property is not set, the path defaults to the user's current working directory.

Google Web Toolkit

Class Benchmark

Examples of benchmarking in action

A simple benchmark example

An advanced benchmark example

Fields

Field Detail

REPORT_PATH