2011-04-05

Testing a Plug-in - User Interface Testing

This post is part of my "Testing a Plug-in" series.

Previous posts are:
Assume you have a plug-in with a number of views, editors, dialogs and wizards - just how do you test the functionality of the SWT controls in one of these?

Please also note that I use the opportunity with this blog series to rewrite, test, refactor and generally pretty up my old test tools. Within the next couple of weeks, I will put the tools put on github for general use, but until then, I will include the relevant parts of the tools in the posts...

As always, I'm very interested in feedback so please feel free to comment with changes, fixes, enhancements....

Different Approaches

This is an areas where there are a number of very different approaches.

You can drive the application from the outside using a tool that basically acts as user. The main problem with this is, of cause, that even smaller changes to the layout of views of dialoga can mean you have to redo major parts of your tests.

You can synthesize the events sent to the application using Display.post(...). In this case the coordinates for the controls can be calculated based on the actual positions of the controls and thus this approach is less sensitive to many changes in the layout than the "outside" tester. Note that it is not always easy to get the Event to be posted right, and there are some rather complecated stuff involved - e.g. to ensure that two mouse clicks are considered two "single" clicks and not a double click you have to know the "double-click time" and wait if needed... Also note that especially the Cocoa edition of SWT have had some very serious problems with key support for Display.post(...) in the past - though these should be fixed with Eclipse 3.7.

You can drive most Controls directly using the SWT API and get the same result as if a user had made changes to the control directly. Note though that there are many smaller differences between the supported operating systems as to what exactly happens when you do this, so only the most common operations should be done this way. Also, there are many operations that cannot be performed this way - e.g. selecting a push Button - where the Display.post(...) method must be used instead.

I had a presentation at Eclipse Summit 2010 about this and you might want to have a look at this for further information.

Eclipse Summit Europe '10 - Test UI Aspects of Plug-ins

View more presentations from The RCP Company

Different Tools

There are many different tools available as well. One official tool is SWTBot though personally I prefer the set of methods I have developed over time, which have a very simple approach to UI testing - see the slides above.

How To Do It


Below, you can find three small examples on how to test some very basic UI related stuff using the second and third approaches above. This is based on the following very simple view. In this view, you can type in characters in the first text field and then press the "Copy" button, at which the text from the first Text field should be copied to the second Text field. As always, you can find a zip here with all the code.

In order to utilize the third approach above, we must have access to the controls of the view. This is done using either public or package protected fields in the view class. If you think this will the make the fields too vulnerable, then consider adding access methods for each of them. I usually make them package protected, which means only Java classes in the same package can access the fields. As only Java classes in this plug-in and in any attached fragment, can ever do thing, we are reasonable safe - or at least safe as you can ever get with fragments.

So...
  • we create fields in the view that are package protected
  • we use a test fragment - as described elsewhere
  • we put our tests for the view in question in a package in the fragment with the same name as in the plug-in
The relevant parts of the view looks as follows:

public class CopyView extends ViewPart {
 /* package for testing */Text myFromText;
 /* package for testing */Text myToText;
 /* package for testing */Button myButton;

 @Override
 public void createPartControl(Composite parent) {
  final Composite top = new Composite(parent, SWT.NONE);
  top.setLayoutData(new GridData(SWT.FILL, SWT.FILL, true, true));
  top.setLayout(new GridLayout(2, false));

  final Label fromLabel = new Label(top, SWT.NONE);
  fromLabel.setLayoutData(new GridData(SWT.END, SWT.CENTER, false, false));
  fromLabel.setText("From:");

  myFromText = new Text(top, SWT.SINGLE | SWT.LEAD | SWT.BORDER);
  myFromText.setLayoutData(new GridData(SWT.FILL, SWT.CENTER, true, false));
  myFromText.setText("");

  final Label toLabel = new Label(top, SWT.NONE);
  toLabel.setLayoutData(new GridData(SWT.END, SWT.CENTER, false, false));
  toLabel.setText("To:");

  myToText = new Text(top, SWT.SINGLE | SWT.LEAD | SWT.BORDER | SWT.READ_ONLY);
  myToText.setLayoutData(new GridData(SWT.FILL, SWT.CENTER, true, false));
  myToText.setText("");

  myButton = new Button(top, SWT.PUSH);
  myButton.setLayoutData(new GridData(SWT.END, SWT.CENTER, true, false, 2, 1));
  myButton.setText("Copy");

  myButton.addSelectionListener(new SelectionListener() {
   @Override
   public void widgetSelected(SelectionEvent e) {
    myToText.setText(myFromText.getText());
   }

   @Override
   public void widgetDefaultSelected(SelectionEvent e) {

   }
  });
 }

 @Override
 public void setFocus() {
  myFromText.setFocus();

 }
}

We can test at least three aspects of the view UI here
  • that the first field have focus initially. This is really more a view aspect here, but the same tests should be done for dialogs and wizards, so I leave it in.
  • that controls are R/W or R/O as needed and all controls are enabled
  • that the functionality of the "Copy" button is correct
To do any of this we must first show the view. This can be done with a method from BaseTestUtils. Likewise the view must be closed again after use.

@Before
public void before() {
 myView = (CopyView) BaseTestUtils.showView("com.rcpcompany.testingplugins.ex.ui.views.CopyView");
}

@After
public void after() {
 myView.getSite().getPage().hideView(myView);
}

To test that the fist field have focus we can do this:

@Test
public void testFocus() {
 assertTrue(myView.myFromText.isFocusControl());
}

Pretty single, I would think.

Likewise to test the R/O of al envolved controls, we can do this:

@Test
public void testFieldsRO() {
 assertEquals(SWT.NONE, myView.myFromText.getStyle() & SWT.READ_ONLY);
 assertEquals(SWT.READ_ONLY, myView.myToText.getStyle() & SWT.READ_ONLY);

 assertTrue(myView.myFromText.isEnabled());
 assertTrue(myView.myToText.isEnabled());
 assertTrue(myView.myButton.isEnabled());
}

Also, very simple.

To test the functionality, it is a little worse.

@Test
public void testFlow() {
 assertEquals("", myView.myFromText.getText());
 assertEquals("", myView.myToText.getText());

 myView.myFromText.setFocus();
 final String TEST_STRING = "test string";
 myView.myFromText.setText(TEST_STRING);

 UITestUtils.postMouse(myView.myButton);

 assertEquals(TEST_STRING, myView.myFromText.getText());
 assertEquals(TEST_STRING, myView.myToText.getText());
 assertTrue(myView.myFromText.isFocusControl());
}

First I test both text controls are empty, just the make sure the initial condition is well-defined.

Then we make sure the first control have focus, because that would always be the case in the real world. Not that it matters in this case, but there are cases with focus listeners where it would matter.

Next the content of the first control can be assigned to. This is identical to pasting the string into the control as both the verify and modify listeners are run - though the key or mouse events for the paste commands is not seen.

To press the "Copy" button, we must post an artificial event, as there are no way to do this via the Button API. Luckily BaseTestUtils includes a large number of methods that can help.

And last, we can test the result...

One could test even further... E.g. it is not clear in the test above that the text of the second Control is assigned to or appended to... I'll leave that as an exersize for interested...

To run the tests, select the project and "Run as..." "JUnit Plug-in Test"... In some cases, the test will not succeed first time, and you will have to set the application and plug-ins in the launch configuration first...

One technical note: if you make any changes to the dependencies of the fragment, then remember to all the "-clean" argument to your launch configuration. Otherwise, OSGi will not pick up the new dependency and you will be left with a long and frustrating time debugging the problems.... I should know :-)

2011-04-01

Testing a Plug-in - Where to put the tests

This post is part of my "Testing a Plug-in" series.

Say you have decided to test your plug-ins, the first question of cause is, where exactly to put the tests. As it comes, there are a number of possibilities each with its own advantages and disadvantages.

In the following, I assume we are using JUnit 4 as this does not put any requirements on the super-classes. Otherwise, with JUnit 3, our options are a little more limited.

The tests can be put inline in the application plug-ins
  • Pros:
    • This is very easy to manage and understand. Also it is very easy to refactor the code, as you do not have to remember to have any specific test plug-ins or fragments in your workspace when you refactor your code - though it is always a good idea!
    • Your tests will have access to everything - you can even put the test in the same class as the methods to be tested.
  • Cons:
    • The tests will be in the final product, though it is possible to ge the Java compiler to leave out the test methods based on a special annotation. If you remove them from the plug-in during the release phase of the project, you have the additional problem, that what you test is not exactly your release...
    • If your tests requires any specific extensions in plugin.xml, then these will be harder to remove from the plug-in during compilation. This can be needed if you deckare any additional extension points in your target plug-ins and need to test these.
The tests can be put into a separate plug-ins
  • Pros:
    • The tests are separate from the product, so the cons of the previous option are not present here.
    • The test plug-in will have the same access to the exported Java packages as any "ordinary" consumer plug-in.
  • Cons:
    • Because of OSGi and the access rules it impose on bundles, you have only access to the exported Java packages. Of cause you can use x-friends to allow a specific visibility for specific package and friendly bundles, but who wants to do that if it can be avoided.
The tests can be put in an attached fragments
  • Pros:
    • The tests have access to everything as a fragment effectively "runs" inside the host plug-in.
    • Like the previous option, the tests are separate from the product.
    • You can even test new extension points and other other stuff that requires additions to plugin.xml...
  • Cons:
    • A fragment is a slightly different "thing" than a plug-in, and many developers shy away from them for this reason.
    • Some build tools have problems running tests that are put inside fragments - though these problems seems to be gone in the 
I usually prefer a mix of fragments and plug-ins for my tests. The fragments are used for all tests of the internals of the target plug-in and the plug-ins are used for the needed black-box tests for the same target plug-ins.

One area where the plug-in approach is needed, is in order to test that the needed interfaces from the extension points are indeed exported and fully available. One problem you might otherwise see, is that super-classes are internal to the target plug-in which can give some very hard to understand error messages.