After 5 year and lots of fun with Eclipse, I will start as a Systems Architect in a local Danish company from September.
A major part of my new job will be to introduce Eclipse technology into the products so I basically moves from being Eclipse service provider to being a consumer. As such, I expect to stay in the ecosystem, and I might meet you all again at EclipseCon...
If any of you have an interest in my training materials, please let me know. I have training modules that covers most aspects of plug-in development including a set of modules for the most common modeling techniques... The very basic parts are yours for free - I have already started to put them on slideshare with more to come - but the more advanced stuff is for sale.
Assume you have a plug-in with a number of views, editors, dialogs and wizards - just how do you test the functionality of the SWT controls in one of these?
Please also note that I use the opportunity with this blog series to rewrite, test, refactor and generally pretty up my old test tools. Within the next couple of weeks, I will put the tools put on github for general use, but until then, I will include the relevant parts of the tools in the posts...
As always, I'm very interested in feedback so please feel free to comment with changes, fixes, enhancements....
Different Approaches
This is an areas where there are a number of very different approaches.
You can drive the application from the outside using a tool that basically acts as user. The main problem with this is, of cause, that even smaller changes to the layout of views of dialoga can mean you have to redo major parts of your tests.
You can synthesize the events sent to the application using Display.post(...). In this case the coordinates for the controls can be calculated based on the actual positions of the controls and thus this approach is less sensitive to many changes in the layout than the "outside" tester. Note that it is not always easy to get the Event to be posted right, and there are some rather complecated stuff involved - e.g. to ensure that two mouse clicks are considered two "single" clicks and not a double click you have to know the "double-click time" and wait if needed... Also note that especially the Cocoa edition of SWT have had some very serious problems with key support for Display.post(...) in the past - though these should be fixed with Eclipse 3.7.
You can drive most Controls directly using the SWT API and get the same result as if a user had made changes to the control directly. Note though that there are many smaller differences between the supported operating systems as to what exactly happens when you do this, so only the most common operations should be done this way. Also, there are many operations that cannot be performed this way - e.g. selecting a push Button - where the Display.post(...) method must be used instead.
There are many different tools available as well. One official tool is SWTBot though personally I prefer the set of methods I have developed over time, which have a very simple approach to UI testing - see the slides above.
How To Do It
Below, you can find three small examples on how to test some very basic UI related stuff using the second and third approaches above. This is based on the following very simple view. In this view, you can type in characters in the first text field and then press the "Copy" button, at which the text from the first Text field should be copied to the second Text field. As always, you can find a zip here with all the code.
In order to utilize the third approach above, we must have access to the controls of the view. This is done using either public or package protected fields in the view class. If you think this will the make the fields too vulnerable, then consider adding access methods for each of them. I usually make them package protected, which means only Java classes in the same package can access the fields. As only Java classes in this plug-in and in any attached fragment, can ever do thing, we are reasonable safe - or at least safe as you can ever get with fragments.
So...
we create fields in the view that are package protected
we put our tests for the view in question in a package in the fragment with the same name as in the plug-in
The relevant parts of the view looks as follows:
public class CopyView extends ViewPart {
/* package for testing */Text myFromText;
/* package for testing */Text myToText;
/* package for testing */Button myButton;
@Override
public void createPartControl(Composite parent) {
final Composite top = new Composite(parent, SWT.NONE);
top.setLayoutData(new GridData(SWT.FILL, SWT.FILL, true, true));
top.setLayout(new GridLayout(2, false));
final Label fromLabel = new Label(top, SWT.NONE);
fromLabel.setLayoutData(new GridData(SWT.END, SWT.CENTER, false, false));
fromLabel.setText("From:");
myFromText = new Text(top, SWT.SINGLE | SWT.LEAD | SWT.BORDER);
myFromText.setLayoutData(new GridData(SWT.FILL, SWT.CENTER, true, false));
myFromText.setText("");
final Label toLabel = new Label(top, SWT.NONE);
toLabel.setLayoutData(new GridData(SWT.END, SWT.CENTER, false, false));
toLabel.setText("To:");
myToText = new Text(top, SWT.SINGLE | SWT.LEAD | SWT.BORDER | SWT.READ_ONLY);
myToText.setLayoutData(new GridData(SWT.FILL, SWT.CENTER, true, false));
myToText.setText("");
myButton = new Button(top, SWT.PUSH);
myButton.setLayoutData(new GridData(SWT.END, SWT.CENTER, true, false, 2, 1));
myButton.setText("Copy");
myButton.addSelectionListener(new SelectionListener() {
@Override
public void widgetSelected(SelectionEvent e) {
myToText.setText(myFromText.getText());
}
@Override
public void widgetDefaultSelected(SelectionEvent e) {
}
});
}
@Override
public void setFocus() {
myFromText.setFocus();
}
}
We can test at least three aspects of the view UI here
that the first field have focus initially. This is really more a view aspect here, but the same tests should be done for dialogs and wizards, so I leave it in.
that controls are R/W or R/O as needed and all controls are enabled
that the functionality of the "Copy" button is correct
To do any of this we must first show the view. This can be done with a method from BaseTestUtils. Likewise the view must be closed again after use.
@Before
public void before() {
myView = (CopyView) BaseTestUtils.showView("com.rcpcompany.testingplugins.ex.ui.views.CopyView");
}
@After
public void after() {
myView.getSite().getPage().hideView(myView);
}
To test that the fist field have focus we can do this:
@Test
public void testFocus() {
assertTrue(myView.myFromText.isFocusControl());
}
Pretty single, I would think.
Likewise to test the R/O of al envolved controls, we can do this:
First I test both text controls are empty, just the make sure the initial condition is well-defined.
Then we make sure the first control have focus, because that would always be the case in the real world. Not that it matters in this case, but there are cases with focus listeners where it would matter.
Next the content of the first control can be assigned to. This is identical to pasting the string into the control as both the verify and modify listeners are run - though the key or mouse events for the paste commands is not seen.
To press the "Copy" button, we must post an artificial event, as there are no way to do this via the Button API. Luckily BaseTestUtils includes a large number of methods that can help.
And last, we can test the result...
One could test even further... E.g. it is not clear in the test above that the text of the second Control is assigned to or appended to... I'll leave that as an exersize for interested...
To run the tests, select the project and "Run as..." "JUnit Plug-in Test"... In some cases, the test will not succeed first time, and you will have to set the application and plug-ins in the launch configuration first...
One technical note: if you make any changes to the dependencies of the fragment, then remember to all the "-clean" argument to your launch configuration. Otherwise, OSGi will not pick up the new dependency and you will be left with a long and frustrating time debugging the problems.... I should know :-)
Say you have decided to test your plug-ins, the first question of cause is, where exactly to put the tests. As it comes, there are a number of possibilities each with its own advantages and disadvantages.
In the following, I assume we are using JUnit 4 as this does not put any requirements on the super-classes. Otherwise, with JUnit 3, our options are a little more limited.
The tests can be put inline in the application plug-ins
Pros:
This is very easy to manage and understand. Also it is very easy to refactor the code, as you do not have to remember to have any specific test plug-ins or fragments in your workspace when you refactor your code - though it is always a good idea!
Your tests will have access to everything - you can even put the test in the same class as the methods to be tested.
Cons:
The tests will be in the final product, though it is possible to ge the Java compiler to leave out the test methods based on a special annotation. If you remove them from the plug-in during the release phase of the project, you have the additional problem, that what you test is not exactly your release...
If your tests requires any specific extensions in plugin.xml, then these will be harder to remove from the plug-in during compilation. This can be needed if you deckare any additional extension points in your target plug-ins and need to test these.
The tests can be put into a separate plug-ins
Pros:
The tests are separate from the product, so the cons of the previous option are not present here.
The test plug-in will have the same access to the exported Java packages as any "ordinary" consumer plug-in.
Cons:
Because of OSGi and the access rules it impose on bundles, you have only access to the exported Java packages. Of cause you can use x-friends to allow a specific visibility for specific package and friendly bundles, but who wants to do that if it can be avoided.
The tests can be put in an attached fragments
Pros:
The tests have access to everything as a fragment effectively "runs" inside the host plug-in.
Like the previous option, the tests are separate from the product.
You can even test new extension points and other other stuff that requires additions to plugin.xml...
Cons:
A fragment is a slightly different "thing" than a plug-in, and many developers shy away from them for this reason.
Some build tools have problems running tests that are put inside fragments - though these problems seems to be gone in the
I usually prefer a mix of fragments and plug-ins for my tests. The fragments are used for all tests of the internals of the target plug-in and the plug-ins are used for the needed black-box tests for the same target plug-ins.
One area where the plug-in approach is needed, is in order to test that the needed interfaces from the extension points are indeed exported and fully available. One problem you might otherwise see, is that super-classes are internal to the target plug-in which can give some very hard to understand error messages.
We all know, we should test our code... and if at all possible, we should write regression tests for the code as well. Just so we can make sure the functionality isn't broken as we add to the code or change the target platform. (And, of cause to make sure that we agree with the customer about the intended functionality of our code, right?)
This is even more relevant for Eclipse plug-in development. Especially since the Eclipse platform provides some very fancy - and sometimes complex - interfaces that can perform all sorts of stuff. One area of Eclipse plug-ins that really requires a special focus when testing is the use of configuration files (most notably MANIFEST.MF and plugin.xml) to specify major parts of the functionality. Here your usual junit tests require some refinements...
It is not because I just love to test... But the last 25 years as a developer have shown me time and time again that I spend more time doing fun stuff if I write a regression test the first time. As I spend all my time writing Eclipse plug-ins, this means that I have developed a large number of techniques for testing different aspects of a plug-in. Some of these are rather trivial while others have taken me a long time to get right.
I have found myself using a lot time on some specific tests... sometimes it is a bit of an intellectual challenge to test something right. This is especially true when you have to fix an odd problem from a customer and want to make sure the problem stay fixed.
As a consultant in everything Eclipse, I'm often asked about best practices for testing Eclipse based applications. Not that I can claim to know the best practices, but in the next couple of weeks, I intend to write a series of post about how I test specific aspects of a plug-in. These posts will focus on the main aspects of a plug-in as I see them.
MANIFEST.MF
User Interface
Everything that connects plugin.xml with the rest of the plug-in
Commands and Handlers
Expressions
Source Providers
Views and Editors
...and the functionality provided by the plug-in itself
Consider these posts my best answer for the best practices.
There are still room for improvements in my testing techniques and I hope that any readers will take the time to point me towards other techniques or tricks that might be relevant. Especially if this means, it is easier to test or if we can get a better coverage of potential problems.
At EclipseCon...
At EclipseCon, I will present some of these techniques in a talk Wednesday just after lunch - though in 20 minutes, we will just have time to touch of the different techniques and not go into any sort of details.
