Testing a Plug-in - Where to put the tests

This post is part of my "Testing a Plug-in" series.

Say you have decided to test your plug-ins, the first question of cause is, where exactly to put the tests. As it comes, there are a number of possibilities each with its own advantages and disadvantages.

In the following, I assume we are using JUnit 4 as this does not put any requirements on the super-classes. Otherwise, with JUnit 3, our options are a little more limited.

The tests can be put inline in the application plug-ins
  • Pros:
    • This is very easy to manage and understand. Also it is very easy to refactor the code, as you do not have to remember to have any specific test plug-ins or fragments in your workspace when you refactor your code - though it is always a good idea!
    • Your tests will have access to everything - you can even put the test in the same class as the methods to be tested.
  • Cons:
    • The tests will be in the final product, though it is possible to ge the Java compiler to leave out the test methods based on a special annotation. If you remove them from the plug-in during the release phase of the project, you have the additional problem, that what you test is not exactly your release...
    • If your tests requires any specific extensions in plugin.xml, then these will be harder to remove from the plug-in during compilation. This can be needed if you deckare any additional extension points in your target plug-ins and need to test these.
The tests can be put into a separate plug-ins
  • Pros:
    • The tests are separate from the product, so the cons of the previous option are not present here.
    • The test plug-in will have the same access to the exported Java packages as any "ordinary" consumer plug-in.
  • Cons:
    • Because of OSGi and the access rules it impose on bundles, you have only access to the exported Java packages. Of cause you can use x-friends to allow a specific visibility for specific package and friendly bundles, but who wants to do that if it can be avoided.
The tests can be put in an attached fragments
  • Pros:
    • The tests have access to everything as a fragment effectively "runs" inside the host plug-in.
    • Like the previous option, the tests are separate from the product.
    • You can even test new extension points and other other stuff that requires additions to plugin.xml...
  • Cons:
    • A fragment is a slightly different "thing" than a plug-in, and many developers shy away from them for this reason.
    • Some build tools have problems running tests that are put inside fragments - though these problems seems to be gone in the 
I usually prefer a mix of fragments and plug-ins for my tests. The fragments are used for all tests of the internals of the target plug-in and the plug-ins are used for the needed black-box tests for the same target plug-ins.

One area where the plug-in approach is needed, is in order to test that the needed interfaces from the extension points are indeed exported and fully available. One problem you might otherwise see, is that super-classes are internal to the target plug-in which can give some very hard to understand error messages.
Post a Comment