Testing Mozilla Stumbler

Programming on Android is 97 kinds of awful.

The default choice is to use JUnit, which is mostly useless if you are using anything in the android.* namespace as a regular JVM will grind to a halt and die.

If you use robotium, you’re now involving at least an Android VM, or possibly running on real hardware – but the costs is that you will need to either spin up that VM or you need to have a device plugged in. Running tests involves loading code into your Android machine, starting up a full application and then instrumentation. This is slow and painful.

For Mozilla Stumbler, we made the choice to use robolectric[1] Robolectric fixes the “time to run tests” problem really well. Tests run in a regular JVM, they run reasonably quickly and you can get things running in a continuous integration platform without too much pain. We use it on Travis for the stumbler. Robolectric will play nice with JUnit4 so it’s not entirely jarring trying to wrap your head around it.

Your best bet here is to start reading some code to see what to do – a good place to start is the code that tests our software update in github.


@Config(emulateSdk = 18)
public class UpdaterTest {

    private Context ctx;
    private MainDrawerActivity activity;

    public void setUp() throws Exception {
        ctx = Robolectric.application;
        activity = Robolectric.newInstanceOf(MainDrawerActivity.class);
        Updater.sLastUpdateCheck = 0;

    public void testUpdater() {
        IHttpUtil mockHttp = new MockHttpUtil();

        Updater upd = new Updater();
        upd = spy(upd);

        // Setup mocks.
        // Replace the HTTP client and clock
        MockSystemClock clock = new MockSystemClock();
        long now = System.currentTimeMillis();
        ServiceLocator.getInstance().putService(IHttpUtil.class, mockHttp);
        ServiceLocator.getInstance().putService(ISystemClock.class, clock);

        // skip the exclusive wifi-only check

        assertFalse(upd.checkForUpdates(activity, ""));
        assertFalse(upd.checkForUpdates(activity, null));

        assertTrue(upd.checkForUpdates(activity, "anything_else"));
        assertEquals("1.3.0", upd.stripBuildHostName("1.3.0.Victors-MBPr"));
        assertEquals("1.3.0", upd.stripBuildHostName("1.3.0"));

Things to note:

You’re going to need the two annotations @Config and @RunWith on the test class to ensure that robolectric is enabled when your test is executed. If you forget these – you’re going to get bizarre errors when the regular JUnit4 tries to execute your testcases.

If you haven’t used JUnit in a long time, you’re also going to have to get used to the @Before and @Test annotations. Apparently things have changed in 10 years and methods named setUp or starting with test are no longer automatically picked up by JUnit.

There’s quite a bit happening in this little test. The Updater class is instantiated and we immediately wrap that with
a mockito spy. This lets us dynamically stub out behaviour – usually to no-op some function. In this case, we want to trick the Updater into thinking that a network is always available.

We also use a service locator pattern to dynamically inject dependencies into the stumbler. In this test, we inject
a fake system clock so that we can instrument time. The Updater class checks the system clock periodically – we want to
ensure that our test case runs the Updater with a known time.

The rest of the test case is the usual JUnit assertions.

If you run the tests from command line using : make test, you’ll get nice HTML output showing you your test results under the android/build/test-report/github/unittest/index.html path.

Library tests are also executed by make test and will show up under libraries/stumbler/build/test-report/debug/index.html

Those are the basics – robolectric lets you instrument android UI bits as well which I’ll cover in the next post.


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s