Package structure

The Symbian OS Unit package available from sourceforge contains:-

• Symbian OS Unit testing framework project (\SymbianOSUnit\)
• This tutorial document (\Tutorial\doc\)
• A Dummy project used to demonstrate the use of Symbian OS Unit (\Tutorial\dummyproj)

The package can be unzipped to your local disk and used with this tutorial without restructuring its contents.

Test Application

We will use the supplied “Dummyproj” application as the application we are going to test. The application doesn’t actually provide any real world functionality but provides us with a good example to demonstrate the use of Symbian OS Unit.

Lets create the project files for VC++ and have a look at the application. The code we want to test is contained in the StackClass.cpp and StackClass.h files. It is an implementation of a very simple stack that allows us to push and pop integers on and off the stack and retrieve the current stack size.

The rest of the application files produce a single dialog based UI containing a list box that allows us to add and remove items from the stack and keep a log of what has happened. Symbian OS unit would typically be used to test application engine code and not UI code so we will ignore this part of the application for this tutorial.
 

Click the Options menu and select ‘Add’. Enter a value of 12345 and click ok. You will see that the value you have entered has been added to the stack and the stack size is now 1. If you select Options and Remove then the value will be removed from the stack and size will now equal 0.

The application UI is allowing us to exercise the CStack class. Now lets move on to how we can test the CStack class using Symbian OS Unit.
 
 

Testing Overview

A Symbian OS unit project is built as an application in its own right. That is, we don’t include it in our application, we include our application files to be tested into a Symbian OS unit project.

Typically we would copy an empty \SymbianOSUnit project folder into our application folder and then set up the framework from there. However, Symbian OS unit can be used from wherever you wish as long as you set the paths to your application files correctly. (Note: For this tutorial we are going to use the structure that is delivered in the downloaded Symbian OS Unit package and not the structure shown here)

Unit testing sequence

The flow chart shown below outlines a typical development cycle when using Symbian OS unit.

Setting up the test framework

In order to use Symbian OS Unit to test our CStack class we need to include the dummy application paths in our mmp file. Navigate to the SymbianOsUnitApp\S60\Group folder and open the mmp file. Add the following lines to the file beneath all of the other SOURCE directives.

USERINCLUDE ..\..\..\..\Tutorial\DummyProj\inc

SOURCEPATH ..\..\..\..\Tutorial\DummyProj\src

SOURCE StackClass.cpp

Our stack class will now be visible to the Symbian OS Unit project. Create our build files (bldmake bldfiles) and our vc++ project files (abld makefile vc6) and then open the project in vc++. You could try building the project at this point but you will get errors as we have not yet written any tests or built the test runner.

You will see in the file browser at framework application and UI files and also a file called TestSource.cpp and its accompanying header file Testheader.h. This is the empty test suite file that accompanies Symbian OS Unit and where we will begin writing our tests.

TestSource.cpp

Before we write our first test we need to add our CStack header file to the test suite. So add the line #include “StackClass.h” to the TestHeader.h file.

We will describe the suites and the teardown and setup functions later in the tutorial. For now, lets start writing our CStack unit tests.

Writing our first test

Lets begin by writing a test to check that our stack object, once created, has an initial size of zero. We add our tests to a collection of tests called a suite. Test suites must be derived from CxxTest::TestSuite. A single empty test suite called CTest is already included in SymbianOS Unit.

All of our tests functions must begin with the word test (lower case) in order for them to be included in the testing framework. Any functions declared within the test suite class that don’t begin with the word “test” are excluded from the tests when we run our test suite. 

Let create a new test in our test suite called testAfterCreationL. We will use the test to verify that our stack object gets initialised correctly after creation.

Add the following lines to our test suite class definition in TestHeader.h:-

and the following lines to the test suite class code in TestSource.cpp :-

The test we have just created firstly creates a new instance of CStack, tests that after creation the current size is zero and then deletes the object.

Symbian OS Unit Asset Macro’s

We always within a test suite assert macro. There are currently eight different types of assert available. The assert we have just implemented allows us to check that the two parameters within the assert evaluate to the same value. If they don’t then the test will fail.

Generate the test runner

The test runner is code generated by a perl script that takes our test declarations and generates a header file called TestDriver.h. This is then used by the framework to execute our tests.

Whenever we update our test framework by adding a new test or test suite we have to rebuild the test runner code by running the testgen.bat batch file located in the

SymbianOSUnitApp\inc folder. Symbian OS Unit is slightly easier to use than some other C++ test framework alternatives since you don't need to "register" your tests. Running the testgen.bat executes a perl script which generates all of the actual tests for you and automatically registers your tests within the framework.

As we have added a new test to our test suite we must rebuild the test runner code now so execute the testgen.bat batch file.

Building the test application

You can now build the test appication within the ide and run the test application on the emulator.

When the application has been loaded you will see the main dialog as shown on the left. The progress/status bar at the top of the dialog shows the progress of the tests being run and the overall status of the tests. The status bar is green initially and if any of the tests fail it turns red.

 

If you click on the options menu you will see a ‘Run All Suites’ option and an option to run each individual test suite. As we only have one suite CTest selecting the CTest option has the same effect as selecting the ‘Run all suites’ option.

 

Select the ‘CTest’ option from the options menu to run our test suite. The test should pass and the status bar should stay green to indicate that all tests passed.

 

Now let’s add another couple of tests to the suite that check that we can push an entry onto and pop an entry from the stack. Let’s also check that an attempt to pop an entry from an empty stack generates an exception.

 

Add the following code to the suite. (Don’t forget to add the function declarations to the class definition).

Save the changes and run the testgen.bat batch file to update our testrunner code. Now run the tests.

On examination of the CStack pop function we can see that we don’t trap for pop being called on an empty stack so lets update our code. We will add an assert that will trap for popping an empty stack and leave with the KErrUnderflow error code. (Note: we will now be changing the pop function to a leaving function so don’t forget to append an L to function name and change the test suite references to popL)

Rebuild the test application and without altering any of our tests and run the test suite again. You will see that all of our tests now pass.

 

Test Fixtures

You will probably have noticed that we created and then destroyed a CStack object in each of the tests. This is time consuming and inefficient so we could employ a fixture to do the work for us. This is where we use the setup() and teardown() functions that you may have noticed declared in our test suite class. They allow us to define the common code that we want to run before and after every test function.

In our example we could move the creation of and destruction of the CStack object to the test fixtures and create a single CStack *iStack object in the class declaration.

Now when we add a new test we know that a newly created CStack object will automatically be created ready for us to test.

Test suites

So far we have only used one test suite in our test build. What if we wanted to write a number of tests that are very time consuming and therefore don’t want to run them every time we test? One solution is to put them in a different test suite. We are then able to just select the other test suite when we want to run the fast test during development maybe and only run the slow test on final build.

Lets create a test that pushes a number onto then off the stack a large number of times and put it in a new suite. You can create a new suite in its own file but we will just add it to our example file.

Lets call our suite CTestSlow to indicate that our tests in this suite are time consuming tests. AS previously stated a test suite must be derived from CxxTest::TestSuite

Add the following code to the testsource.cpp file:-

Add the new test suite class definition to the testheader.h file:-

Don’t forget to run testgen.bat to rebuild the testrunner before building the application for testing.

If you add a new test suite to your project (e.g. TestNewSuite.ccp,.h) in its own code and header file then you must update the test runner build file testgen.bat to include the new suite header filename:-

and also the ExtraTestBuildTasks.bldmake file:-

Memory leak checking

Symbian OS unit implicitly checks for memory leaks for each test. If a leak is detected at the end of a test then the test will fail. To demonstrate this functionality update the CStack constructor by adding an allocation is shown below.

 

 Logging

Now run the the CTest suite of tests. You will notice that all the tests fail with a “Test failed: Memory leak” test fail message because we are now allocating a byte during construction but not de-allocating it during destruction.

A log file containing a similar output to the text output on the application dialog is created automatically by Symbian OS. The file is named SOSUnit.log and is created in the “c:\logs\” folder of the emulator or target hardware. If this folder doesn’t exist then the log wont be created.

The contents of the log file is shown below for the memory leak tests we have just run above.

UIQ, UIQ3 and Series 80

                    UIQ                                  Series 60

                                      Series 80

UIQ3

 Important Note: 

When you now run the test application the menu will contain an option for our new test suite CTestSlow. Selecting this option will only run that suite. It will also be run as well as the original test suite when you select the Run All Suites option.

One of out tests has failed! The status bar has turned red to indicate a test failure and a description of the failed test is given in the output. The failing test is located at line 36 of file Testsource.cpp in the CTest suite. The output shows that we were expecting iStack->pop() to throw an exception but it threw nothing so we need to examine our CStack file to find the error.