The default Qt approach
When we created our cm-tests project, Qt Creator helpfully created a ClientTests class for us to use a starting point, containing a single test named testCase1. Let's dive straight in and execute this default test and see what happens. We'll then take a look at the code and discuss what's going on.
Switch the Run output to cm-tests, and compile and run:
You won’t see any fancy applications spring to life this time, but you will see some text in the Application Output pane in Qt Creator:
********* Start testing of ClientTests *********
Config: Using QtTest library 5.10.0, Qt 5.10.0 (i386-little_endian-ilp32 shared (dynamic) debug build; by GCC 5.3.0)
PASS : ClientTests::initTestCase()
PASS : ClientTests::testCase1()
PASS : ClientTests::cleanupTestCase()
Totals: 3 passed, 0 failed, 0 skipped, 0 blacklisted, 0ms
********* Finished testing of ClientTests *********
We can see that three methods have been called, the second of which is our default unit test. The other two functions—initTestCase() and cleanupTestCase()—are special methods that execute before and after the suite of tests in the class, allowing you to set up any preconditions required to execute the tests and then perform any clean up afterward. All the three steps pass.
Now, in client-tests.cpp, add another method—testCase2()—which is the same as testCase1() but substitute the true condition for false. Note that the class declaration and method definitions are all in the same .cpp file, so you need to add the method in both places. Run the tests again:
********* Start testing of ClientTests *********
Config: Using QtTest library 5.10.0, Qt 5.10.0 (i386-little_endian-ilp32 shared (dynamic) debug build; by GCC 5.3.0)
PASS : ClientTests::initTestCase()
PASS : ClientTests::testCase1()
FAIL! : ClientTests::testCase2() 'false' returned FALSE. (Failure)
....cmcm-testssourcemodelsclient-tests.cpp(37) : failure location
PASS : ClientTests::cleanupTestCase()
Totals: 3 passed, 1 failed, 0 skipped, 0 blacklisted, 0ms
********* Finished testing of ClientTests *********
This time, you can see that testCase2() tried to verify that false was true, which of course it isn’t, and our test fails, outputting our failure message in the process. initTestCase() and cleanupTestCase() are still executed at the beginning and end of the suite of tests.
Now we've seen what passing and failing tests look like, but what is actually going on?
We have a QObject derived class ClientTests, which implements an empty default constructor. We then have some methods declared as private Q_SLOTS. Much like Q_OBJECT, this is a macro that injects a bunch of clever boilerplate code for us, and much like Q_OBJECT, you don’t need to worry about understanding its inner workings in order to use it. Each method in the class defined as one of these private slots is executed as a unit test.
The unit test methods then use the QVERIFY2 macro to verify a given boolean condition, namely that true is, well, true. If this fails, which we have engineered in testCase2, the helpful message failure will be output to the console.
If there is a QVERIFY2, then presumably there must be a QVERIFY1, right? Well, nearly, there is QVERIFY, which performs the same test but does not have the failure message parameter. Other commonly used macros are QCOMPARE, which verifies that two parameters of the same type are equivalent, and QVERIFY_EXCEPTION_THROWN, which verifies that an exception is thrown when a given expression is executed. This may sound odd, as we don’t ideally want our code to throw exceptions. However, things aren’t always ideal, and we should always write negative tests that verify how the code behaves when something does go wrong. A common example of this is where we have a method that accepts a pointer to an object as a parameter. We should write a negative test that verifies what happens if we pass in a nullptr (which is always a possibility, regardless of how careful you are). We may expect the code to happily ignore it and take no further action or we may want some sort of null argument exception to throw, which is where QVERIFY_EXCEPTION_THROWN comes in.
After the test case definitions, another macro QTEST_APPLESS_MAIN stubs out a main() hook to execute the tests and the final #include statement pulls in the .moc file produced by the build process. Every class that inherits from QObject will have a companion .moc file generated, containing all the magic metadata code created by Q_OBJECT and other associated macros.
Now, if you’re thinking “why would you test if true is true and false is true?”, then you absolutely wouldn’t, this is a totally pointless pair of tests. The purpose of this exercise is just to look at how the default approach that Qt Creator has pulled together for us works, and it does work, but it has a few key failings that we will need to work to fix before we write a real test.
The first issue is that QTEST_APPLESS_MAIN creates a main() method in order to run our test cases in ClientTests. What happens when we write another test class? We’ll have two main() methods and things won’t go well. Another issue is that our test output is just piped to the Application Output pane. In a business environment, it is common to have build servers that pull application code, perform a build, run the unit test suite, and flag any test failures for investigation. In order for this to work, the build tool needs to be able to access the test output and can’t read the Application Output pane in the IDE like a human can. Let’s look at an alternative approach that solves these issues.