Testing is crucial in the software development phase. It helps ensure easy debugging, agile code, and enhanced reusability. Performing tests that cover all use cases helps prevent a codebase from breaking — minimizing exposure to vulnerabilities. Python has two main testing frameworks that developers can use, doctest and unittest.

What is Doctest?

Doctest is an inbuilt test framework that comes bundled with Python by default. The doctest module searches for code fragments that resemble interactive Python sessions and runs those sessions to confirm they operate as shown.

Developers commonly use doctest to test documentation. The doctest module looks for sequences of prompts in a docstring, re-executes the extracted command, and compares the output with the command’s input given in the docstrings test example.

The default action when running doctests is not to show the output when a test passes. However, we can change this in the doctest runner options. In addition, doctest’s integration with the Python unittest module enables us to run doctests as standard unittest test cases.

What is Unittest?

Unittest test case runners allow more options when running tests, like reporting test statistics such as tests that passed and failed.

Unittest uses methods created in classes to manage tests. It has support for automation, setup, and shutdown code when testing. Unittest has several rich, in-built features that are unavailable in doctest, including generators and group fixture managers like setUp and tearDown. 

Since unittest follows the object-oriented method, it’s more suitable for testing class-based methods in a non-production environment. Continuous delivery tools such as Jenkins or Travis CI work better for production environments.
We’ll create a real-world example in the following sections, some code with customer information and discounts, and test it using doctest and unittest. Then we’ll analyze the tests and recommend the best ways to make further improvements.  

Using Unittest 

Let’s implement a Customer class and explore how we can test its methods using unittest.  

At the terminal command prompt, create a tests directory, then create a subdirectory named unittest. We’ll later create our Python testing scripts in the unittest directory. 

Use these commands to create the directory and subdirectory:

mkdir tests
cd tests 
mkdir unittest && cd unittest

Inside the unittest directory, create a Python script named unittest/customer.py. Then create a Customer class in that file as follows:

class Customer:
    """A sample customer class"""

    discount = 0.95

    def __init__(self, first_name, last_name, purchase):
        self.first_name = first_name
        self.last_name = last_name
        self.purchase = purchase

    @property
    def customer_mail(self):
        return f'{self.first}.{self.last}@email.com'

    @property
    def customer_fullname(self):
        return f'{self.first} {self.last}'

    def apply_discount(self):
        self.purchase = int(self.purchase * self.discount)

In the Customer class, create a constructor to instantiate our objects: first_name, last_name, and purchase. Then create these three methods: customer_mail, customer_fullname, and apply_discount. 

The customer_mail method returns a formatted customer’s email, customer_fullname returns the customer name, and apply_discount applies a five percent discount on the goods purchased. The @property decorator allows us to define the class methods like attributes through getters and setters.

To test the various methods we have created, we must create another Python file in the same directory and name it unittest/test_customer.py. It is a Python convention that all test files begin with a test prefix followed by the name of the file we’re testing.

Write the tests in the test_customer.py file like this:

import unittest
from customer import Customer

class TestCustomer(unittest.TestCase):

    def setUp(self):
        self.customer_1 = Customer('John', 'Brad', 5000)
        self.customer_2 = Customer('Tina', 'Smith', 3000)

    def test_customer_mail(self):
        self.assertEqual(self.customer_1.customer_mail, '[email protected]')
        self.assertEqual(self.customer_2.customer_mail, '[email protected]')

    def test_customer_fullname(self):
        self.assertEqual(self.customer_1.customer_fullname, 'John Brad')
        self.assertEqual(self.customer_2.customer_fullname, 'Tina Smith')

    def test_apply_discount(self):
        self.customer_1.apply_discount()
        self.customer_2.apply_discount()

        self.assertEqual(self.customer_1.purchase, 4750)
        self.assertEqual(self.customer_2.purchase, 2850)


if __name__ == '__main__':
    unittest.main()

Now, we import unittest from the Python package, then import the Customer class we created in the customer.py file. In the TestCustomer class, we pass the TestCase from unittest as it allows us to check specific responses for every set of inputs. We then create three individual tests with methods starting with the prefix test. The test prefix shows the test runner which methods to test. 

The setUp method is the first to execute when our tests run, and it’s where we define the customer instances. We use the assert method, assertEqual, to check equality for expected results in each test method.

For example, test_customer_mail checks whether the customer’s email corresponds to the one provided using the logic defined in the unittest/customer.py file and the attributes passed for that particular object. The same applies to the test_customer_fullname method.

In the last method, we apply the discount to the initial purchase and check the value passed. To learn more about various assert methods used in unittest, check out Python’s documentation. 

The final block shows a simple command-line feature that enables us to run our tests. If we execute our tests in the terminal using python3 test_customer.py, the output is as follows:

...
----------------------------------------------------------------------
Ran 3 tests in 0.000s

OK

All our tests passed. If we try to alter the last method in our test and change the discount of the second customer to “2750”, our code throws an error as shown below:

F..
======================================================================
FAIL: test_apply_discount (__main__.TestCustomer)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "test_customer.py", line 23, in test_apply_discount
    self.assertEqual(self.customer_2.purchase, 2750)
AssertionError: 2850 != 2750

----------------------------------------------------------------------
Ran 3 tests in 0.000s

FAILED (failures=1)

This testing helps to debug quickly and shows us where in the code we went wrong. By using unittest and writing tests that cover all use cases, we can improve our program’s readability, functionality, and logic flow.

Using Doctest

Using doctest is more straightforward than unittest, and has fewer steps. Although doctest is easy to use, we should use it wisely as there are several caveats. 

Let’s start our demonstration using the same example as before.

In the tests directory we created, create a new directory, name it tests/doctest and create a doctest_customer.py file in it using these commands:

mkdir doctest
cd doctest && touch doctest_customer.py

We now have two test directories, unittest and doctest. The unittest directory is where we created our unittest examples earlier. 
Inside doctest/doctest_customer.py, create the Customer class as follows:

class Customer:
    """
    A sample customer class
    """
    discount = 0.95

    def __init__(self, first, last, purchase):
        self.first = first
        self.last = last
        self.purchase = purchase

    def customer_mail(self):
        return f'{self.first}.{self.last}@email.com'

    def customer_fullname(self):
        return f'{self.first} {self.last}'

    def apply_discount(self):
        self.purchase = int(self.purchase * self.discount)
        return self.purchase

In the terminal, type python3 to get into the Python shell, then perform this operation:

Python 3.8.10 (default, Nov 26 2021, 20:14:08)
[GCC 9.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from customer import Customer
>>> customer_1 = Customer("John", "Brad", 5000)
>>> customer_2 = Customer("Tina", "Smith", 3000)
>>> customer_1.customer_mail()
'[email protected]'
>>> customer_2.customer_mail()
'[email protected]'
>>> customer_1.customer_fullname()
'John Brad'
>>> customer_2.customer_fullname()
'Tina Smith'
>>> customer_1.apply_discount()
4750
>>> customer_2.apply_discount()
2850
>>>

In the snippet above, we call some of the methods in our class from the command line. 

To run the doctest, place the output above into the class docstring like this:

class Customer:
    """
    A sample customer class

    >>> customer_1 = Customer("John", "Brad", 5000)
    >>> customer_2 = Customer("Tina", "Smith", 3000)
    >>> customer_1.customer_mail()
    '[email protected]'
    >>> customer_2.customer_mail()
    '[email protected]'
    >>> customer_1.customer_fullname()
    'John Brad'
    >>> customer_2.customer_fullname()
    'Tina Smith'
    >>> customer_1.apply_discount()
    4750
    >>> customer_2.apply_discount()
    2850
    """
    discount = 0.95

    def __init__(self, first, last, purchase):
        self.first = first
        self.last = last
        self.purchase = purchase

    def customer_mail(self):
        return f'{self.first}.{self.last}@email.com'

    def customer_fullname(self):
        return f'{self.first} {self.last}'

    def apply_discount(self):
        self.purchase = int(self.purchase * self.discount)
        return self.purchase

if __name__ == "__main__":
    import doctest
    doctest.testmod()

To be able to execute our tests, we import doctest below the file, and run our tests by executing python3 doctest_customer.py -v in the terminal. 

The output is:

Trying:
    customer_1 = Customer("John", "Brad", 5000)
Expecting nothing
ok
Trying:
    customer_2 = Customer("Tina", "Smith", 3000)
Expecting nothing
ok
Trying:
    customer_1.customer_mail()
Expecting:
    '[email protected]'
ok
Trying:
    customer_2.customer_mail()
Expecting:
    '[email protected]'
ok
Trying:
    customer_1.customer_fullname()
Expecting:
    'John Brad'
ok
Trying:
    customer_2.customer_fullname()
Expecting:
    'Tina Smith'
ok
Trying:
    customer_1.apply_discount()
Expecting:
    4750
ok
Trying:
    customer_2.apply_discount()
Expecting:
    2850
ok
5 items had no tests:
    __main__
    __main__.Customer.__init__
    __main__.Customer.apply_discount
    __main__.Customer.customer_fullname
    __main__.Customer.customer_mail
1 items passed all tests:
   8 tests in __main__.Customer
8 tests in 6 items.
8 passed and 0 failed.
Test passed.

All the tests passed. This demonstration makes it safe to say that doctest is more suitable for writing executable documentation for a package while unittest is better suited for testing documentation, and we can use this feature when reducing bugs in software documentation. For more information about doctest, see Python’s documentation.

Next Steps

Testing is a critical software development process since it ensures the code is clean and works as expected. Choosing the right way to test your code before executing tests is crucial. 

Performing tests using the unittest framework allows us to test our code in an object-oriented approach. It supports many features such as test automation, setup, and code test teardown. Doctest, in contrast, is more suitable for documentation since we can insert it in the code’s docstrings.

This blog post was created as part of the Mattermost Community Writing Program and is published under the CC BY-NC-SA 4.0 license. To learn more about the Mattermost Community Writing Program, check this out.