Testing Call Order in Python's `mock` Module

Testing Call Order in Python's `mock` Module 01/13/2018

Today I had to test something I'd never needed to test before... the order of calls in a validation method mattered, and I wanted a unit test to confirm I never break this in the future.

Here's the code that needs to be tested:

class MyClass(object):

    def _validate_a(self):
        # do some stuff

    def _validate_b(self):
        # do some stuff

    def validate(self):
        self._validate_a()
        self._validate_b()

Here's a trivial test class that confirms validate() calls both validation methods:

import unittest
import mock

class MyClassTest(unittest.TestCase):

    def test_validate(self):
        item = MyClass()
        item._validate_a = mock.MagicMock()
        item._validate_b = mock.MagicMock()
        item.validate()
        item._validate_a.assert_called_once()
        item._validate_b.assert_called_once()

... but this does not confirm the order they were called in.

The Solution

The solution is to set up a Mock object to act as a manager of mock calls. We then add the methods to the mock object (as well as to the item under test), call our validate() method, and confirm the call order in the manager:

class MyClassTest(unittest.TestCase):

    def test_validate_order(self):
        item = MyClass()
        item._validate_a = mock.MagicMock()
        item._validate_b = mock.MagicMock()

        mock_manager = mock.Mock()
        mock_manager.attach_mock(item._validate_a, 'a')
        mock_manager.attach_mock(item._validate_b, 'b')

        item.validate()

        mock_manager.assert_has_calls([
            mock.call.a(),
            mock.call.b()
        ])

We are doing the following things:

  1. set up the item to be tested and mock out the methods we are looking for
  2. create a manager entity that can track what's happening to the mocks on our item being tested
  3. calling the method under test
  4. comparing the list of manager actions to the expected actions

Note that when we call attach_mock on the manager that we pass both the mock to be watched and a name for that method. We will refer to this method within the manager as a property named the same as the passed string.

Note also that the assert_has_calls method only ensures that items are called in the order specified. The manager does not confirm other methods were or were not called in between. Think of the list passed to assert_has_calls as a sub-list of the list of all calls on the mocks, and so long as this is a superset of the full list of calls then the test will pass. For instance:

        # setup as before
        mock_manager.assert_has_calls([
            mock.call.b()
            mock.call.a(),
        ])

... fails because the methods were not called in order, but:

        # setup as before
        mock_manager.assert_has_calls([
            mock.call.a(),
        ])

... passes because the call to the listed mock did occur within the overall call list, and:

        # setup as before
        mock_manager.assert_has_calls([
            mock.call.a(),
            mock.call.a(),
        ])

... fails because a() was only called once, not twice.