contrarian notes on software engineering, Open Source hacking, cryptocurrencies etc.

My case against mocking frameworks

Since “mocks” and “mocking” is somewhat vague, and nuances between mocks, fakes, mock objects, test doubles, spies, etc. are confusing, let's start with what I mean by “mocking”.

What I mean by it is intercepting and/or substituting internal and often arbitrary function calls to test your code.

A great example of a mocking approach is a mocking framework like Mockito, where the tests look like this:

@Test
public void testAddCustomer_returnsNewCustomer() {
    when(daoMock.save(any(Customer.class))).thenReturn(new Customer());
    Customer customer = new Customer();
    assertThat(service.addCustomer(customer), is(notNullValue())); 
}

(copied from some Mockito tutorial, that I have now lost the link to).

In case it's not clear: In this test, a call to daoMock.save(...) is set to return new Customer(), instead of doing whatever real daoMock.save(...) would do.

Why am I opposed to mocking? Because I believe that composability is one of the most important qualities of code, and mocking results in code with low composability.

The first problem is that the whole idea of mocking relies on the test code knowing what the code under test will do, which is pretty much a definition of “testing internals” violating the “Test the interface, not the implementation” rule of thumb – described so many times, that I'm going to leave it at that.

Before diving into my argument any further, let me start with what I advise you to do instead of mocking: structure your code to be composable and thus easily testable:

Writing code in a functional way allows simple testing because one can just drive the inputs, and check the outputs, and there's no need to mock anything.

It's possible to write internal-behavior-exercising tests with dependency injection similar in essence to mocking-based tests. But by using DI, one is at least forced to explicitly describe the contract between the code under test and its dependencies and design it in a way that allows testing.

In my opinion, mocking is a very effective enabler for creating very highly coupled code. Since mocking frameworks allow one to go around normal boundaries and twist and turn anything to enable testing, it's very easy to test even the most coupled code, and that's the biggest problem with it in more complex projects.

I would like to say it again: it's not that mocking itself is a problem directly. It's just that mocking makes it easy to write highly coupled code and then write a test for it that kind of works.

Sure. In theory, it's possible to use mocking frameworks in a responsible way to test well-designed, decoupled code. In practice, in group settings, over time, with developers with various experience levels, opinions, and preferences, with deadlines, you're doomed to maintain poorly designed and tightly coupled code and its tightly coupled tests that require a lot of maintenance and slow you down.

Structuring code well, and then maintaining it across time to stay this way, is just difficult and a lot of work. Why bother with introducing DI in a given code, if it's easier to hardcode the dependency and then just mock it whenever needed? It works. Seductively, it works best when the project is small, and complexity fairly low like in a typical CRUD application. And then down the line, it's increasingly difficult to refactor anything significant, because everything is coupled, including thousands of tests that depend on some very particular details of implementation that they mock.

By not having a mocking framework available, you force developers to figure out a decent code organization, that allows writing tests by using only its composability. Yes, it might be more inconvenient, yes it might take a bit more effort and time, but in a long term, it will pay tenfolds.

It's a strategy similar or complementary to TDD. The goal of TDD is to force developers to design better code, by dealing with testing requirements upfront. Only this time it's about denying them access to any internals, so they can't avoid dealing with coupling upfront.

One can say: “you just need X” where X is a workaround like “higher quality code reviews”, “more experienced developers”, “better style”, “better patterns”, etc. This argument is analogous to the “dynamic vs static typing” debate to me. Yes it is true that typing is somewhat inconvenient when writing code and takes some time, and it is possible to write reliable JavaScript code, and use tests to ensure correctness, and attention to detail, etc. can help with weakness of dynamic typing, but in practice, there's never enough time to test everything, tests have cost too (runtime, maintenance), individuals and groups are unreliable to meet their desired standards; while (among many other benefits) modern type systems are lightweight and expressive, types steer people towards thinking more about how their data is organized, and help read and refactor and pay for themselves in the long run. Hard constrains of methodologies and tools drive the behavior of all developers more reliably then other means.

So in the same way that I am pro-static-typing, I am against mocking. Sure we can get the same results with mocking, possibly faster and easier. But I just don't think we actually will.