Mocks and their influence on software design

When speaking about unit testing, one automatically also talks about mocks and the need to mock away dependencies. It seems to be quite a common attitude towards mocks along the lines of "Mock every external dependency of the class under test" and this attitude can be quite dangerous. Therefore, here are some words of caution on the implication that the heavy use of mocks in your code base can have in terms of the overall system design or architecture.

TL;DR

Mocks are used to provide isolation in unit tests and thus should help to create software that is easy to test and easy to change. But in fact, an over-usage of mocks in any unit test suite can cover up underlying issues in the software design that make the software hard to test and hard to change. So Mocks should be treated as one tool in your toolbox that one can use to build high-quality and maintainable software. That means one should know when to apply this specific tool, or when should rather use another tool from the toolbox.

Mocks are hard to refactor

Be cautious when utilizing mocks extensively because it can be hard to refactor classes automatically. This is because IDEs do not provide robust support for refactoring classes that are heavily mocked, and tools like PHPStan may not effectively detect these mock-related issues.

More broadly speaking, it is hard to guarantee that the mock behaves in the same/or intended manner as the real implementation (especially when the underlying implementation changes).

Use mocks only where you need to because:

• creating the objects is hard as you need tons of nested dependencies to create the object.

or

• the class produces some side effects you don't want in unit tests (e.g., DB writes).

For all other cases, use real implementations and rely as minimally as possible on the magic of PHPUnit's mocking framework.

Focus on behavior, not implementation: Effective unit testing principles

Relying heavily on mocks creates a bad pattern in unit tests of testing `how` something is implemented and not `what` the implementation actually does. If tests are implemented in a mock-heavy way, they are tightly coupled to the implementation, meaning they rely on implementation details and may fail more often when the implementation details change than when the actual behavior of the class under test changes. Consider these two example changes to some classes:

Before:

After:

Before:

After:

By definition, both changes are a pure example of refactoring:

Refactoring is a disciplined technique for restructuring an existing body of code, altering its internal structure without changing its external behavior.

-> Martin Fowler

But when the unit test mocked the repository or connection dependencies, the unit tests will fail after the change, even though the external behavior (that's what a test should really test) was not changed.

Using mocks is ok in some cases, but not all.

Probably the examples from above are ones that are totally valid (as the mocked classes rely on a DB), which can be commonly encountered in real life.

Furthermore, the intention of this document is to keep you aware of the downsides that come with using mocks.

Mocks might indicate your class is not well-designed

In a well-designed and testable system, it is relatively easy to isolate individual classes or modules and distinguish them between the components that contain the core business logic, which should be extensively unit tested, and the portions responsible for interfacing with the external environment and generating side effects. These side-effect-prone elements should be substituted in the unit tests. In fact, it is advisable to perform integration testing since their primary purpose is to abstract and facilitate the replacement of side effects in tests.

This kind of abstraction follows when you apply the principles of Domain Driven and Hexagonal Architecture (aka Ports & Adapters)

The absence of such abstraction in the existing shopware/platform codebase is one of the reasons why it is so hard to write "good" unit tests for shopware, but that does not mean that we should keep designing our code as we used and keep writing "bad" (meaning unit tests tightly coupled to the implementation) unit tests.

However, it's the opposite; we start designing our code in a way that makes it easy to write "good" unit test that does not rely that much on a "magic" mock framework.

So, a heavy reliance on mocks when writing unit tests can indicate a potential issue with the software design, suggesting insufficient encapsulation. Hence, designing code to promote better encapsulation and reduce the need for extensive mocking is advisable. This can lead to improved testability and overall software quality.

Better options than mocks

There are better options, but that depends on the use cases. Here are a few alternatives:

1. Use the real implementation (this means the real thing is easy to create and does not produce side effects)

2. Use a hand-crafted dummy implementation of the real thing, that is easy to configure and behaves like a stub in that use case (this means that the real thing probably needs to be designed in a way to be easy to replace, examples of this in our test suite are the StaticEntityRepository or StaticSystemConfigService)

3. Fallback to using PHPUnit's mocking framework (when the real thing is not designed to be replaced easily)

The way you design your codebase directly impacts whether you can rely on option 1 or option 2 without resorting to heavy mocking.

Conclusion: Write tests first!

When you write tests first, most of the points described above should come out of the box! Nobody who starts with a test would start with configuring a mock. While we provide insights on this, it is essential to validate the information. So we encourage you to explore the following references to gain a deeper understanding and form your own opinion.

References

Frank De Jonge on the exact same topic (with more examples in PHP): https://blog.frankdejonge.nl/testing-without-mocking-frameworks/

Martin Fowler on the differences between mocks (option 3) and stubs (option 2): https://martinfowler.com/articles/mocksArentStubs.html

Presentation by Mathias Noback on testing hexagonal architectures: https://matthiasnoback.nl/talk/a-testing-strategy-for-hexagonal-applications/

Here are some good real-life examples of unit tests in PHP: https://github.com/sarven/unit-testing-tips

A great write-up on testing in general: https://dannorth.net/2021/07/26/we-need-to-talk-about-testing/

Quite an old (1997!) paper on how not to use code coverage: http://www.exampler.com/testing-com/writings/coverage.pdf

Great blog post series on how to avoid mocks: https://philippe.bourgau.net/categories/#how-to-avoid-mocks-series