Observations re: Hand Mocks vs Tool-Generated Mocks.

A while back, Micah and I had some debate about hand mocks vs tool generated mocks. Much water has passed under the bridge since then, and I have some observations I'd like to share.

First of all, hand coded mocks are concrete. Tests depend on mocks. That dependency violates the Dependency Inversion Principle (don't depend on concrete classes).

"So what?", you may ask. "Tests implicitly violate DIP because they depend on the classes they test". This is correct, but the important distinction here is that mocks are NOT under test. They support testing.

So what's the problem with this particular dependency?

Mostly it has to do with evolution of a system. In our current project, the tests use a mix of hand coded and tool generated mocks. As the interfaces of our mocked classes change, I've noticed that I have to make changes to the hand mocks and/or the tests that use them with pretty much every change that comes along. I've also noticed that most of the time this is not true of the tool generated mocks.

Additionally, there is a service layer that we mocked in many of our tests. Over time, we produced an entire mock service layer. This seemed to really aid development at the time, but has resulted in a mess of painful dependencies.

Not only do we have to make changes far away from our immediate problem, but there's an even more severe problem: changing the design of our service layer means changing the design of our mock service layer. Think about that. I want to improve the design of part of the system, but I have to do it to two parallel hierarchies. That's discouraging. And that has led to decisions along the way to leave things as-is rather than improve them right then.

Now, I'll confess that I'm part of the team that introduced and allowed these dependencies to persist. They really did make initial development appear less painful. It is only later in the project, when we try to make changes and find that we have to address them not only in our system, but in this parallel mock system (and the tests that use it), that the long term problem reveals itself.

It's ironic because we go out of our way to minimize dependencies in the code under test, and we go out of our way to ensure that the quality of our test code is up to the same standards as our system. In this case, it led us down a subtly evil path of parallel hierarchies.

The good news is this: the result of these observations has been that our service layer is morphing back to it's original form - very thin, simple APIs favoring dependency injection over service locator for both tests and the system under test.

One might ask if the problem is really one of hand mocks vs tool generated mocks? I'll tell you this. When the evil parallel hierarchy appears in a test using tool generated mocks, it is painfully obvious just looking at the test code. And that exposes the problem and discourages use of such a parallel hierarchy.

With hand mocks, it is more subtle because you can't see the problem all at once. You notice it as you're changing the system because you keep having to come back to change classes and tests that have nothing to do with the problem you're trying to solve right now.

There are other issues that are leading me towards a more firm commitment to mock tools over hand mocks. Look for more on this soon.

---

!commentForm