Yes, this does make sense.

Notice that the set up code of the real object is one line less than the set up of the mock example. I think they are both clear too (IMHO).

As for whether to mock, I think that if the test kept breaking because of changes to Transaction validity, that would signal the need to mock. So this would be a good example of state based testing using real objects being too coupled with the test.

Maybe what we really need are some general guidelines for when to use mocks, stubs, real objects, etc., and when to use state based vs. interaction based testing. I can see using real objects with state based testing for stable, value objects. I can see using mocks with interaction based or stubs with state based testing for interfaces. It gets trickiest when your collaborating objects are classes. Maybe it depends on the stability of those classes, how expensive they are to use, how much set up is involved, how clear the test specifies the behavior?

Clint - consider the following examples:

context "An ATM with sufficient cash available" do
  setup do
    @dispenser = mock("dispenser")
    @atm = Atm.new(@dispenser)
    @atm.load(1000.dollars)
  end
  specify "should dispense requested cash when the transaction is valid (using a mock)" do
    transaction = mock("transaction")
    transaction.should_receive(:valid?).and_return(true)
    transaction.should_receive(:amount).and_return(50.dollars)
    @dispenser.should_receive(:dispense).with(50.dollars)
    @atm.dispense_cash(transaction)
  end
  specify "should dispense requested cash when the transaction is valid (using a real account)" do
    account = Account.new(100.dollars)
    transaction = Withdrawal(account, 50.dollars)
    @dispenser.should_receive(:dispense).with(50.dollars)
    @atm.dispense_cash(transaction)
  end
end

Imagine that the real transaction asks the Account if it is valid as part of determining its own validity. Every time a new rule is imposed on the Account to define validity (has_been_open_more_than_7_days?, !frozen?, etc) this example would have to change. Note that Account is a collaborator of the Transaction - it's twice removed from the Atm, which is the class we're interested in.

Whereas, in the example using the mock, changes to the meaning of Account and/or Transaction validity will never affect it. The things that might affect it are changes to the requirements of the Atm itself, but not it's collaborators.

Make sense?

What I disagree with is making a general statement about mocking everything out. Perhaps I’m misunderstanding Uncle Bob here: “We want to test the unit, the whole unit, and nothing but the unit.” I do believe the primary purpose and focus of a unit test is to test (or specify) the behavior of a single object, but if a real collaborating object is used, then you get some secondary testing.

We have choices available when considering how to set up the object under test with collaborating objects. Sometimes I use a stub, sometimes I use a mock, sometimes I use a fake, sometimes I use a dummy, and sometimes I use the real thing. So my point is to consider the alternatives and make the appropriate choice based on the contextual circumstances at hand.

In response to #1, I agree. If the collaborator didn’t exist, I’d consider an interface for the collaborating object and mock it out using a mocking library. If I later realized it was trivial and I didn’t think the interface was really needed (it added more complexity than benefit), I might swap in the real object. It all depends.

In response to #3, I refer to above. Consider the pros and cons and pick accordingly.

As for #2, I’m not quite clear what you mean. I think you have set up no matter if it’s a mock or stub or the real object. They all can reduce clarity of the test. As for changing the contract of a collaborating object and going back and changing the test, wouldn’t you have to change the mock anyway to match the new contract? Again, weigh the pros and cons.

Clint - I can think of a few cases when you may want to use a mock even for something seemingly trivial.

1 - the collaborator doesn't exist yet. You can use a mock in order to get your work done. When the real collaborator is ready, if it really is trivial and inexpensive to get it into a known state, then you can replace it.

2 - the things you need to do to get the collaborator into a known state are not clear in the context of the test you're writing. For example, setting a couple of properties on the collaborator so that when the class under test asks if the collaborator is valid? it answers yes. This sort of situation does two really bad things (IMO):

• One is it hides the true intent of the setup. A developer looking at that test may have to do a bunch of research in order to understand why the test is setting the values it is setting. Of course that could be resolved w/ a well named helper method or, god forbid, a comment. But I still prefer to keep this sort of noise out of my tests.
• The other is that should the criteria for the validity of the collaborator should change, this test would have to change even though this test is about a different object (violating SRP).

3 - getting the collaborator into a known state requires exposing its internals through setters, or worse, through methods that are "test only" methods (setUpKnownStateForTest).

I'm curious to hear your thoughts on this.

This is the first time I can remember disagreeing with Uncle Bob – Oh no. Specifically, “a good unit test tests nothing but the unit under test.”

Would you mock out a String (if you could, it’s final)? What if your object under test depended on a pretty stable and easy and inexpensive to set up object? Would you mock that out? What if you developed the test and code and near the end of the task realized an extract class refactoring of a simple, value class? Would you go back to the test and mock out the new class? You want to be able to refactor without having to go back and change tests.

I think mocking makes sense for interfaces and hard to create and expensive objects, but not for everything.

I think you are absolutely right.
What I would like to respond to Matts nice post is here:
http://sheyll.blogspot.com/2006/09/purpose-of-mock.html

I think you are absolutely right.
What I would like to respond to Matts nice post is here:
http://sheyll.blogspot.com/2006/09/purpose-of-mock.html

I think you are absolutely right.
What I would like to respond to Matts nice post is here:
http://sheyll.blogspot.com/2006/09/purpose-of-mock.html

Tanton - I agree with you. The overcomplication of handcoded mocks is one of the primary reasons that I prefer runtime mock object generation tools like jMock, nMock, and the built in mocks in rspec. Using these tools supports a consistent use of mocks without having to develop any code at all to support using them. You just use them.

This allows you to focus on design questions rather than whether or not you have to make a very complex mock in order to satisfy the needs of an interface.

I don't think it is possible to test a class in isolation. You are at least testing the class with respect to how it interacts with the mock objects. In effect, you are replacing one dependency (the production object) with another dependency (the mock object). The dependency on the interface is constant. Is this bad? Not necessarily and in many cases it is very good. If the production object is a "heavy-weight", then it is a good thing to get rid of the dependency. However, it can be the case that your mock object is just as complicated as your production object. In that case, you have added a component to the system without any gain (i.e., no dependency simplification). I think we need to be more careful about our mock usage and ensure that we really are replacing a more complicated dependency with a less complicated one.

Steve, I thought the very same thing about the setup (if you're talking about the first setUp() method, the 12-liner), which is one of the reasons I felt I must be doing things incorrectly. Would you agree that the rewritten version at the bottom is much cleaner?

-Matt McGill[?]

You're absolutely right. The arguments that Cedric make apply just as much to state-based testing, the issue is about the scope of the test not the technique.

In response to Matt, I don't understand why your setup is quite so complicated. Presumably it's doing a lot of work with some kind of data source implementation to create the connection. If possible, I would have mocked the data source to cut the dependency and tested the real connection creation elsewhere.

The last point is to make is that Mock Objects become more interesting when used to drive the design of an interface, not simply implement it. One of our Training Wheel rules is, Never Mock a Type You Don't Own. The point is to use Mock Objects, as with all TDD, to help drive the design of your code.

S.

Silly me. Now that I've looked up the wiki markup for code posting, here's the code for my offending tests:

  @Override
  protected void setUp() throws Exception {
    ds = createStrictMock(DataSource.class);
    conn = createStrictMock(Connection.class);
    ps = createStrictMock(PreparedStatement.class);
    md = createStrictMock(DatabaseMetaData.class);
    rs = createStrictMock(ResultSet.class);
    xaDsFactory = createMock(XaDataSourceFactory.class);
    defFactory = createMock(DefinitionFactory.class);
    typeMap = createMock(TypeMap.class);
    expect(defFactory.getTypeMap()).andStubReturn(typeMap);
    expect(typeMap.integer()).andStubReturn("INTEGER");
    db = new DatabaseHelperImpl(ds, xaDsFactory, "sqlDialect", defFactory);
    replay(defFactory, typeMap);
  }

  public void testExecuteStatement() throws Exception {
    expect(ds.getConnection()).andReturn(conn);
    expect(conn.prepareStatement("SQL Statement")).andReturn(ps);
    expect(ps.execute()).andReturn(false);
    conn.close();
    replay(ds, conn, ps);

    db.executeSql("SQL Statement");
    verify(ds, conn, ps);
  }

  public void testExecuteSqlThatThrowsException() throws Exception {
    expect(ds.getConnection()).andReturn(conn);
    expect(conn.prepareStatement("Failing SQL Statement")).andReturn(ps);
    SQLException exToThrow = new SQLException();
    expect(ps.execute()).andThrow(exToThrow);
    conn.close();
    replay(ds, conn, ps);

    try {
      db.executeSql("Failing SQL Statement");
      fail("Should throw exception");
    } catch (SQLException ex) {
      assertEquals("Wrong exception", exToThrow, ex);
    }
    verify(ds, conn, ps);
  }

Here's the code-under-test:

  public void executeSql(String sql) throws SQLException {
    Connection conn = null;
    try {
      conn = dataSource.getConnection();
      conn.prepareStatement(sql).execute();
    } finally {
      if (conn != null) conn.close();
    }
  }

And here's the rewritten tests for that same code:

  protected void setUp() throws Exception {
    fixture = new DbFixture();
    xaFactory = createMock(XaDataSourceFactory.class);
    defFactory = createMock(DefinitionFactory.class);
    db = new DatabaseHelperImpl(fixture.getDataSource(), xaFactory,"SQL DIALECT", defFactory);
  }

  public void testExecutingSql() throws Exception {
    db.executeSql("SOME SQL");
    assertTrue(fixture.wasExecuted("SOME SQL"));
  }

  public void testThatConnectionIsClosedOnException() throws Exception {
    fixture.throwOnNextStatementExecution(new SQLException());
    try {
      db.executeSql("THROWS EXCEPTION");
      fail("Should have thrown exception");
    } catch (SQLException ex) {}
    assertEquals(0, fixture.numOpenConnections());
  }

Just the other day I had a struggle over how to characterize good and bad applications of mock objects. I agree that a 'deceptive sense of confidence' is not really a problem. But what about coupling of tests to the code under test? Quickly, a disclaimer: I am not suggesting a problem with mock objects, rather what I believe is an example of their misuse. Now then, what about using mock objects to eliminate a test dependency on JDBC? I tried this a couple days ago, while working on some database helper code. My tests tended to revolve around verifying that Connections were close()ed despite thrown exceptions, that the correct SQL was generated, that sort of thing. I was using EasyMock[?] to mock the DataSource[?], Connection, DatabaseMetaData[?], PreparedStatement[?], and ResultSet[?] objects.

Here's what I found: because my code interacted so much with the JDBC API, my tests became a line-for-line specification of the desired code path through the code under test. I went along with it for a bit, until I had to change a couple methods in a trivial way - and realized that it was impossible to make *any* change without a subsequent modification to the tests. My tests were as tightly coupled to the implementation I chose as they could possibly be.

This led me down the following line of thought: what is the difference between a unit test and the unit it is testing? Both are specifications of the same behavior. The unit under test is specifying the desired behavior in terms of a logical sequence of more finely-grained units of behavior. The above experience causes me to say that unit tests should specify behavior by describing the desired outcome (outputs + side effects) given a well-defined starting state (the test fixture).

The problem with naively mocking the JDBC API like I did is that it requires me to specify the desired outcome (or behavior) in terms of more finely-grained units of behavior - which is exactly what the code under test is doing! So I came at the problem from a different angle. I'm still mocking the API, but with a custom set of dynamic proxies which monitor the JDBC calls, and a set of higher-level methods I can call to check on what has occured.

I'd love to post code, but without the ability to preview I have no idea what it would look like. I blogged this in more detail, with prettily-formatted Java illustrating my problem and solution. I'd *love* for some feedback from all you experienced old hands. I'm pretty new to all this OOD/TDD/BDD/XYZ stuff (I have a few more courses until my BS, but am working full-time as a developer), and have been following the postings and discussions on this site avidly. Here's the blog link: http://coffeedrivenjava.blogspot.com/2006/08/unit-tests-specify-post-conditions-not.html

-Matt McGill[?]
P.S. Is there some forum or group of forums where posters/readers of this site hang out and discuss? I'd love to get in on it!