Duplication between Customer and Programmer Tests
Question from a post on the yahoo fitnesse group:
Say you got a person object and requirements state that the person passport id must be unique. Would you then write a test for that as a customer. Surely the same test will exist on a unit level. Also how would you specify it? Through an "Add Person" column fixture, adding the same person twice, or through a "Unique Person Passport ID" Test?
There's going to be some duplication between the customer tests and the programmer tests and that's perfectly OK. They serve different purposes. Assuming you're writing tests first, use fitnesse to describe how a customer uses the system. Then use xUnit to describe how some code can use other code.
In your passport example, you'd likely use a AddPersonFixture (ColumnFixture) and a PeopleFixture (RowFixture) on a page with a title "TestUniquePassports". This page would describe the requirement in free text and then use the fixtures as part of the description. Here's what the fitnesse page might look like:
The passport is the unique identifier for people in the system, and the system should not allow two user records with the same passport.
Say for example you add a person:
Now add the same person again. The system should inform the user that this passport is already present in the system.
Now add a different person, but use the same passport. Again, the system should inform the user that this passport is already present in the system.
And then make sure that there's only one user in the system.
Say for example you add a person:
| add person | |||
| first name | last name | passport | submit() |
| Joe | Smith | 987654321 | ok |
Now add the same person again. The system should inform the user that this passport is already present in the system.
| add person | |||
| first name | last name | passport | submit() |
| Joe | Smith | 987654321 | exception[PassportExistsException: "Duplicate passport"] |
Now add a different person, but use the same passport. Again, the system should inform the user that this passport is already present in the system.
| add person | |||
| first name | last name | passport | submit() |
| Jane | Roe | 987654321 | exception[PassportExistsException: "Duplicate passport"] |
And then make sure that there's only one user in the system.
| people | ||
| first name | last name | passport |
| Joe | Smith | 987654321 |
So what I'll do is comment out the fixture code that calls on my UserDatabase so that my code can compile and then start to develop just enough functionality to satisfy the needs of my fixtures, doing it all using TDD practices, which you can read about in a variety of places. Once enough code is unit tested and developed to uncomment the code in my fixtures, I'll do so. The likelihood is that I would get to the point where you can add users to the database and then get all of the users - nothing more. So I'll uncomment the fixture code, run the test and it fails because I didn't handle the error yet.
Now here's were a lot of people make what I view as the wrong decision. You've got a failing test in fitnesse, and it's interacting w/ your UserDatabase through your fixture, so the tempation is to just use the fitnesse test as your safety net and start coding. Here's the flaw in that thinking. The fitnesse page itself does not compile with your code. You're going to take a lot of steps to get that fitnesse test to pass and you're going to keep getting deceitful green bars that lie to you telling you that your code is sound. The problem is that you won't discover that you introduced a bug until you run the fitnesse tests, which you won't do for a variety of reasons, nor should you - they're not there to verify soundness of your code, they're there to verify requirements.
So write your unit tests even though exactly the same stuff is being tested in fitnesse (maybe even with virtually the same test).
FYI - the fitnesse example is a shameless plug for the exception[Type: "message"] syntax, which is only available in .NET Fitnesse at the moment. You can learn more about it and other syntax based cell handlers at http://fitnesse.org/FitNesse.DotNet.SuiteAcceptanceTests.SuiteCellHandlerTests
Check out UB's blog on customer and programmer tests as guides.
!commentForm
David, while I understand what you are saying, that unit tests are for verifying the soundness of code, and customer tests verify requirements, I still long for a way to test both with one mechanism. I mean, wouldn't it be nice if I could write a unit test of some sort that could then be plugged into FitNesse without having to duplicate anything. I'm a developer, and as such, nothing rubs me the wrong way more than duplication. I suppose you could share some helper functions between your unit test fixtures and your fit test fixtures. How do you feel about that?
In the rare case in which the customer and programmer tests align, it is likely only temporary. The customer and programmer tests move on different paths, at different speeds. Sometimes requirements are getting flushed out and the customer tests are getting modified. Sometimes development is underway and programmer tests are changing shape as structural changes reveal themselves as necessary. Relying on a mechanism that allows you to express that test in one way and use it in both worlds would tie your structure to behavioral requirements and vice versa.
Also, we rely on programmer tests for coverage. We trust our fellow developers to maintain our tests with the skills and knowledge they possess as developers. Putting that responsibility on the customer as well would create risk and be a burden on the customer. Can you imagine saying to a customer that the soundness of the system relies on her always getting your approval before making changes to the customer tests?
Also, we rely on programmer tests for coverage. We trust our fellow developers to maintain our tests with the skills and knowledge they possess as developers. Putting that responsibility on the customer as well would create risk and be a burden on the customer. Can you imagine saying to a customer that the soundness of the system relies on her always getting your approval before making changes to the customer tests?
I guess I never answered your specific question, Ben. How do I feel about having some helper functions that can be used in both customer and programmer tests?
Personally, I would avoid even that. What if a change I need to make to that method (to support a change in a unit test) has an adverse affect on a customer test? The likelihood is that this doesn't get noticed for a bit because although I run the customer tests regularly, I run them much less often than the unit tests.
Personally, I would avoid even that. What if a change I need to make to that method (to support a change in a unit test) has an adverse affect on a customer test? The likelihood is that this doesn't get noticed for a bit because although I run the customer tests regularly, I run them much less often than the unit tests.
Is thera a measurement of Customer test coverage ? Would one be meaningful ?
What an interesting question!
I think it would be meaningful, but not in terms of gleaning coverage as much as exposing unnecessary code. Imagine a scenario in which a requirement that has already been developed has been dropped, and the customer removes the acceptance test. You've now got code that is gumming up the system that should arguably be disposed of. Using the results of a Customer Test Coverage tool to identify the un-covered code as unnecessary code would be valuable.
I think it would be meaningful, but not in terms of gleaning coverage as much as exposing unnecessary code. Imagine a scenario in which a requirement that has already been developed has been dropped, and the customer removes the acceptance test. You've now got code that is gumming up the system that should arguably be disposed of. Using the results of a Customer Test Coverage tool to identify the un-covered code as unnecessary code would be valuable.
If a relational database is used, tests can be made completely unnecessary by having a unique constraint on the database table's passportId column. Then, the database will ensure that all passport ids will be unique. No tests, less work, everybody is happy!
Ravi - without any tests, if someone removes the unique constraint then no tests will fail, and the resulting bug would likely silently plague the system until it became a big problem. So regardless of where uniqueness is implemented, there MUST be a test for it if it is a requirement.
The debate here is whether it's OK to have that same thing tested in two places. Not whether it's OK to abandon testing it entirely. In my view, that is only OK if you want to build unreliable, unmaintainable software.
The debate here is whether it's OK to have that same thing tested in two places. Not whether it's OK to abandon testing it entirely. In my view, that is only OK if you want to build unreliable, unmaintainable software.
My experience has been that wherever there is good control, nobody removes constraints from databases just like that. Generally only the DBA has the rights to do so. As such, the sudden removal of a constraint occurs very rarely. And it does lead to robust software because the integrity constraints are in one place, the database, not spread all over different applications.
If the application developers want to write tests for it, that is fine. Time permitting, writing tests is fine.
To me, it is a question of separation of responsibilities. The database is excellent for certain types of integrity checks, and I prefer to use its capabilities for them. It is the responsibility of the database to ensure that some fields are unique, that there is referential integrity, etc. As an application developer, I'd rather spend my time on other types of business rules that are generally not handled in the database.
As a consequence of my approach, in this particualr instance, I would not have the problem of deciding all the places I have to write this test.
If the application developers want to write tests for it, that is fine. Time permitting, writing tests is fine.
To me, it is a question of separation of responsibilities. The database is excellent for certain types of integrity checks, and I prefer to use its capabilities for them. It is the responsibility of the database to ensure that some fields are unique, that there is referential integrity, etc. As an application developer, I'd rather spend my time on other types of business rules that are generally not handled in the database.
As a consequence of my approach, in this particualr instance, I would not have the problem of deciding all the places I have to write this test.
Ravi, in my opinion I would still want a unit test that addPerson() properly handles attempts to add duplicate rows.
Assuming that the persons.passport column has a UNIQUE constraint, the database is going to throw an exception when my code tries to add a duplicate row, that is fine, but what does my code do when that happens? The unit test will (among other things) verify that the expected behavior occurs.
Assuming that the persons.passport column has a UNIQUE constraint, the database is going to throw an exception when my code tries to add a duplicate row, that is fine, but what does my code do when that happens? The unit test will (among other things) verify that the expected behavior occurs.
Add Child Page to DuplicationBetweenCustomerAndProgrammerTests