Not Yet Test Driven
I don't think I'm test-driven yet.
I've done a lot of testing in the past, and I know the value of it. That's not the issue. I've used various test frameworks, unit and otherwise. I'm learning FitNesse[?] (as a good OM should). I do testing while doing development, and am even now learning to test first. But it's not happened. I'm not well-schooled, having only seen the live presentation once, and then picking it up on-the-job (with mentors) but it's not happened yet. I'm not test driven.
I've learned about mocks, and am clumsy and have to slowly think through them. Why? Because I'm not test-driven yet, possibly? Or because I'm old (over 40 and still coding!) and slow in the head?
Am I writing tests first? Yes, as long as I have a partner or force myself, which means 100% of the time now. But even with mocks, and fitness, and testing first, and pairing, and appreciating testing, I'm not really test driven yet. So if I'm carrying out all the practices, what's missing?
Because I'm having to research code (including existing tests) to know what tests to write. It's not fluid and it's not a design activity for me really. I'm still testing code. I'm having to either do a lot of research, or tie up hours (open-endedly) in trial-and-error testing. Neither seems to bee what we're after.
Admittedly, I'm joining a project in progress already. There is a significant code base I know almost nothing about. Granted, I'm learning how to function in a new team, in a new programming language, in a new/existing code base, and on a different operating system and library set than I already know. It's a lot of learning curve.
I think I'm not Test-Driven because I don't have any magical powers yet.
A sample solo flow:
1.Write some test (naively) as a starting point.
2.Run it so it fails wrongly because other objects interact.
3.Add objects to the test harness
4.Fail because the interaction isn't well understood, other method calls are needed
5.Add more “expected” methods to the test
6.Run again, failing the wrong way still because there are more objects and more method calls in the existing system not taken into consideration in the test (ie. It has fan-out).
7.Read the code (a non-TDD behavior?), learn what the interactions are, add them to the test.
8.Run the test, maybe finally getting a "proper" failure.
9.Stare blankly at the screen, hoping you can still remember why you're writing it.
10.Reread the test as-written so far, looking for clues.
11.Rename some variables and methods to make it more clear.
12.Wonder what all those methods and variables are doing, and why I need them just for test envisioned in step #1
13.Read a bunch more code (very non-TDD I'm told).
14.Scrap the whole test and start over on a different track, hoping that I don't end up scrapping it as well.
Is all of this trial-and-error code “discovery” really a productive use of my time? If I were TestDriven[?], perhaps I would know. Perhaps I wouldn't have to go read the code. I'm told that I should be able to “just sit down and write tests” without knowing the SUT. I have my doubts, not being test-driven.
A sample pair situation:
Someone who already knows the code base flashes from screen to screen, typing snippets of code into functions I don't know in classes I've never seen before, narrating verbally on the fly, writing tests that somehow always seem to be implementable in the system, and then demonstrating that they fail correctly first, then succeed correctly secondly.
If I were test-driven, I'm told, I wouldn't be lost and I would be able to anticipate and follow better. Instead of being told what to write when I'm “driving”, I would just know, you know “use the force” or something. Of course, I see a lot of trial-and-error code discovery even among people who seem to be Test Driven, and false starts too. So perhaps I am just being hard on myself.
I'm not convinced that there is a reasonable substitute for knowing what you're doing. I'm not convinced that even someone who is magically imbued with power from TDD will automagically write tests that can be made to pass in unfamiliar territory. I'm not convinced that a few hours of trial-and-error is more productive than a few hours of “backgrounding.”
It may be that I've not reached that pinnacle. Hopefully, someday I'll be able to look down on it and laugh.
"If I were test-driven, I'm told, I wouldn't be lost ..."
I'm not sure where you learned that (hopefully not from ME), but that's absurd.
Regardless of your level and experience, a new system takes some time to get to know. When you're joining an active team I think that your team-mates bear the responsibility of bringing you up to speed on the system. This can be facilitated by pairing, but if a pair is moving too fast for you, then you both share in the responsibility of finding a pace and communication that serves the purpose of getting you up to speed. The benefits of you becoming productive more quickly outweigh the costs of getting you there (in my view).
I'm not sure where you learned that (hopefully not from ME), but that's absurd.
Regardless of your level and experience, a new system takes some time to get to know. When you're joining an active team I think that your team-mates bear the responsibility of bringing you up to speed on the system. This can be facilitated by pairing, but if a pair is moving too fast for you, then you both share in the responsibility of finding a pace and communication that serves the purpose of getting you up to speed. The benefits of you becoming productive more quickly outweigh the costs of getting you there (in my view).
Tim, the way I do it is to use tests to explore my design space. When I want to test something I have to express the very thing that I am testing, so that I have to write down a call to a believable snippet of code that should produce whatever kind of result I intend to obtain.
After you have written that snippet of code you can sketch it by just declaring some barebone classes and methods. Just enough to get your code to compile.
In the meantime you also verify that your test can specify exactly the context for the calls that you want to make and that it is able to check that it obtained a correct result. This will probably force you to rethink the initial code calls so that you can set up the object properly and perform checks on the output. This step usually forces you to decouple your code and to introduce some kind of dependency injection.
I have written something on the topic that you might or might not like. You can find it at: Maru Batsu and at intent.
This is not proper unit-testing but it seems to work for me.
After you have written that snippet of code you can sketch it by just declaring some barebone classes and methods. Just enough to get your code to compile.
In the meantime you also verify that your test can specify exactly the context for the calls that you want to make and that it is able to check that it obtained a correct result. This will probably force you to rethink the initial code calls so that you can set up the object properly and perform checks on the output. This step usually forces you to decouple your code and to introduce some kind of dependency injection.
I have written something on the topic that you might or might not like. You can find it at: Maru Batsu and at intent.
This is not proper unit-testing but it seems to work for me.
Maybe I missed your point or maybe you missed mine. I'm not sure.
It's not a matter of more literate style. The problem isn't in how cleanly I can express my tests. It's not that I don't get the flow or the three rules of TDD. However, those things are much easier said than done when you're poking around in an unfamiliar code base (hence the snarky "magic powers" bit).
The whole of the issue is in "write down a call to a believable snippet of code that should produce whatever kind of result I intend to obtain." It seems that you kind of have to know the existing code base (or not have existing code) in order to pull that off. That's the sentence in which I can easily spend 1/2 a day or more, and have to revert (via version control) and start over a few times when we realize that I'm in a cul-de-sac.
I hope part of it is that I'm leaving a set of productive habits for a new set of untested (by me) habits. I do not see that a few hours of poking around trying to write a credible snippet of code via the AT and UT only to revert and start over is really productive learning time. It feels like a total waste of time, and I know that I could read the code and draw diagrams and on my own know what code to write. Then I can write the tests presuming the code I know I'm going to write next.
The problem isn't whether it's proper unit testing or not, but rather it's TEST-DRIVEN DESIGN that I'm doing. If I go with my habits and proclivities, I'd be doing code-driven testing, rather than test-driven design. What I'm hoping I can learn here is how to use testing to learn my way around an unfamiliar system in a productive way. When reading the code and drawing a picture, I get a sense of how far I've come and what's left. I build context. When writing a snippet and running the test (or just the compiler, whatever's needed), I'm feeding a code machine rather than learning a system. It feels a bit random. I need some kind of "hill climbing" algorithm: the idea being that if you're wanting to go to the top of a hill, one method is to keep moving in an upward direction. There are times in testing when I can't tell if I'm driving toward a solution, away from a solution, or entirely parallel to one.
Frankly, I think that the test syntax in those links is wonderful, but they aren't an answer for *this* problem. Syntax of the tests isn't the issue at all; content, focus, and direction are needed.
It's not a matter of more literate style. The problem isn't in how cleanly I can express my tests. It's not that I don't get the flow or the three rules of TDD. However, those things are much easier said than done when you're poking around in an unfamiliar code base (hence the snarky "magic powers" bit).
The whole of the issue is in "write down a call to a believable snippet of code that should produce whatever kind of result I intend to obtain." It seems that you kind of have to know the existing code base (or not have existing code) in order to pull that off. That's the sentence in which I can easily spend 1/2 a day or more, and have to revert (via version control) and start over a few times when we realize that I'm in a cul-de-sac.
I hope part of it is that I'm leaving a set of productive habits for a new set of untested (by me) habits. I do not see that a few hours of poking around trying to write a credible snippet of code via the AT and UT only to revert and start over is really productive learning time. It feels like a total waste of time, and I know that I could read the code and draw diagrams and on my own know what code to write. Then I can write the tests presuming the code I know I'm going to write next.
The problem isn't whether it's proper unit testing or not, but rather it's TEST-DRIVEN DESIGN that I'm doing. If I go with my habits and proclivities, I'd be doing code-driven testing, rather than test-driven design. What I'm hoping I can learn here is how to use testing to learn my way around an unfamiliar system in a productive way. When reading the code and drawing a picture, I get a sense of how far I've come and what's left. I build context. When writing a snippet and running the test (or just the compiler, whatever's needed), I'm feeding a code machine rather than learning a system. It feels a bit random. I need some kind of "hill climbing" algorithm: the idea being that if you're wanting to go to the top of a hill, one method is to keep moving in an upward direction. There are times in testing when I can't tell if I'm driving toward a solution, away from a solution, or entirely parallel to one.
Frankly, I think that the test syntax in those links is wonderful, but they aren't an answer for *this* problem. Syntax of the tests isn't the issue at all; content, focus, and direction are needed.
Tim, I know in what pain you are in. Sometimes, TDD-enthusiasts (like myself) get carried away with the "magic of TDD" and stuff.
But for me TDD must enable you to achieve simple design. If you are trying to solve a seemingly simple problem, then you should be able to mock out all dependencies. If the problem is not that simple, then your pair should take the time to explain the existing design to you.
TDD is a force which helps us to question the design : if you do have many dependencies (step 12), maybe it's time to stop for a while and question the existing design. For me, NOT questioning the design and flashing from screen to screen is a bit like falling over to the dark side...
But for me TDD must enable you to achieve simple design. If you are trying to solve a seemingly simple problem, then you should be able to mock out all dependencies. If the problem is not that simple, then your pair should take the time to explain the existing design to you.
TDD is a force which helps us to question the design : if you do have many dependencies (step 12), maybe it's time to stop for a while and question the existing design. For me, NOT questioning the design and flashing from screen to screen is a bit like falling over to the dark side...
mmhh.. I see that I missed you were refering only to an existing system. I was talking about doing TTD with a new system. In this case it looks to me that it is more a case of getting to know the system, which you should be able to do by looking to the tests. If the tests are not expressing the intent of the underlying code then, IMHO, there is something wrong with either the tests or the code.
What about a step-by-step refinement of the tests? You could start with coarse grained tests, where you just state in pseudo code what result you want. Then you look at other tests to work out exactly what classes to call to get what you want.. but it seems to me that somehow you need to go back to the code, which seems to highlight a problem with the expressivness of the existing tests. Since tests document code behaviour I assume that if you need to dry run the code in your head by looking at it, then the existing tests are not that clear.
I think that understanding this issue could help other people too. I like the way you 'reflect' on your own actions and thinking processes.
What about a step-by-step refinement of the tests? You could start with coarse grained tests, where you just state in pseudo code what result you want. Then you look at other tests to work out exactly what classes to call to get what you want.. but it seems to me that somehow you need to go back to the code, which seems to highlight a problem with the expressivness of the existing tests. Since tests document code behaviour I assume that if you need to dry run the code in your head by looking at it, then the existing tests are not that clear.
I think that understanding this issue could help other people too. I like the way you 'reflect' on your own actions and thinking processes.
in your sample solo flow...
if you need to pull in a complicated tangle of objects to run your first test, it seems like the system isn't designed for testability, or perhaps your test isn't a unit test so much as an integration test.
if you need to understand how to use a method call on an existing class, the first place to look is the unit tests for that class.
fitnesse should not be about unit testing, it's integration/acceptance testing.
imho, tdd is about unit testing.
i personally don't use mocks - martin fowler talks about diiferent testing styles in http://martinfowler.com/articles/mocksArentStubs.html
if you need to pull in a complicated tangle of objects to run your first test, it seems like the system isn't designed for testability, or perhaps your test isn't a unit test so much as an integration test.
if you need to understand how to use a method call on an existing class, the first place to look is the unit tests for that class.
fitnesse should not be about unit testing, it's integration/acceptance testing.
imho, tdd is about unit testing.
i personally don't use mocks - martin fowler talks about diiferent testing styles in http://martinfowler.com/articles/mocksArentStubs.html
While you're reading Fowler's article, you should also read this one that describes a very useful approach to mocks http://www.jmock.org/oopsla2004.pdf
Sounds like this blog shouldn't be about TDD but rather about pairing.
Well, the reason for this was a comment that rather than spending time learning the system, I should perhaps dive straight into writing tests and let the tests teach me the system. I'm not convinced that TDD can do that. Maybe pairing can, maybe individual exploration can, but I'm not sure that TDD is the answer. Or maybe, that it's never been intentionally focused that way before.
I think you can use TDD by itself to learn a system, but I don't see any way of keeping it from being like your steps 1-14 above. IMHO, it's doable but not the most effective way, especially if there are people around to ask questions of and work with.
What might be interesting about all of this is consideration of what you take for granted when you write your next test, and what you take on faith. That's interesting. You probably already know the design so far, and the platform, and certain things about what kinds of fixtures you can write and how your change will be evidenced. That's a fair amount of backgrounding. Having that in mind, you can write about any test you want. Lacking all of that, how would you progress?
Tim,
Try to spend as much of the pairing time "driving" as possible. This will force your pair to bring you up to speed bit by bit. Yes, the productivity will be hammered in the short term, but if it was worth bringing another programmer onto the project, then it's worth the productivity hit of bringing them up to speed!
Try to spend as much of the pairing time "driving" as possible. This will force your pair to bring you up to speed bit by bit. Yes, the productivity will be hammered in the short term, but if it was worth bringing another programmer onto the project, then it's worth the productivity hit of bringing them up to speed!
Add Child Page to NotYetTestDriven