Collateral Effort

I have something ugly to talk about, involving feelings and practices and resistance. The ugly phrase is Collateral Effort.

The concept is the set of things you have to do so that you can finally do the set of things you have to do. It has a resonance with the phrase "wasted time." In fact, the older term for this is simply "waste."

Agile software development is all about eliminating collateral effort. No unnecessary documents, no unnecessary meetings, no unnecessary code, no unnecessary anythings. The mantra is "maximize the amount of work not done." You eliminate as much as you can (and no more). Cutting out waste helps you to deliver important work sooner. At first, XP cut so much out of the process that people thought it was too radical to last, that management would never go for it.

It is good to have a least-encumberance mindset. The little voice in the back of our mind says "this is harder than it should be" and we start to look for a way to free up our time for more pleasant and productive tasks. Mind you, the young and eager get swept up in novelty and dogma, and may not hear those voices at all, but in experienced developers the voices are very strong.

The problem is that when you start doing agile development, the basic practices *feel* like collateral effort. Whether the perception is true or not, it feels like wasted time. It's especially poignant when you are installing agile practices in a non-agile code base. We find that a lot of effort goes into writing tests, and even more effort goes into trying to make the code testable. Now we have more files, more interfaces, more libraries (unit test libs, acceptance test libs, mock libs, helper functions), uglier constructors, more setters and getters, and maybe even more sophistication in our build/deploy systems to deal with all the new stuff. More time is spent in "collateral effort" of getting the system *ready* to test than goes into making functionality changes. Then when the functionality works, we end up spending even more time distilling the code into the smallest, most literate form we can imagine. It simply feels like time is being wasted. Even if we know better.

Managers feel it, and ask "wouldn't this be faster if you didn't pair" and "wouldn't it be faster if you didn't test?" They hear the developers complain, and they wonder if these practices add to the work load of every developer on the team. Is it "collateral effort"? Shouldn't an agile organization minimise the work product and the work effort?

That's the thing about reality; it does not feel constrained to meet our expectations. It is counter-intuitive that _additional_ practices let you complete the code _sooner_ and with better quality (and almost no overtime). Explaining that the extra steps are an investment (or an installment payment) won't exorcise the feeling. Explaining that you are actually moving in a more certain direction doesn't help. Telling people to feel differently doesn't help, it only makes them feel bad about feeling bad. You might as well give them permission to feel bad, so you don't double the pain of it.

While I don't know how to deal with someone else's feeling of wasted effort wrt agile process, I have noticed a DO/HATE/DONT/DO cycle:

DO: the developer does agile practices because they're told to.
HATE: the developer feels disgust at the perceived collateral effort
DONT: Given a chance, the developer tries going back to coding without tests or collateral, but it's too late.
DO: Having relived the non-agile way for a few days, the developer comes running back and convinces his friends that agile is a good idea.

Maybe we need to educate the little voices better before pushing the practices, to reduce the time it takes to get through the cycle. Maybe we need to find a way to educate the voices so that the middle two steps (HATE/DONT) are not necessary. I'm not sure what to do with this, to tell the truth.

I guess it's just an observation. Now that I've said it out loud, maybe you and I can try to reason about it.

!commentForm