There is one very important benefit of good test coverage that I'm a bit surprised to see missing from this chain - protection from/for "the next guy". Meaning this: even if /you/ are a genetically-altered coder who can naturally write 100% bug-free code 100% of the time, without tests your perfect code is at high risk to be speedily turned to bug-ridden mud by the poor soul who inherits your legacy once you've moved onto management.

Although I agree with your general sentiments, "it is not possible to know that the code works unless it is tested" is ignoring the points (effectively made in previous comments) that 1) if the app works at all, it's passing some tests by implication; 2) having lots of tests doesn't mean the code will work under all conditions. It may be helpful if you distinguish between broad high-level tests (it appears to work overall), finer-grained unit testing (these parts appear to work in isolation) and correctness (it *should* work).

I wonder if "Code that is not tested but works" actually means "code that is manually tested but works". I find it very hard to believe that anyone would write any program without at least trying to use it and throw some bad input at it to see what happens. That is testing too and I'm sure we all do it regardless of deadlines.

I could buy the argument that manual testing can ensure the same level of quality than automated testing (albeit with less efficiency) and I could agree that automated tests are not the only way to tell your code works. Sure you can allocate a bunch of resources to do manual testing and do it all over again when anything changes. Sure you could even reach the same level of coverage and ship quality code (even though you couldn't actually measure coverage).

The argument is that automated tests save you time compared to manual testing if only because you can do regression testing with one click.

The time it takes to actually write the tests depends on how familiar you are with the tools you are using. There will be a learnig curve but after that the tool becomes seconde nature and writting the test is just part of the development task. You have not completed a task until you have a test for it (something TDD takes care of nicely).

In this way testing is no longer this extra burden at the end of a cycle (be it manual or automated tests) but it is an integral part of development itself.

I think Cedric's point is good, I mean the middle ground may do exist, but his point his not good enough. There may be some guys who can write code without any testing, while they can also reach the quality that the test coverage over 90% could have. I am really admiring those guys who can write that sort of beautiful code, but the point is how can you identify who are that sort of guys, and how can you prove this right guy can write such a high quality code at this right time. I really doubt about that.

And somewhat this is the same as the US law system, you have the right to keep silent, but if you have something to say to the court, you have to get the evidence to prove it first.

There are many (including Aaron Korver, below) who point out that 100% code coverage doesn't imply correctness. Of course, Uncle Bob never claimed that; just that low coverage implies ignorance about correctness.

Some of Cedric's supporters on his blog posting really amaze me, though, and I wonder if Cedric is embarrassed to have them as supporters. Some mistake code coverage with all possible permutations of input: "There is no and never will be any code that is 100% tested. The time and effort to provide 100% testing would require infinite resources and time" Some seem proud of shipping buggy code: "we had extrem [sic] deadline, so we programmed it, shipped and then tested. there were lot of bugs. But having tight deadline that was our only alternative -- and we succeded. By the way, that software is for military logicstics use, and is installed in several countries:-) And yes, around 80% of code is NOT TESTED and it works without critical bugs." [I wonder if these critical bugs will only show themselves at critical times.] Some imply that testing is antithetical to real work: "This is why overzealous consultants who have never worked on a product with multiple releases and stiff competition have no business advising others what they 'should' be doing. It's almost downright reckless." Do they think that Cedric is arguing for lower code coverage on principal?

All, of course, assume that tests with high code coverage take a lot more time. Personally, I find that TDD development *saves* me time, and gives me a pretty respectable test-coverage by itself. Unit testing is not a panacea, but it's certainly no slouch at improving quality (when combined with corrective action).

Be careful about using the following code -- I've only proven that it works, I haven't tested it.

Donald Knuth

I'm sorry, but how does having a high test coverage mean that we aren't shipping shit? I defer to Steve McConnell[?].

"Testing by itself does not improve software quality. Test results [and code coverage] are an indicator of quality, but in and of themselves, they don't improve it. Trying to improve software quality by increasing the amount of testing is like trying to lose weight by weighing yourself more often. What you eat before you step onto the scale determines how much you will weigh, and the software development techniques you use determine how many errors testing will find. If you want to lose weight, don't buy a new scale; change your diet. If you want to improve your software, don't test more; develop better."

Steve McConnell[?] Code Complete

Thus, lets focus our attention on the development practices (TDD included) that help the quality go up, and stop the silly debate about code coverage.

The cost of going from 80% coverage to 100% coverage may well follow the Pereto principle if you write your tests after you have written the code and write the code after you have completed your design (ie. classic waterfall development). However, you have probably spent a lot of time writing code that isn't used or introducing unnecessary flexibility into your designs.

Following a more agile approach that is based on Test Driven Development where the tests are written before the code and the design is continuously refactored in light of the changing needs of the project can dramatically affect the amount of code that is written, and that therefore needs to be tested.

This goes a long way to paying for the "extra" 20% of test coverage. The rest will be paid back in reduced debugging and fixing of the software once it has been released.

In my opinion, our view of professionalism defines how we approach software development as a whole. If our view is that "I will do my utmost to deliver well designed, clean, and tested software" then we will naturally gravitate towards thoroughly testing the software we write. On the otherhand, if our view is "Ship it at any cost" then we won't. Unfortunately it's not as black and white as that in practice because of the variety of road blocks put in the path of a software team.

As a Project Manager who started out as a Software Engineer, I have come to realise that we must find an appropriate balance between delivering well tested code, and delivering less well tested code. Note that I said "less well tested" because we should NEVER ship untested code since it will come back to bite us. In my view, the goal should always be to develop tests that thoroughly test your code. Unfortunately, in my experience the cost of going from 80% coverage to 100% coverage follows the Pereto Principle (80:20 rule) and is therefore very expensive. I'm not convinced that we can get to 100% code and branch coverage in all cases, but we should endeavour to cover as mush as possible - preferrably > 80%. Sound engineering judgement must be applied when deciding on the level of testing being performed. I don't believe that a it is unprofessional to stop testing at 80% coverage if there are valid reasons to do so; however stopping at 80% WITHOUT having a sound reason IS unprofessional in my view. In the absence of sound reasons, we should strive to get the highest coverage we can.

A previous post hit the nail on the head (unfortunately) with the statements "most of the companies are in this business because of the money. Shipping shit so that nobody notices is a good business if you already got paid". This is poor business practice in my view, however it seems to be the predominent view in industry and it reflects badly on our profession. Regardless of whether we work in a team dedicated to shipping quality software, it is my belief that each Software Engineer must choose whether they are going to thoroughly test their code or not. A professional Software Engineer will use their engineering judgement when making that decision. Professionalism is a state of mind, and can be applied in any organisation.

Andrew Wall's comment about the need to test branches in order to approach 100% coverage got me thinking about how language features, and the way we use them, affect our ability to test comprehensively. I'm working with a client which has used way too many massive case statements and if blocks where polymorphism would have been preferred. (See the reafactorings in Fowler's book, "Replace Type Code with Class", "... with State/Strategy", "... with Subclasses". It just occured to me that the preferred polymorphic solution is much easier to cover with tests; something that isn't always mentioned when discussing refactorings like these.

What about language features themselves? Besides polymorphism through inheritance, do features like closures and continuations improve or inhibit testability? Has anyone researched this subject?

Is being professional important?

I mean, most of the companies are in this business because of the money. Shipping shit so that nobody notices is a good business if you already got paid.

That's what happens with most software projects and project managers know that. They might test critical code, meaning code that if it doesn't work, they may get sued. But otherwise, it makes no business sense.

I agree that it is unprofessional. On the other hand, companies are compared on ROI, not on professionalism. That's why most projects fail, by the way, because they guess wrong. The best professionals will never get the big bucks because they will walk the extra mile to make sure the software works before they ship. This means loyal customers and less ROI.

He names you in the second paragraph.

• Yes, he send me email the day I posted this, and very courteously asked if I would prefer being cited, to which I answered in the affirmative. So the citing, and the courtesy issues have been resolved.

If you search the internet you will find comments about code coverage. Several people say that 100% coverage is a minimum to ensure quality (they mean that you also need good branch coverage).
I would love the code I work with to have automated tests and I am working to improve the situation and I use TDD when I can to add new features.
However, I can confirm that 100% line coverage /is/ possible and >98% is a natural outcome of TDD.