Sunday, February 8, 2009

Test Everything That Could Possibly Break - A Guide To Better Testing

Joe: "Writing this test will make sure that we find bugs quicker. It will let us change the code without breaking anything and it will help us to write decoupled code."
Jim: "Maintaining this test will be a nightmare. It is tightly coupled to the class we're writing and we cannot change anything without changing the test. It will be a pain."
Joe: "How do you know?"
Jim: "Well, how do you know?"
Joe: "I have 20 years of experience not writing unit tests."

So what?
When I'm writing new code I am never sure whether I test enough, or if my tests are on the right level of abstraction. For complicated core functionality of a distributed system this is a no-brainer - I use TDD, which by the very definition gives me 100% code coverage, and add some nice integration and acceptance tests. But there are a myriad of cases where going forward in the baby-step TDD way seems a waste of time, and it would really help me to find some sensible rules to apply.

The simplest and best rule I have found so far is an idea from Extreme Programming:
Test Everything That Could Possibly Break

Unfortunately this rule is not as simple as it looks. My approach to that is the good old mantra of try-measure-adapt, where try means to just do whatever a randomly selected guy thinks is the new cool-aid, measure means to listen to that whimsical thoughts my brain produces while doing it and adapt means to look at the results and change my behavior.

So here's my guide to better testing, from the beginner to the professional level:

Beginner: Do some unimportant project by following the description of TDD step-by-step. Don't waste your employers money with that if you don't have any idea of how to do it - the first time you use it will be a disaster. Writing yet another Sudoku solver in your favorite programming language might be a cool idea.
The important part is that you don't have an idea of what "test everything that could possibly break" means, so your best bet is to assume everything might break. Even those getters and setters over there. Remember that you're not allowed to rant about made-up scenarios of why too many tests might be bad if you have never experienced what it means to maintain a program with too many tests. Do that first and come back later and read on.

Advanced: So you already have some experience doing TDD and know how it feels to write all those little unit tests. You got some feedback on when those tests caught a stupid bug that would have taken an hour to find if you hadn't written the test. You now know what the impact of unit tests on your ability to do refactorings is. Now go and break the rules by various degrees. Try to be less exhaustive with your tests and bundle your baby steps into bigger units of work. See how that affects your ability to find bugs. Test your assumptions and be aware of when they break. When you find a new bug that takes some hours to debug, think about what kind of test would have helped you find it quicker and write those tests from now on.

Expert: If I were an expert I could probably tell you more about what to do in that case. I still hope that repeating the advanced guidelines will finally make me as wise as Joel and Jeff in their discussions or Jay Fields when he writes about developer tests. Perhaps listening to those guys will enlighten you. You could even take a look at the very interesting discussion of the idea to test everything that could possibly break.

In a nutshell:

  • Start by testing everything, even if it looks stupid (don't do it at work).

  • Do slightly bigger steps and see what happens.

  • Adapt whenever you experience a situation in which different behavior would have made more sense.


  1. Nếu bạn đang muốn đăng tin bán nhà hay bán đất hoặc bạn muốn ban nha quan 12 , nha dat xinh thì hãy đến với chúng tôi rao vat mien phi, với chất lương hàng đầu chúng tôi sẽ giúp các bạn , đăng tin và xem các khu vực nha dat go vap, nha dat quan 9, nha dat thu duc , nha dat binh tan , nha dat tan phu , nha dat tan binh và các khu vực khác trên NguyenManhKha toàn quốc với uy tín và hiệu quả cao khi bạn đến với chúng tôi.

  2. Nếu bạn đang cần chuyển hàng từ thái lan về việt nam hãy liên hệ với chúng tôi. Chúng tôi sẽ giúp bạn mua hàng thái lan online một cách dễ dàng. Chỉ cần đưa thông tin sản phẩm bạn cần cho chúng tôi, chúng tôi sẽ mua và vận chuyển về Việt Nam. Khi chỉ là mỗi Thái Lan chúng tôi còn có các dịch vụ khác như: dịch vụ chuyển hàng từ mỹ về việt nam, dịch vụ vận chuyển hàng đi campuchia, dịch vụ đặt hàng quảng châu giá rẻ, .... Nếu bạn cần mua hàng trung quốc giá rẻ hay cần mua hàng trên taobao hãy liên hệ và sử dụng
    dịch vụ order hàng trung quốc của chúng tôi. Chúng tôi sẽ giúp bạn
    chuyển hàng trung quốc về việt nam một cách nhanh chóng và tiện lợi nhất.

  3. Are you looking for a gui hang di My but prestige and quality ?

    Long Hung Phat Co., Ltd - we are one of the internationl courier service providers, shipping to USA via office, internationl priority shipping provider tha providers you with the perfect balance between speed and price to meet the best demand for people in the coverage area around the world. By using the network spread across 3 regions of the country for all internationl shipments to other countries in the world, Long Hung Phat Express an provide you with a list of delivery destinations including America, Australia, Canada, Netherlands, New Zealand,...

    More info about Gui hang di My bao nhieu 1kg?

  4. Thanks for sharing.
    Nếu cần gửi hàng đi campuchia thì a/c liên hệ bên mình nhé.
    Hotline: 0903751981