So the excitement of the days when I wrote stuff like Do You Understand XP? is gone. Rapid Development is a good book that reviews every practice it preaches with a critical eye, but I’m not overly excited. Yes, I learned, again, that many so-called Agile practices are known for a long time. But hey, I knew that after reading the Mythical Man Month. Excitement is hard to find in books these days. If I only could find one of those thought-provoking books, that rips your universe apart.

I finally discovered the truth about why developers rather rewrite a 1MLOC project from scratch than trying to understand a fellow programmer’s code:

We’re all code narcissists!

And the reason for that can easily deducted in a tiny logical chain:
(The best code is easy to understand) ^ (I can understand my own code the easiest, duh!)
=> (My own code is the best code)

Unfortunately this is only true for me, or perfect clones of myself. Which rules out everybody else I work with. Which reminds me… What was the reason that programming is done in teams?

So what?
When I’m writing new code I am never sure whether I test enough, or if my tests are on the right level of abstraction. For complicated core functionality of a distributed system this is a no-brainer – I use TDD, which by the very definition gives me 100% code coverage, and add some nice integration and acceptance tests. But there are a myriad of cases where going forward in the baby-step TDD way seems a waste of time, and it would really help me to find some sensible rules to apply.

The simplest and best rule I have found so far is an idea from Extreme Programming:
Test Everything That Could Possibly Break

Unfortunately this rule is not as simple as it looks. My approach to that is the good old mantra of try-measure-adapt, where try means to just do whatever a randomly selected guy thinks is the new cool-aid, measure means to listen to that whimsical thoughts my brain produces while doing it and adapt means to look at the results and change my behavior.

So here’s my guide to better testing, from the beginner to the professional level:

Beginner: Do some unimportant project by following the description of TDD step-by-step. Don’t waste your employers money with that if you don’t have any idea of how to do it – the first time you use it will be a disaster. Writing yet another Sudoku solver in your favorite programming language might be a cool idea.
The important part is that you don’t have an idea of what “test everything that could possibly break” means, so your best bet is to assume everything might break. Even those getters and setters over there. Remember that you’re not allowed to rant about made-up scenarios of why too many tests might be bad if you have never experienced what it means to maintain a program with too many tests. Do that first and come back later and read on.

Advanced: So you already have some experience doing TDD and know how it feels to write all those little unit tests. You got some feedback on when those tests caught a stupid bug that would have taken an hour to find if you hadn’t written the test. You now know what the impact of unit tests on your ability to do refactorings is. Now go and break the rules by various degrees. Try to be less exhaustive with your tests and bundle your baby steps into bigger units of work. See how that affects your ability to find bugs. Test your assumptions and be aware of when they break. When you find a new bug that takes some hours to debug, think about what kind of test would have helped you find it quicker and write those tests from now on.

Expert: If I were an expert I could probably tell you more about what to do in that case. I still hope that repeating the advanced guidelines will finally make me as wise as Joel and Jeff in their discussions or Jay Fields when he writes about developer tests. Perhaps listening to those guys will enlighten you. You could even take a look at the very interesting discussion of the idea to test everything that could possibly break.

In a nutshell:

• Start by testing everything, even if it looks stupid (don’t do it at work).
• Do slightly bigger steps and see what happens.
• Adapt whenever you experience a situation in which different behavior would have made more sense.

I acted just like I did back in school. Or worse. I waited for the last possible moment before I started preparing the talk. When I finally started with the slides, a little perfectionist devil Manuel sat down leisurely on my left shoulder, telling me that this crap is just not up to my own standards. I worked long hours the night before the big day, and woke up early just to be able to rehearse the whole play before entering the stage.

Of course everything worked out just fine. Well, besides me saying basic-a-lly all the time. So why was I so freaked out? Well, I was obviously leaving my comfort zone.

Now the interesting thing beneath all that personal drama I’m ranting about is that I suddenly realized for how long I did not really step out of my safe little comfort zone. Granted, applying at Google is not the most un-stressful experience I ever had. And obviously starting a new job with all those bright people around me did not exactly make me fell warm and cozy.

But at that very moment I stood there, my peers gazing absent-mindedly into their laptops, my hands slightly sweating, I suddenly realized that ever since I started working I was doing everything exactly in the way I was most comfortable with. And that was kind of a shock.

Of course after giving the talk I felt great. I had known that in advance, but I realized that without being nudged enough I’d probably have tried to wiggle out somehow.

Lessons learned:

1. It’s easier to step out of the comfort zone when somebody kicks your ass.
2. It’s a long way from leaving your comfort zone once to real change.
3. I must learn how to leave my comfort zone on my own.

Do you know tricks that make it easier to leave the comfort zone?

So I’m quite fond of using a simple script to execute my script’s unit tests whenever I save it. This concept is not new, and certainly not an original idea in itself, but the simplicity of an editor independent unit test executor in 10 lines of code has a certain appeal for me:

#!/bin/bash
stat_command="stat -c '%Y'"
file_name=$1
last_modification=""
while true; do
  current_modification=$( $stat_command $file_name )
  if [ "$current_modification" != "$last_modification" ]; then
    clear
    $file_name --test
    last_modification=$current_modification
  fi
  sleep 1
done

This script stats the script file until it detects a change. Whenever a change is detected, the script is called with –test, which is my personal way to tell a script that it should just execute it’s unit tests and exit. See my blog post about integrating unit tests in Ruby scripts to learn how this can be done in Ruby. A very similar approach is possible for Python:

#!/usr/bin/python
import unittest
import sys

if sys.argv.count("--test") > 0:
  sys.argv.remove("--test")
  unittest.main()

Now I can simply call the test bash script, giving it the script under test as parameter:

./run_tests.sh ./script_under_test.py

The beauty lies in the simplicity of the solution: Even when I remote edit a script on some server with vi, I can simply launch a new console and execute run_tests.sh, watching the test results whenever I type “:w”.

Update: The “sleep 1″ really helps to keep I/O load down. Thanks to Philip for pointing this out. And yet another nice example of how hard it is to write 10 lines of bugfree code without a test.

#!/usr/bin/ruby -w
require 'test/unit'

if ARGV.include?("--test")
  ARGV.delete_at(ARGV.index("--test"))
else
  Test::Unit::run = true
  puts "Running..."
end

Now you can simply run the program by typing

$ ./testme.rb
Running...

or run the tests with

$ ./testme.rb --test
Loaded suite ./testme
Started

Finished in 0.0 seconds.

0 tests, 0 assertions, 0 failures, 0 errors

The nice thing is that only the first “–test” will be removed, so you can still leverage the Test::Unit command line argument interface.

assertThat { assertThat(false) } abortsWith TestFailedException

Today I played a little with OCaml, sorted my functional programming skillz out, and finally arrived at my first test driven unit test environment for OCaml! Note that it’s nearly what I wanted to be able to write, but unfortunately there’s not enough syntactic sugar for lambda expressions (or I didn’t find out, yet), so I’m stuck with using the quite ugly ( function () -> expression ) syntax. But hey, it’s really close to heaven.

exception TestFailed
exception TestError

let failsWith expectedError expression =
	try
		expression ();
		false
	with error ->
		expectedError = error

let isTrue expression: bool =
	expression

let isFalse expression: bool =
	not (expression)

let assertThat expression conditionMatchesOn =
	if not (conditionMatchesOn (expression)) then
		raise TestFailed
	else
		()

let _ = (
	assertThat true isTrue;
	assertThat (isTrue true) isTrue;
	assertThat (not (isTrue false)) isTrue;

	assertThat false isFalse;
	assertThat (isFalse false) isTrue;

	assertThat (TestFailed = TestFailed) isTrue;
	assertThat (failsWith TestError (function () -> raise TestError)) isTrue;
	assertThat (failsWith TestFailed (function () -> raise TestError)) isFalse;
	assertThat
		( function () -> assertThat false isTrue )
		( failsWith TestFailed );

	Printf.printf "OK\n";
);;

The idea of Test-After Development is to write a set of automated white-box tests after writing your production code. Since probably every CS student in the world has learned that unit tests are a good idea, you’d expect unit testing to be an industry state standard for quite a while now. Interestingly the idea of automated unit and integration test is lately becoming more popular due to the widespread use of Test-Driven Development.

So why do we need Test-Driven Development to be able to efficiently write automated unit tests?

• If you write your code first and don’t think about how to test the code, the code will not be testable. Thus testing becomes expensive and frustrating. Test-Driven Development will guide your software design by the old mantra of “how-do-I-want-to-use-this-class”, leading to a highly decoupled design.
• When you write your tests, you’ll discover a lot of errors. But instead of the red bar in Test-Driven Development, which you expect, the red bar in Test-After Development is the demotivating sword of reality.
• The most important reason why I have never seen Test-After Development work, is that developers just don’t believe in errors once they wrote the code. This seems to be an eternal wisdom of software development psychology: once the code works, why bother testing it? Let’s just implement the next feature.

“Don’t you want to start?”, she asked and watched me curiously. “Um”, I said, obviously always finding the right words at the right moment. “Um. – Not before you buckle up…”. She frowned at me: “But I never buckle up”. I replied “Well, if you don’t buckle up, we’re not gonna go anywhere tonight”. “Oh come on!”, she now somewhat furiously stated. “Nope!” I insisted eloquently, finally feeling her shield of stubborn resistance falter. It took a few seconds before she realized that I really wouldn’t drive her home without her being properly protected from falling through the windshield and decomposing her pretty head by hitting the next best fireplug. So she buckled up.

Even back than I was so used to the secure feeling of the protective belt that just thinking of driving unbelted drove an uneasy quiver through my guts. This feeling is so strong that if I don’t drive strapped when moving the car just a few inches there’s always a sense of awareness that makes me want to fasten the seat belt immediately.

Today I felt exactly the same way while writing code.

The path of the test

At the beginning of the last iteration we identified a story that affected some legacy modules in our code base. When we recognized that the changes we needed to make would touch more code than we had thought, so we decided to try to test drive a part of the system from scratch to replace the tangled old code. So Richard, Reinhard and myself started to pair on the story alternately. Besides some private experiments with a Sudoku solver in Java this was the first time I was doing real full time TDD pair programming for a couple of days. Aside from some initial irritation and the constant realization that pair programming is hard to learn I was quickly pulled into the red-green-refactor cycle, as usual. But this time I held the pace for longer than ever. And was pulled into the cycle deeper and deeper. Write a test, make it work, look for redundancy. Write a test, make it work, look for redundancy. Write a test…

Today I wanted to quickly integrate the changed interface into an existing module. I didn’t have a test yet. The cpp file was readily opened in my editor. Just a quick edit, nothing more than integrating this interface. A few simple edits. Only three lines or something. And suddenly a nagging question materializing in my head:

How can I make sure this works?

At this moment I felt like driving unbuckled. I felt unsafe. I wanted my cozy safety net back. Like an addict I went for the next test.

What use is a seat belt when you hit a tree at 200 MPH?

Since I’m the one driving the adoption of XP in our company, I wanted to try TDD for myself on a save playground to learn more about the ins and outs before applying it at work. Since Java has really nice tools for TDD, I started test driving a small Sudoku solver in Java. This was my first real test driven code and I often wondered about how nicely the test suite covered my errors. Spirited in the Agile fashion, I began with a really straight forward brute force implementation. Everything went a lot more smoothly than I had expected and after some coding I had a simple solution that needed over 90 seconds for one simple Sudoku.

After a while I wanted to optimize the runtime. So I introduced some caching variables. I struggled with the failing tests as my solution grew more and more sophisticated, but the tests helped me to get to a deeper understanding of the real problem. Finally I arrived at a point where the algorithm managed to work through 1400 Sudokus in less than a second. I was thrilled. And I wanted more. So I installed a profiling framework to find out where the next optimization sweet spot would be hidden. When I browsed the profiling data I realized that the real solver didn’t even call the algorithm. So I had benchmarked a program that didn’t solve any Sudoku at all.

At this moment I felt like hitting a tree with 200 MPH, suddenly realizing that it is not a good idea to drive that fast into a 90-degree turn on a wet street, even if you have a seat belt.

After the blood had returned to my head on it’s way to my brain I implemented a test into the main program to check every solution with a simple algorithm before claiming to have solved anything. In the meantime I have a solution that runs 1400 Sudokus in 6 seconds on my core 2 notebook. I’m even quite convinced that I got the solution part correct…

Do I get my driver’s license?

So, here’s the lesson I learned on this journey on my path to the test:

• Don’t rely on your tests too quickly.
  If you want to heed the XP advice to “test everything that can possibly break”, be aware that it’s often the things of which you think that they can’t break that finally break.
• Use a healthy mixture of tests on all abstraction levels.
  Unit and functional tests are orthogonal – they cover different aspects of the code. But of course you’ll already have a lot of unit and functional tests if you don’t rely on your tests too quickly.
• Buckle up!
  The unsafe feeling while trying to modify code without having a test was a very impressive experience for me. I know that from now on I’ll fasten my code’s seat belt.

Karl: ‘Let me see what you wrote just now…

usUserName = request.getParameter("UserName");

Um… This variable, usUserName, what does the us-prefix stand for?’
J.B.: ‘Well, that is the unescaped user name the way we get it from the user. I wanted to make sure that we don’t accidentally write it into a database or send it back in it’s evil, unescaped form to the webbrowser. If we use the us-prefix every time we have an unsafe string, we’ll immediately recognize any error that could otherwise escape us because we will learn to look for such errors. This is the application Hungarian notation I read about over at Joel’s site, where you actually use a prefix that has a meaning instead of just a shorthand for the type.’
Karl: ‘So… why not just call it unescapedUserName?’

Confusion

In a time where you enter veryLongVariableNames by typing ‘v’, ‘e’, ‘r’, Crtl-Space, I don’t see why we can’t finally get rid of TLAs. You know that the Hungarian notation got out of hand when your colleagues check in code that changes ucpBuffer to pucBuffer (”fixed a segfault”). Why not just name a variable for what it contains, in plain old English? I definitely know that I should think about my method name if my partner asks during a pairing session: “And what exactly do you intend to do in this method?”.

In which example is the error easier to spot? Does the second example really take longer to write? To read? To understand?

for(unsigned int i = 0; i < iLineCount(); ++i)
{
  for(unsigned int j = 0; j < iNodeCount(i); ++i)
  {
    pGetNode(i, j)->layout();
  }
}

for(unsigned int lineIndex = 0;lineIndex < getLineCount(); ++lineIndex)
{
  for(unsigned int nodeIndex = 0; nodeIndex < getNodeCount(lineIndex); ++lineIndex)
  {
    getNode(lineIndex, nodeIndex)->layout();
  }
}

If you can really remember mnemonic prefix TLAs (or any TLAs for that matter) and think Hungarian notation or abbrVarNames are a great way to safe yourself some typing, please let me know.