Software Development Processes

Today, the software development processes are a topic of heated discussion. The whole industry faces a stiff breeze from the agile faction that promises a new course on the quest for high quality software. The agile department on the other hand is confronted with FUD like not having processes at all.

So what are processes in a software development context? When we refer to processes we usually mean rules that specify when or how to do work. Of course we always follow some informal "process" when we work, but I'll refer to processes in the more formal sense of defined rules.

When I was a kid many of my friends had to finish their homework before they were allowed to play outside, while I was free to do my homework as long as I didn't get problems at school. The school thus imposed a when-process on me (I had to finish my homework for the next class) while my friend's parents created a process for their kids.

The school is also a nice example for how-processes. My eight year old nephew Marvin will learn how to do subtraction by hand this year. His teacher summoned all the parents and told them that this year a slightly different version of the borrowing algorithm will be taught instead of the method that pupils learned in German elementary schools for over five centuries (the difference is to "borrow" -1 from the minuend instead of +1 from the subtrahend if the digits can't be subtracted directly). This way the school defines how the children do their calculations - if they use a different way, they'll get bad marks.

So now that we know what processes really are, how do we find the best processes for a software development process? As always, this is a highly recursive problem. If processes help at the lowest level, they probably will help us to develop better processes. So we have to find processes to optimize the processes that optimize the processes. The good old fixed point problem - again.

Usually you try start a fixed point recursive problem with a trivial termination case. There's an easy recursive when-process.

When (not) to use processes.

If people always did the right thing, we would not need processes at all. In a software development environment we usually assume that all people work towards the same goal - delivering high quality software. But even if all team members had the same goals and were motivated to always do the best possible task for the company, they usually have different opinions on what is "best". Those different opinions may lead to childish fights about who should have done what and who's responsibility it was. Both parties think their solution is the best for the company and in this case a defined process can help to resolve conflicting goals.

And this is exactly the mother of all processes: Use processes (only) to enforce decisions when you're quite sure that no superior outcome can be achieved without defining processes. This is often hard to anticipate. At school I used to do my homework in the morning of the day I needed it, sometimes even during the breaks. But I always did all my homework myself. Some of my friends who were forced to finish their homework used to lie about finishing it, just to copy it at school. Those usually didn't do very well. Here the process obviously didn't help at all.

So let's try to get to the next step: what is the most obvious process that a software development team is willing to accept but wouldn't start on their own?

Use a process definition and optimization process.

Let the team figure out some processes for themselves. Let the team decide on the processes. Since the team usually has most experience in what runs smoothly and where to find the rough edges, the team should be able to identify processes that work for them. The team needs to buy into the processes anyway, and the biggest chance to get them to buy in is to have them decide it.

Of course not every team will discover processes like pair programming and test driven development for themselves. But they don't need to. It's no problem to present a fine set of processes, like XP or Scrum to the team for evaluation.

If the process definition and optimization process doesn't work, impose processes.

Let's face it: some teams just don't want to define processes or have anything to say about the way they work. They just want to get their paycheck and do as little as possible just so they don't get fired. And this is not a small part of the human species. In such a team you have to be weary because many good processes will fail, and you'll probably end up with a myriad of process definitions that are used as an excuse not to think.

But as long as you didn't try the process definition and optimization process you don't know for sure that it doesn't work. Software development is a highly creative process where motivation can cause a difference in productivity of an order of magnitude. Development processes are tightly coupled to a good work experience and thus to motivation. All this leads to the conclusion that software development processes are a very important aspect in software development.

Today, as a grown-up I'm mostly responsible for the processes I use myself. Nobody tells me when or what to learn, when to work out and what to eat. And I find that I learn more, work out harder and eat healthier. A coincidence?

The Touched Piece Rule

Agile software development is a fixed point solution to a recursive problem.

When I was a little kid, my mother put me into the local chess club. It took me until today that I realize the impact the training had on me - other than spending my weekends with a bunch of geeks and not being able to get near girls (I don't count the chess playing ones).

As I learned later when I myself became a chess trainer, one of the difficult things in training young kids is to keep them interested in 'boring' things like playing chess. They usually start with a lot of enthusiasm, learning the rules, playing their first few games - until they recognize that there's a huge difference between knowing the rules and playing chess.

I remember when I was in that situation myself. It was just so hard to figure out which of the many options is the best move. I didn't spend another thought on what happened back than for years. But today I realize that Reinhard, our chess trainer, didn't just lecture us if we made a mistake. He just said: "What was the reason you made this move?".

It didn't matter if I found the best move during a game. It just mattered that I thought about the situation for some time and found the reason why I believed my move would be a match winner. For chess there's even an interesting rule that encourages thinking things through before making your move in a rush. It's called "the touched piece rule": if you touch a piece on the board you have to move it - even if it may crush your position.

One of the interesting things of games like chess is that they closely resemble life per se. Every moment of your life you've got a myriad of options to choose from. Every moment of your life you make a decision which move is best. Being a self-conscious human being makes this experience highly recursive and results in a fixed point solution: you may even choose the best way (to choose the best way)* to find the best move.

Many people try to sell you their silver bullets for software development just like a ninety-five percent non-smoking cure. Usually those methods are presented as professional sounding TLAs, lulling you into a mood of dedicated devotion. I try to sell one single piece of the great software equation today.

Adhere to the touched piece rule.

Which basically means: Think before you write code. This sounds obvious, doesn't it? You'll always think about whether the code you just rinse from your brain is correct in exactly the same way that a child learning to play chess asks herself if the move she wants to make is according to the rules, or if there are any current threats lurking from the other side of the board.

Correctly working code is not why you made the move. It's why you moved at all, why your fingers hammer on your keyboard. So the admittedly recursive question is: what is the right question to ask?

Ask yourself for every line of code why you write it exactly that way in this very position.

Just like some moves in a chess game that don't look too bad by themselves can draw a devilish grin in your opponents face, you can easily make the god of entropy smile smugly upon you with a few lines of correctly working code that are a little out of place. Introduce some dependencies here and there and let the butterfly effect do the rest.

This strong dependency on the initial condition in a highly dynamical process like software development gave birth to a methodology of heavy up-front architecture and design. So what's the problem if we can solve our architecture and design problems by thinking things through in the beginning? It's recursivity (again).

Design is code and code is design. If you design up-front you still have to watch every step. This design will not only influence all the design you do from this point on until the project dies or you retire, but it will also influence all the code. And because at the beginning of the project you don't have any experience with the problem domain you'll make a lot of bad moves.

This all culminates in a vicious circle of inexperience and bad design smells. This is the curse of software development: you'll always find a better way to solve the problem after you've solved it. Agile development practices counter this phenomenon by using small iterative cycles in development. Specify what you want - write a solution - refactor using the experience you just gained - redo from start.

This is often criticized as trial and error programming and taken as an excuse not to think about how you write software. But it is the opposite. It's about thinking how to write software before you write it, evaluating what you did after you made the move and then adjusting the code according to what you just learned.

A fixed point solution to a recursive problem.

Zweitgeist - Meet Anywhere On The Web

A myriad of new virtual world communities besieges us to join the holy ranks of virtual knights or just lead a shiny new second life if you don't want to cope with the onerous exercise that reality imposes on us humble human beings any more.

Today I encountered a refreshingly new concept: Zweitgeist. The name of the service is a somewhat weird combination of words, literally meaning "second ghost", and at the same time a corny word transformation joke that is derived from the word "Zeitgeist", which seems to be a normal English word after all.

Zweitgeist is basically nothing more that the inverse function of Swarm which empowers the seeking web wanderer to find her hopefully like-minded information freak. Zeitgeist now utilizes the good old web as well established basis for it's virtual world - the web is the world, and you can meet people on web pages just like you would meet them at the bus stop or in your favorite movie theater. The virtual world experience is just another plug-in to your favorite web-browser that displays some cute little avatar pictures at the bottom of the pages you're visiting at the moment.

Normally all this would be filtered out by my "yet another noble business idea" low-pass pop-up-pop-down business filter. But if enough people manage to ignore the penetrating skirl of their George Orwell alarm this may become an interesting plug-in to your typical web experience. While I'd really like to know how they plan to scale up for a web site like google.com (a thousand little avatar pictures at the bottom of my web browser doesn't seem too appealing) perhaps meeting a few people on Joel On Software that share my obscure interest in good software might be a juicy change.

In the end the most interesting thing about this service will probably be if they manage to beat Metcalfe's Law, regardless of whether the specifics are right or wrong when you happily apply the law to Web 2.0 (I just couldn't resist) personality stripper networks.

Review: Windows Vista - Unix Concepts Improved

While I didn't get a laptop I'm happy to share my experience with using Windows Vista for two months as main development platform at work.

The first feature that I really like about Vista is:

I can finally work as non-root user!

This is a feature that the Unix world knows for a few decades. So why did it take Microsoft until 2006 to make it possible to use a normal user account in Windows? Probably it's backward compatibility. They finally solved it in Windows Vista with a very tricky solution: if a legacy application writes to a system directory Vista transparently redirects the system calls to a file that belongs to the current user. This is one nice example for the old saying: try to get it right the first time!

The next change I noticed was the integrated desktop search. You can press the Windows button on your keyboard and the search line is focused (Microsoft rediscovers the shell). Now when you enter the first few letters of a word the menu gets dynamically updated and shows only programs (and emails) that match your search. While this feature is well known from KDE (alt-F2) and is probably available for Windows XP as an extension program, I really like this feature and use it more and more to search my emails and start programs.

The desktop search is not restricted to the search bar. The Windows Explorer now features a nice little search input line on the upper right side (just like a web browser) which you can use to perform a quick full text search on your files.

Now let me count. I discovered two (2) new features in Windows Vista that I really use. But I have to admit that those two features are exactly the concepts I missed in Windows for a long time now (since I knew them from Unix) and I don't regret switching to Vista as work environment (yet).