I finally discovered the truth about why developers rather rewrite a 1MLOC project from scratch than trying to understand a fellow programmer’s code:

We’re all code narcissists!

And the reason for that can easily deducted in a tiny logical chain:
(The best code is easy to understand) ^ (I can understand my own code the easiest, duh!)
=> (My own code is the best code)

Unfortunately this is only true for me, or perfect clones of myself. Which rules out everybody else I work with. Which reminds me… What was the reason that programming is done in teams?

I acted just like I did back in school. Or worse. I waited for the last possible moment before I started preparing the talk. When I finally started with the slides, a little perfectionist devil Manuel sat down leisurely on my left shoulder, telling me that this crap is just not up to my own standards. I worked long hours the night before the big day, and woke up early just to be able to rehearse the whole play before entering the stage.

Of course everything worked out just fine. Well, besides me saying basic-a-lly all the time. So why was I so freaked out? Well, I was obviously leaving my comfort zone.

Now the interesting thing beneath all that personal drama I’m ranting about is that I suddenly realized for how long I did not really step out of my safe little comfort zone. Granted, applying at Google is not the most un-stressful experience I ever had. And obviously starting a new job with all those bright people around me did not exactly make me fell warm and cozy.

But at that very moment I stood there, my peers gazing absent-mindedly into their laptops, my hands slightly sweating, I suddenly realized that ever since I started working I was doing everything exactly in the way I was most comfortable with. And that was kind of a shock.

Of course after giving the talk I felt great. I had known that in advance, but I realized that without being nudged enough I’d probably have tried to wiggle out somehow.

Lessons learned:

1. It’s easier to step out of the comfort zone when somebody kicks your ass.
2. It’s a long way from leaving your comfort zone once to real change.
3. I must learn how to leave my comfort zone on my own.

Do you know tricks that make it easier to leave the comfort zone?

Now after I reinstalled ubuntu and upgraded to gutsy which comes right now with a 2.6.22er kernel I tried to build the driver from source. I had to patch the sources to make it work, but since it works flawlessly right now I provide my patch and a little compilation howto.

Download the current NDAS sources and my NDAS patch for linux kernel 2.6.22.

# installed some packages. I don't know which exactly, but you'll need
# at least the following:
apt-get install build-essential checkinstall linux-headers-generic

# extract and patch the ndas sources...
tar xvzf /path/to/ndas-1.1-2.tar.gz
cd ndas-1.1-2
patch -p1 < /path/to/ndas-1.1-2_kernel-2.22.patch

# you only need to set NDAS_KERNEL_VERSION if you
# don't want to compile ndas for the currently running
# kernel for example, if you're compling from within colinux
NDAS_KERNEL_VERSION=2.6.22-6-generic
make

# ndas_root must be exported for make install and
# checkinstall to work
export ndas_root=$(pwd)
# somehow I had to make install before checkinstall...
# this is no problem, since checkinstall will clean up
# the whole mess again
sudo make install
sudo checkinstall

After that you can start the NDAS service by issuing

/etc/init.d/ndas start

Configure your device by following the Ximeta NDAS driver documentation.

My goal is to set up a colinux configuration for truely mobile computing. That means that I use wireless networking in Windows and Kubuntu natively as my primary networking interface. I want colinux networking to work whenever my Windows has network access and I want a single configuration on linux and Windows. The configuration you find here is based on the two years I use colinux now and constant fidling with the configuration parameters.

As a prerequisite you need a running GNU/Linux on your computer. I installed an Kubuntu Feisty onto a spare partition. Kubuntu is my favorite distribution at the moment, because you get up-to-date basic packages like in debian/sid, and up-to-date KDE packages like SuSE. After installing GNU/Linux I rebooted into Vista. Let’s rumble.

First I installed colinux 0.8.0 pre from Henry Nestler. My experience with colinux is that the latest unstable you can get at Henry’s site is the most stable version available.

Once you installed it (you can safely skip downloading the linux image, we’ll boot from a real partition instead) head to C:\Programme\coLinux and copy example.conf to colinux.conf. Now edit colinux.conf with WordPad. Since I have activated UAC I can’t simply edit the system file but use an administrator notepad. Start – Type “Notepad” – Right Click the appearing notepad and select “Run as administrator…”.

Now my colinux configuration looks like this:

kernel=vmlinux

# my swap partition
sda3=\\Device\\Harddisk0\\Partition3

# my root partition
# I usually have to play around with the partition numbers before
# I get the final digit right.
sda4=\\Device\\Harddisk0\\Partition4

# this should point to the root partition...
root=/dev/sda4
ro
initrd=initrd.gz
mem=768

# a slirp device for internet access; when I boot
# kubuntu natively, this is my wired ethernet device
eth0=slirp

# when I boot kubuntu natively this is my wireless
# connection; since I don't want any configuration
# changes from native to colinux I inserted a
# unusable eth1 device in the colinux configuration
eth1=pcap-bridge,"Unknown"

# internal high-speed connection between colinux
# and vista only
eth2=tuntap

# I want to see my Vista files and cdrom in colinux...
cofs0=c:
cofs1=d:
cofs2=e:

Now I can start colinux. Configuring the network devices takes some time, since the stuff is not yet configured on the linux side.

Edit /etc/network/interfaces. Remove the default entry for eth1 (it will be handled by knetworkmanager later, we don’t want to use it in colinux) and change eth2 to a private network:

auto eth2
iface eth2 inet static
address 192.168.42.2
netmask 255.255.255.0

Now configure the tuntap device on Windows to ip 192.168.42.1. Start colinux and you should be able to ping colinux at 192.168.42.2 from Windows. You can’t ping back since the Windows Vista Firewall is active on the tuntap device.

Install Xming and cygwin. Make sure you select openssh when installing cygwin, you’ll need it later to log into your colinux.

Run XLaunch (from Xming) and configure an X Server without access control and save the configuration to your local documents directory. To start the X server without access control at login time link the config.xlaunch file into your Autostart menu. Make sure that you allow your X through the firewall. Don’t do this if you don’t have an extra firewall to the internet, since otherwise people from outside would be able to contact your X server!

Now you have to make sure you can log into your colinux from cygwin without a password. Luckily ssh features public-key based authentication, and it’s not that hard to set up.

Start a cygwin shell.

ssh-keygen -t dsa

Copy the file to the .ssh directory on the colinux server. Make sure you have a .ssh directory in your $HOME before doing this:

scp .ssh/id_dsa.pub manuel@192.168.42.2:~/.ssh/authorized_keys

Now we’re able to ssh manuel@192.168.42.2 without using a password. Edit a file konsole.bat on your desktop.

c:
cd \\cygwin\\bin
run bash --login -c 'ssh manuel@192.168.42.2 "export DISPLAY=192.168.42.1:0; konsole > /dev/null 2>&1"'

Now all that is left to do is to run colinux as a service. At an adminstrator command prompt (Start->Run “cmd”, Right-Click->Run As Administrator) c:\Programme\coLinux write:

colinux-daemon.exe @colinux.conf --install-service colinux

Open “Services” as administrator and edit the properties of the colinux service. Set the start mode to automatic. Voila.

To access your windows partitions from colinux you can simply add them to your /etc/fstab:

0 /media/c cofs user,defaults,rw 0 0
1 /media/d cofs user,defaults,rw 0 0
2 /media/e cofs user,defaults,rw 0 0

Now your init-scripts will mount your cofs devices (a.k.a. Windows Drives) when colinux boots.

I use this configuration on a Dell XPS M170 laptop. Of course you can use it for workstations, too. I myself prefer a winpcap’ed configuration for workstations, though. It’s easier to get access to the colinux that way. Unfortunately winpcapping the device doesn’t work on most wireless networks in the wild.

See also:

Information Overflow & Colinux WLAN Networking

Colinux On Windows Vista

Now that I’m back home I get lost in the information that I missed in just two weeks. I’ve got some IEEE magazines and journals, some GI resources and the extreme programming group which clogs my Gmail account – and I don’t want to miss any of it.

This week the extreme programming group features some nice discussions about “good code”, the IEEE software engineering journal features an article about pair programming and an other interesting story on theory usage in software development. The IEEE spectrum contains an article on a guy who implanted two RFID chips in his hands.

After spending some hours reading today I caught up on extreme programming and finally realized that my colinux networking still totally sucks.

I finally found out how to use colinux with my Intel 2200BG adapter: It’s simply a matter of setting the adapter to non-promiscuous mode and add the adapters MAC address to the configuration:

eth1=pcap-bridge,"Connection Name",,nopromisc

Now I can set an ip address in linux and happily access the network. Unfortunately this is not a very mobile setup for a laptop. If you are at home in varying wireless and wired networks you usually just want to use dhcp and use whatever ips the network provides.

If I use dhcp in linux and windows on a bridged wireless adapter, both windows and linux will get the same ip (since I must specify the same MAC address – otherwise the WLAN router won’t accept my packages). This leads to an unusable network (again). Of course I could just specify a fixed ip but then again I have to change configuration when roaming wireless networks.

The next requirement is that I want to be able to have some linux services accessible via my broadband router when I’m at home. Therefore I need to know an ip to configure some port forwardings. And I want to be able to use the same configuration regardless if I boot into windows and start colinux or if I boot into linux directly.

Now I came up with a configuration that allows me to do all this: I use slirp for external networking and a tuntap device for windows to linux networking (tuntap is faster than slirp). I configured some port forwarding into slirp and forward the required ports from my broadband router to the ip of my laptop’s wireless network device.

Then I configured my kubuntu desktop to use network manager (basically by throwing everything away in /etc/network/interfaces) and to use the same MAC address for the wired and wireless interfaces. This way I can really plug and play without loosing any port forwardings from my broadband router. Neat. This whole configuration is quite complicated, so I should probably post a blog entry about the configuration before I forget it.

For all of you celebrating: Happy Easter!

“With hardware, theoretically anything is possible.” – This statement emphasizes the limits of hardware. Theoretically you can do whatever you want, but you’re limited by the material the earth provides. I admit that human beings are highly innovative – memory chip designers happily use techniques today that were believed to be physically impossible not long ago.

“With software, practically anything is possible.” – Software has no inherent limitations. The concept “software” consists of a bunch of algorithms neatly packed together into a structured specification of how to solve a problem. Of course, software is limited by the hardware it is run on, which is why I usually have a problematic relationship with hardware which sometimes leads to mutual physical violence.

There is a point in the life of a software developer when you realize this (the limitlessness of software, not my problem with hardware). That realization marks your promotion to Programmer Level 1. At that time you usually know a bunch of programming languages, acquired some basic algorithmic competence and finished some minor projects. You leave the hunting grounds of not knowing how to implement things and ascend to the heavenly plains of being capable of anything.

You’re Turing Complete.

It’s tempting to stay at this level. It’s a great feeling – you know that you can do whatever task comes your way. You are a professional. All you need is somebody to tell you what she needs and you’ll develop the solution. Some of them will need some creativity with regards to runtime performance, but hey, that’s just another technical challenge. There’s nothing more you need to learn besides exciting new technological tools.

It’s even more tempting to completely ignore level 2. Some people even vehemently deny it’s existence. Acknowledging level 2 is the first step on your long journey to the highest programming nirvana. You’ll reach level 2 once you can write maintainable software. If you ask “now, buddy, what is so hard about that?” then you’re a long way away from mastering level 1.

The principle behind maintainable software is easy:

Write software that your fellow programmers can understand.

The easier it is for others to understand your code, the better your code. So how can you write understandable code? This is the hard one. It’s not easy. There are no “top ten guidelines” you follow and somehow become a good programmer over night. There’s no “30 minutes Howto for Dummies”. It means hard work. It means lifelong learning. And it means to have a certain attitude towards coding:

• Be a teacher. Write code that teaches the reader what the program does. Don’t be the Riddler. Communicate your intent clearly.
• Be a student. Watch how people react to your code. Be open to ideas and criticism. Learn from your fellow coder.
• Be pragmatic. Don’t do stuff “just because”. Always know an answer to “why”. Introduce convenience to get things done.
• Be a methodologist. Be fierce when sloppiness threatens the clarity of your design and code. Don’t give in to convenience if this makes it harder to read your code.

Reaching Programmer Level 2 is an ideal, an illusion. When you realize that being able to code anything is not enough you somehow enter Level 2 metaphysically, knowing that you’ll never reach it for good. But if you want to enter the life long journey and defy the perils of the Code Monster, here’s a hint on how to start:

• Read The Pragmatic Programmer. This will give you the initial force you’ll need to keep on going.
• Read some more: Code Complete is one of the most acknowledged writings on the subject.
• Never stop learning. You can take a look at my Library, or just search for programming books at Amazon.

The only known effect of BPCD is that the infected person writes exceptionally clean, well-structured, bug-free and easy to maintain code. The benign character of the illness is the root of a devilish new pest called WBPCD (Wannabe Best Possible Coder Disease). Curing WBPCD is hard, since the infected person usually believes to suffer from BPCD and stubbornly refuses medication.

Since spreading BPCD is supposed to have positive effects in corporate environments, you can find quite a lot of studies about effectively spreading BPCD. Mostly getting infected by BPCD involves close collaboration with people you believe to suffer from BPCD, or mind-to-mind intercourse which is easy to practice by reading.

Unfortunately you can never diagnose BPCD for yourself, since if you believe you have BPCD you’re just a WBPCD case and should consider professional help. If you don’t believe you have BPCD – well, it could be that you’re right, but since you don’t believe it this won’t help you. The only known solution to this problem is to try to get BPCD infected for the rest of your life, always hoping to be infected, but always doubting your current condition.

That said, it’s not too hard to diagnose BPCD if you know the symptoms. If you suffer from WBPCD you may subconsciously use this information to make others believe you’re a BPCD patient, so watch out! Don’t fool yourself.

Symptoms of BPCD:

• Unbreakable optimism – BPCD patients usually start getting infected with Morbus Codus (Coding Disease). They recognize quickly that it’s impossible to write bug-free code. This is the turning point in the course of disease. Some people just stop there, running away to business or law schools. The other lot is optimistic enough to hope that this time everything will be all right. While WBPCD patients just assume that their code is top notch (they believe to be BPCD infected), real BPCD people realize that optimism must be controlled with caution and test their assumptions thoroughly.
• Easily bored and clumsy – Doing repetitive tasks will bore a person suffering from BPCD to death. If the person is also clumsy this results in a high error rate. While WBPCD patients just ignore those errors and think they will manage to do everything correctly the next time, you will find a high level of automation in the BPCD infected work environment.
• A strange kind of perfectionism – While WBPCD causes the infected person to believe to be already perfect and requesting perfectionism from other people, BPCD makes you strive for perfectionism yourself. The typical BPCD patient is always in a mentally driven state, trying to learn and improve and help others get BPCD infected as well.
• Annoying curiosity – Living or working with a BPCD infected person can be a challenging experience. Always seeking for answers the BPCD patient never stops questioning every possible process until he either recognizes it’s worth or starts trying to make you change. People who do not want to change should try to avoid contact with BPCD infected persons to prevent physical damage.
• Bad memory – An unimaginably bad memory can help to catch BPCD. Some patients can’t remember what or why they did whatever they did last week, or if NMBR is shorthand for “number” or “Non Memorable Abbreviation”. Thus they find themselves reading their own code with the eyes of a stranger. This makes BPCD infected programmers write code in a way that people who know close to nothing about the project can easily read their code. For WBPCD infected persons this just means being late to meetings.

If you want to get BPCD infected you can read this nice article about becoming a better developer at willcode4beer’s.

The article finds that XP is a cult. Well, I totally agree. But than again – so what? All great movements that come to my mind started as small cults. For example <insert your preferred transcendent association>. I don’t really know about anything but Christianity, but I believe that other religious leaders had a hard time being accused of cult leadership, too, when they started their business. Now of course I counter polemics with polemics – but hey, I’ll flagellate myself later with my cult-colored whip.

Speaking about movements, don’t forget emancipation, civil rights or the Internet. In the beginning it’s always a few nerds with some crazy idea who build up a cult. And in the beginning nobody can say if it’s going to be the next big movement or just an other flash in the pan that didn’t beat Metcalfe’s Law.

The article doesn’t have a single argument against any XP practice. Funny, eh? The value of XP is judged purely by the author’s perception of XP in the media. Can you tell if an apple is sweet without taking a good bite?

Apart from tagging XP a cult, which is true, the article raises some claims that are – let me put it bluntly: wrong. For those of you who favor polemics over reasoning, stop reading here.

Elitist Attitude
Cited from Extreme Deprogramming.
This is definitely a group of people who think they have got it, and that anyone else not similarly enthused is a laggard.
That knowledge of what is and isn’t OK is seen to be held by a central authority and is not in the hands of the practitioners themselves [...]

Those citations give the impression that XP’s great ol’ ones are arrogant know-it-alls, who just want to make a lot of money. OK, they probably want to make a lot of money, but if you get involved with the extreme programming yahoo group, you’ll see that they answer your questions patiently. Especially Ron gives away a lot of consulting time for free (or he is a quick writer). The community behind XP is caring, open-minded and welcoming, always reflecting upon their own flaws.

All or nothing
This all-or-nothing thinking is typical of cults. Members must display total dedication to the cult and its objectives, or they are labeled impure and expelled from the community.

I don’t know where this insight comes from, but when I read the yahoo group for XP many people advice you to try to implement a practice only if you think it addresses a problem you have.

Most of the author’s knowledge about XP seems to stem from the newsgroup comp.software.extreme-programming. I can only do some guesswork: Mayhap there are many agile zealots who don’t have a clue of what XP is about and they manage to build up some weird image in open newsgroups. I can only recommend to take a look at the archives over at the XP group and lean more about the community.

Conclusion
The long term success of XP will probably depend a lot on whether it really works. To get the answer you can try it out for yourself. Apart from that only time can tell if XP will be a movement that changes the way we implement software systems or yet another hype.

Last week I gave ruby a new chance to enter my programming tools portfolio. I’m developing embedded C++ for a living and usually automate coupling of code and data by using scripts. Now the task was to write a script to parse some enums in header files which use a predefined layout for comments, parse the whole code base to check whether those codes are used consistently (> 100.000 LOC) and generate and check some XML files that define default configurations for the software product.

Most of the scripts we used in the past are written in perl, since python just lacks the convenience. But since I always have a problem building up data structures in perl and knew that this script required some heavy data structure building I wanted to give ruby a try. So I opened some ruby introductions and documentations in my browser and watched myself learning ruby.

I was impressed. The documentation is very good and I was able to write productive code right from the start. One interesting thing is that while I learned more about ruby the code I had already written didn’t seem awkward. With ruby I was able to write nice and clean code right from the start. This is probably what Yukihiro meant when he said he tried to make ruby “natural”.

Ruby adapted to my OO style beautifully. Writing new objects and building up data structures works like the language was tailored to my needs. The convenient regular expression handling methods (which include the good ol’ $1 .. $n variables we know from perl) make string parsing as easy as in perl. And while I don’t have any data available, the perceived performance is quite impressive – parsing our complete code base with multiple regular expressions just takes a couple of seconds. Ruby even provides a perl-compatible command line argument set for in-line editing that can be controlled via built-in variables (see the Ruby Language Reference Manual).

After working with ruby for some hours, I find it very attractive that you automatically stick to a functional programming style for most of the time while introducing imperative data manipulating statements where necessary. This makes programming a far less error prone experience.

But the ruby also shows some less shiny sides. During the first hours of using ruby I discovered some interface anomalies that are not made explicit. For example it took me some time to figure out the difference between the foreach and the each function. Now that I understood that foreach is parametrized by an additional layer of abstraction objects I really like the concept, but still the naming got me confused at first – for me for each is executed for each element, so what’s the difference?

Talking about interfaces: I used ruby libxml for xml parsing and generation. Everything worked fine in linux, but I was not able to install ruby-libxml via gem on windows without compiling the code by myself. So I looked for a different xml library that is available in windows, too. I found REXML, but REXML and ruby-libxml are not interface compatible. In a statically typed language the compiler would have taken care of this, but in ruby I don’t get errors until late at runtime. It’s especially hard in such a case when the interfaces contain methods with the same names that do different things. But than again perhaps such a high level of interface coherence is only possible with integrated commercial libraries like Java and .Net.

All in all ruby has blow my mind and I’ll use it as primary scripting language from this day on. If you are looking for a clean object oriented dynamically typed language with a nice functional touch give ruby a try.

Then I took a look at extremeprogramming.org. I poked around for a while, read about Pair Programming, Test Driven Development and Customer Involvement and concluded: This just can’t work.

How can doing more work make you more productive?

This is the way many people react to the XP ideas when they first hear about it. About a year ago, just after I read the Pragmatic Programmer, I decided that the time was ripe to find out what XP was all about. So I ordered me a copy of Extreme Programming Explained. With two years of real experience as a software developer XP made a lot more sense. But still I was not convinced. I didn’t see the big picture yet.

Over the last year I started to read some “classic” software development literature, like the Mythical Man Month or Code Complete. While studying software processes I realized that the ideas of XP were not new. In fact, some of them were 30 year old knowledge distilled into a set of best practices.

It took me about a year of studying and work experience to grasp the essence of XP.

Why is XP counterintuitive at a first glance? The answer to this is easy: Nearly every concept of XP looks like it makes you less productive. Only once you understand the concept of the cost of errors (see this article by Steve McConnel and his book Code Complete) you’ll recognize why XP pays off. Let’s consider some of the XP practices in detail.

Pair Programming: How can it pay off in terms of ROI to have two programmers work at the same problem at the same time? Don’t we live in the age of specialization, where working on highly specialized tasks in parallel guarantees maximum output? The problem is the cost per error.

Software development is a complex process where errors are easy to make and hard to catch. High level languages and even MDA don’t reduce system complexity, they only help ordering our mind. So one of the maxims of software development is to get quality into the system when the code is written. If errors are found while pondering over the problem, they’re fixable within a fraction of a second. So pondering with a programming peer helps to build quality into the system, to reduce waste when a problem is found that the peer can easily solve and to keep focused but not zoned out while writing code.

As of today you can find some studies about Pair Programming, which are unfortunately mostly built upon a University experiments, so the validity of the results are questionable. That aside, all studies I found so far claimed increased productivity by the pairs.

Test Driven Development: It took me quite some time to buy into this one. Kent Beck admits in Extreme Programming Explained that you’ll write as much LOC for tests as for the real system. So there’s the magic factor of two again. Just like with Pair Programming, everybody agrees enthusiastically that implementing unit tests is good, but it’s hard to believe that writing as many LOC for unit tests as for the target will improve productivity.

This is one of the concepts where you have to “be there” before you’ll really understand. The value in unit tests is not only that you have a nice regression test suite during development that helps you refactor your system without breaking old features, but it helps you to understand your own code. This is magic. Try it. Feel it. And you will believe.

Continuous Integration: Of course everybody wants to do continuous integration. But is it really possible? This is where XP’s practices interconnect: Without the high level of quality that Pair Programming and TDD yield, it’s hard to maintain a stable development version. And once you have the required high quality processes in place, continuous integration is for free and feeds back into your high quality loop.

Collective Code Ownership: Some things about software development are purely in the head of the developers. But even without owning a degree in experimental psychology it’s easy to see that Pair Programming is hard to implement if code “belongs” to individuals. If everybody is accountable for “his” achievements, Pair Programming feels like giving away time to your team mates. So much for being a team player.

Story Cards: Yeah, so using a different kind of medium will solve all my problems, right? First, no. No single practice of XP claims to solve all your problems. And second, take a look at this article by Ron Jeffries. In combination with fixed iteration cycles, story cards can give you valuable feedback about how fast you’re going (warning: this may be a shocking experience).

Fixed iteration cycles: In my view this is one of the practices that is coupled the most to the other XP principles. To implement fixed iteration cycles successfully you need continuous integration – otherwise you’ll spend your time managing myriads of branches. As stated above, this implies high quality code, which is all but impossible without automating your tests and constantly reviewing your code. And you’ll get most out of your fixed iterations if you can measure your velocity (the sum of completed story points) at the end of each iteration.

Customer Involvement: Isn’t it impossible to have a customer on-site if you produce for a range of users? The first thing that is easily misinterpreted about customer involvement is the very concept itself. Customer involvement doesn’t necessarily mean to place a real customer next to your developers. It just means that a person that knows the user (which may very well be your in-house product management) prioritizes the user stories. This way you maximize the ROI in every iteration.

-

At a first glance most of the XP principles are counterintuitive. They seem to cost a lot of programmer time while lacking measurable return of investment. The only way to see through the top layer and to understand the inner workings of XP is to learn more about XP or agile processes in general. If you want to understand XP better or just start a flame about software development processes I recommend the Extreme Programming Yahoo Group. Over there Ron Jeffries, Kent Beck et al. patiently answer your questions (in Ron’s case mostly with more questions).