Sunday, July 1, 2007

Understanding The Fuzz About Engineering In Software Development

Steve McConnell responds to Eric Wise's article Rejecting Software Engineering that Rumors of Software Engineering's Death are Greatly Exaggerated". There's a lot of fuzz about the usage of the word "engineering" when it comes to software development. What's this all about?

What is engineering?


According to wikipedia, engineering is defined by the ECPD as

The creative application of scientific principles to design or develop structures, machines, apparatus, or manufacturing processes, or works utilizing them singly or in combination; or to construct or operate the same with full cognizance of their design; or to forecast their behavior under specific operating conditions; all as respects an intended function, economics of operation and safety to life and property.


When I look at this definition I don't really see anything that would not be applicable to software development. Steve McConnell generously points out that software engineering is recognized and practiced for some time now, so where does this new-fashioned stubborn refusal to call the child by it's name come from?

The engineering process


The real problem comes to light when you look at the engineering process. You'll find a description of the engineering process on wikipedia. The article describes the engineering process in four stages:

  • Conceive

  • Design

  • Realize

  • Service


Now there are some engineers who map the engineering stages to software development in a rather funny way: They think that "Design" is the process of drawing good looking UML diagrams and that "Realize" is "Coding". When you look at the wikipedia article, you'll see that for engineers "Realize" stands for "Manufacturing". In a software context, manufacturing means running the compiler and pressing some CDs or deploying some binary over the Internet.

So when engineers claim that software developers should look at how engineers do their design and all this talk about software processes would be settled once and for all, they're ignoring that software development is a design-only activity and that software has a lot less problems with the "Realization" stage than traditional engineering.

When a software developer writes code she is building an executable, mathematical model of reality.

Conclusion


When software developers prefer not to use the title "engineer" to describe what they're doing, they're trying to avoid a mapping of the engineering process onto the software development process that is wrong. In the end, software developers will build mathematical models (source code) and apply scientific methods (complexity analysis) to solve problems. If this is not engineering, than we're not engineers.

14 comments:

  1. To "Realize" is not just "Manufacturing" - for a one-off, it is more like "Build". There is a huge variety in how things get built across various engineering disciplines.

    I take issue with the sort of software development where there is not measurement, analysis, and evaluation of fitness - just a sort of "cut it and see methodology". Maybe it is because a quick edit/recompile/run is so easy to do, this sort of develpment method seems easier to fall into for software projects. This sort of development is not engineering.

    I have seen this sort of ad-hoc development method employed for hardware projects as well, and there it also leads to schedule/cost overruns, non-scalable performance, and unreliability.

    That is not (good) engineering.

    ReplyDelete
  2. Ivan,

    thanks for your feedback.

    What sort of measurement do you want to install?
    Something like a productivity measure? Do you want to measure code structure? What do you want to do with what you measure?

    Cheers,
    Manuel

    ReplyDelete
  3. No, I don't mean a productivity measurement or code structure. I think these measures mistake the trees for the forest. I mean measurement and analysis of whether a section is working properly and how it is performing.

    Usually the reason you are doing engineering is that you have a problem to solve with certain constraints (material/economic/time). So your measurements should be used to verify that the solution is satisfying your constraints.

    ReplyDelete
  4. Ivan,

    so would you consider TDD to be a sort of measurement? Automated customer acceptance tests?

    What constraints would you want to measure exactly? All measurement that I know of with regards to code is structural, since semantic analysis is mostly out of scope of todays tools due to high computational needs.

    Cheers,
    Manuel

    ReplyDelete
  5. Well Unit Testing is very good. I think of it like the component level tests done on Integrated Circuits. You can have good confidence in the way your components will perform when they are thoroughly characterised.

    I am of two minds about TDD - sometimes it seems to mean specify some tests, then hack away until your code passe the tests. This does not seem very useful for producing a solution in a predictable amount of time.

    Used another way, it is very good. You have a set of tests, you have a design, you have an implementation plan, and you know your implementation is progressing correctly because it passes more and more of the tests.

    Automated customer acceptance tests - for sure, whatever can be automated should be.

    ReplyDelete
  6. TDD as I know and practice it is a one test at a time approach. The idea behind this process is that experience tells us that we learn about our design when the computer tells us whether our code works as expected or not.

    What do you mean by "hack away"? You say "you have a set of tests, you have a design". Does this imply that you think one should specify all unit tests, than do a higher-level design and then do the code? What if half-way through I find out that the design is clumsy / too flexible / not flexible enough?

    Cheers,
    Manuel

    ReplyDelete
  7. I suppose I see a rather clear distinction between implementation (coding) and design. I think this is an important feature of an engineering project.

    Of course, during the design you will certainly do some exploratory or prototype coding and simulation. But in my opinion the bulk of the implementation coding should wait until AFTER you are confident the design will satisfy the requirements.

    ReplyDelete
  8. So when I make a jig to hold some wood when cutting with a circular blade saw I'm engineering? Because that fits the bill of that definition.

    Also to call yourself an "engineer" in most states you need to pass a test, and usually be an apprentice / sponsored by another engineer.

    ReplyDelete
  9. "They think that "Design" is the process of drawing good looking UML diagrams and that "Realize" is "Coding". When you look at the wikipedia article, you'll see that for engineers "Realize" stands for "Manufacturing".

    For years I try to convince people that there is no production in software development, just design (R&D). Production is, as you wrote, the press of a button to compile the product or the pressing of CDs. When Ivan says building equals production, then only if someone follows a plan (as people building a bridge are), without any design decisions on their own. But with software development, developers not only follow a plan but decide on design issues and the domain when writing code (to fill the gap between the most detailed model they've got and the solution which is required).

    And when we're at that, there is no coding also. Just modeling. The developer has different models to a solution, which starts with a requirement model, a domain model, etc. After detailing every model to get to the next one, developers end with a very fine model describing the solution to the computer (something most people call source code).

    Great post,
    Peace
    -stephan

    --
    Stephan Schmidt :: stephan@reposita.org
    Reposita Open Source - Monitor your software development
    http://www.reposita.org
    Blog at http://stephan.reposita.org - No signal. No noise.

    ReplyDelete
  10. Ivan,

    I believe from what you write that you're an experienced and thoughtful developer. This is why I keep asking questions - I really want to learn the deep assumptions behind both our point of views.

    I see code as merely a form of design. A design that takes place on many abstraction levels. If I write an interrupt routine in C I can use the same intent revealing techniques as when I write a script in ruby or write a design document on a very high level.

    You say that you want to be "confident the design will satisfy the requirements". This is exactly the point where I think we differ - I believe that I can't be confident that the design satisfies my requirements unless I have tested the running code. This stems from my experience that the software I write is too complicated for me to understand, so I constantly introduce errors.

    I have a few questions w.r.t. your remark:

    On what level should the design be completed before coding?

    Why is the distinction of design and coding important to call it "engineering" - or how do you map those tasks upon traditional engineering tasks?

    How is code different from a circuit board layout?

    Looking forward to your reply,
    Manuel

    ReplyDelete
  11. Manuel, you said "I see code as merely a form of design. A design that takes place on many abstraction levels."

    I would say "I see a design as essentially independent of the code. The code is a concrete implementation."

    Manuel, you say "I believe that I can't be confident that the design satisfies my requirements unless I have tested the running code.".

    I would say slightly differently "I believe that I can't be confident that the _code_ is correct unless I have tested the running code."

    I can be confident that my design will satisfy the requirements when I have analysed it and simulated it enough. How much is enough? This is a matter of judgement - get it wrong and you will not be a profitable engineer.

    There is another feature required of the design, and that is that it must also be feasible. When I have a feasible design, and I am confident it satisfies the requirements, then I know that competent coders will be able to go from there and develop implementation code that will satisfy the requirements.

    So, at this point I can give the client a good cost/time estimate and I can run a good implementation project. Since the implementation phase can cost 10 x as much as the feasibility and design phases, in my experience the client likes some certainty in this phase. This is why the separation between design and implementation is important for good engineering.

    This _is_ all traditional engineering. Whether you are a civil engineer building an aqueduct out of stone, a hardware engineer building a microcontroller in silicon, an aerospace engineer building a rocket, or a software engineer building a system out of code.

    Engineering is the method, not the materials or the object being built.

    ReplyDelete
  12. Ivan,

    thanks for the insightful answer.

    How do you simulate your design? How do you know the design is correct without having running code? (which is essentially just one question :-)

    I would really like to learn how I can come up with a design for anything but a simple project that I wouldn't do totally different when I learned what the requirements really are and when I learned what I didn't think about in the first place...

    Cheers,
    Manuel

    ReplyDelete
  13. Thanks for sharing information. I think Software engineering is basically the part of software development process that include systematic, disciplined quantifiable approach, and the purpose of improving the reliability and maintainability of software systems.

    ReplyDelete
  14. Hey guys i want to share with you a way i make $500 every day and i only spend 15 minutes doing it a day! Anyone can do it, you dont NEED to have a website. I strongly suggest you check their site out as there is really a brilliant video that explains every thing you need to know. Check them out at [url=http://mobile-mastermind.com/]Mobile Monopoly[/url]. That's the name of the System and i recommend should you own a web site that you at-least go and take a peak, you wont regret it...

    ReplyDelete