Sunday, January 07, 2007

Some history on the software development process model

People always have different ideas or mentalities towards software development process model. Veteran folks ask “I remember software development used to be as easy as 1-2-3, and just follow the steps things eventually get done one way or the other. We never had to worry about iterations, methodologies, or the process, so what good is this iterative/agile approach?”, and on the other hand greener folks would ask “Why aren't we following a iterative approach, and by the way what is this waterfall model you guys are talking about?”. More than 10 years after the iterative approach was introduced to the industry, you will still be amazed that just how many of the practitioners, including both developers and managers, can not tell the difference between the spiral model and the iterative model, and also just the number of the organizations that still following the decades old waterfall model or even worse still in the darkage. That is why I am writing this short article on the software development process model history hoping to help explain two questions: Why are we here? and How did we get here?

Waterfall Model


Winston W. Royce proposed the initial waterfall model concept in 1970 recognizing the cost of discovering defects late in the development cycle, adopted a logical stepwise process, including steps from Requirements -> Design -> Coding and unit test -> System Integration -> Operation and Maintenance, progress through this set of fixed steps to produce software system. Comparably the waterfall model is considerably better than its predecessor, the two-phase “code and fix” model, where by developer directly jumps into coding right away and then keeps fixing defects until it couldn't be fixed anymore. The water fall model is widely adopted in the industry during 70s-80s including some most influential players in the field employed by NASA and US Department of Defense.


The most criticized problem of the waterfall model is ironically also the single fact that contributes to its success - the fixation of the steps. The waterfall model promotes fixed steps as well as barriers created between these steps, for example in a pure waterfall model the requirements are “frozen” after the requirement definition stage throughout the entire life-cycle. Although this practice sounds logical from the software engineer's point-of-view, it is very counter-intuitive from the business perspective. In many cases with pure waterfall model, over time the development team becomes so disengaged from the real world requirement on which the project was originally based on due to the nature of the ever changing world of business. Contrary to widely accepted belief, waterfall model is exceptionally ineffective when used to implement large project. Because of the broad scope of the requirement in a large project, it is extremely difficult to get all the requirement captured and defined correctly in one shot without venturing into the construction stage, but because lack of feedback loops in the waterfall model often the development teams are forced to march through construction with an ill-defined requirement which in result inevitably leads to nothing deliverable and chaos, or even sometimes the team manages to deliver something but it is not what the customer wanted.

Spiral Model


In Barry Boehm's article A Spiral Model of Software Development and Enhancement published in 1988, he recommended a different approach for guiding the software development process. In his spiral model, Boehm acknowledged the weakness in the waterfall model when applied to large, complex, and high-risk project the upfront requirement analysis stage is usually too superficial, without getting into design and implementation details, to fully define the requirements which becomes a common pitfall for many waterfall driven projects. Thus Boehm recommended a more risk-driven and incremental development method in which after requirement definition and concept validation stage one or more prototypes are built to assist in early confirmation of the understanding of the requirement of the system as well as mitigate any major technical risks followed by a structured water-fall like process to produce the software system. The major advantage of this process model is it provides multiple feedback opportunities early on to verify the requirement and mitigate risks before entering the construction phase so it is more likely to produce the final product on time as well as to produce what the customer truly wants. The spiral model became the role model of software development process for the whole 90's, I still remember in my text book on Software Engineering when I took the course in 97 it was still considered the state of art process model.


Criticisms to this rigorous approach are mostly focused at its inability to perform in a more dynamic and competitive environment due to two main weakness: 1) It is very expensive and time-consuming making two or three non-deliverable prototypes to just confirm the requirement and mitigate risk 2) It lacks of effective mechanism to handle requirement change after it enters the construction phase. Despite the criticisms, spiral model does provide us a valid but expensive way to systematically develop a system at any size therefore even today it is still a very popular choice for large complex but rigid projects at US Army(the famous Future Combat System is a good example for the spiral model) and NASA.

Iterative Model


In 1995 Philippe Kruchten described a new approach, one that combines the best of two worlds from waterfall model as well as spiral model, called Iterative approach. In the traditional software development model as time progress the development process moves forward through a series of sequential steps, but in iterative model the software itself is broken down into smaller pieces and one or more of these pieces gets developed in each iteration. The whole project will typically go through multiple iterations till the whole system is implemented, and every each one of these iterations is like a mini-waterfall life-cycle has multiple phases. In addition, each iterations is like a spiral model designed to mitigate risks for that specific iteration. More recently accompanied by the advancement in engineering practice, such as refactory and automation, some developers have pushed iterative model to its extreme with shorten iterations as well as simplified life-cycle model to harvest more productivity out of the iterative model for certain kind of projects, and this highly optimized iterative model is often referred as Agile process model.


The main advantages that iterative model brings into the industry is basically allowing us to revisit various activities, such as requirement definition and design, multiple times during different iterations in a project, hence allowing us to correct and refine them along the way to deliver what the customers really want in a more timely and predictable fashion. Additional to this, the iterative model also enable us to design and build something even without fully understanding the requirement since in each iteration we only work on the parts that we have already understood, and furthermore, not like the spiral model, in the iterative model every iteration is a complete mini-lifecycle and produces a incremental but executable product vs. a mere prototype in the spiral model. A true incremental delivery ability which is the key in today's highly dynamic and competitive world.

No comments: