Introduction

Ever more increasingly sustainable competitive advantage stems directly from innovative design and fast delivery of software solutions. Ever more increasingly business stagnation stems directly from constraints imposed by brittle, inflexible and unresponsive software. Never before have the consequences of project success or failure been greater. Yet despite this 71% of software development projects are destined to fail outright, overrun, go over budget, or fail to deliver the intended scope. 

For organisations intending to survive and prosper, it's time to take a good look at the operational performance of the IS function, with the clear objective of ensuring it provides a sustainable source of competitive advantage in the years to come.

The Five Sources of Competitive Advantage

At its heart the IT function is operational with the purpose of creating and delivering IT solutions to its customers. As such it can be measured against the five sources of operational competitive advantage:

*    Cost - deliver with fewer or cheaper resources

*    Quality  - meet customer needs right first time

*    Dependability - deliver on time

*    Speed - deliver quickly

*    Flexibility - deliver different things, or deliver them differently

These sources of competitive advantage do not stand-alone but are inter-related, contributing to each other either in a vicious or virtuous circle, thus increases in quality can also decrease cost and increase speed, with the reverse also being true.

How do our traditional software development processes match up to our sources of competitive advantage? The following sections take each one in turn.

Cost

IT systems are extremely expensive to develop largely due to the relatively high wages enjoyed by IT professionals. Faced with pressure to minimise costs, management therefore had two options, find ways of making more effective use of their current expensive resource pool, or, find cheaper sources of resource. With countries such as India, and more latterly the old Eastern Bloc states, offering a supply of highly educated but relatively low paid workers the second option, cheaper resource, was taken up by large numbers of organisations, and the rush to off-shore was on. However though the explicit cost of off-shoring is low, the solution has two major drawbacks. The first is that it re-enforces and amplifies the inefficiencies of the traditional waterfall development process.  The second is that it addresses cost in isolation of the other four sources of competitive advantage. In fact it promotes cost at their expense.    

Quality

The traditional measure of quality of software applications is the extent to which the completed system matches the requirements, specified in detail at the start of the project. Professional test specialists measure this after the software has been developed using test scripts sourced from the requirements specifications.

We can see that this approach is dependent on the specifications being correct in the first place and therefore a huge amount of effort is expended on reviewing the specifications and gaining sign-off from the user community. The problem is that it's extremely difficult, some would say impossible, to specify up front, exactly what is required, to the level of completeness, correctness and consistency, needed to develop a quality software solution. As a consequence we often find that quality, as measured by conformance to specification, is high, but quality as measured by whether the solution actually meets the real day-to-day business needs of the user community is often low.

Dependability

Studies tell us that a large percentage of projects over-run and that the average size of the over-run is 80% of the original schedule. This is hardly surprising when estimates of the development effort are carried out at the beginning of the project, when relatively little is known about the eventual solution, and when progress can only be effectively measured towards the end of the development phase, many months afterwards, and far too late to take any significant action to address the problems.

Dependability, or lack of it, is the real killer for the IT function. Breaking promises late in the day breaks trust and leads to adversarial relationships between the IT and business communities.

Speed

Traditional development processes attempt to transform on mass, one large block of requirements into a software solution via a series of sequential steps, each needing to be fully completed and formally accepted before proceeding to the next. There is no attempt to distinguish between high priority and low priority requirements, nor easy to deliver and difficult to deliver requirements. The process moves at the pace of the slowest.

Attempts to speed the process come down to the amount of resources than can be thrown at it, either through increasing the number of people working on the project, or increasing the hours worked by those already involved. However neither of the approaches work, at least in the medium to longer term.

As a source of competitive advantage speed is becoming perhaps the major factor in a world where the products have to be continually refreshed and enhanced to keep pace with the market, and the lifetime or products, and therefore the time to gain ROI, is reducing.

Flexibility

The traditional processes, with their emphasis on transforming on mass large bodies of requirements, detailed, signed-off specifications and fixed dates, budgets and scope, is extremely brittle in the face of change. As the project progresses, and requirements specifications, technical specifications and code harden, this brittleness increases as the cost of re-work, and peoples reluctance to change high ceremony work products, increases. The typical outcome is the introduction of formal change control processes, designed to limit the chance of change. 

Despite this upwards of 25% of requirements will change during a projects life-time, often shattering the project schedule and cost model, and providing a ready made excuse for non-delivery. 

Summary

All in all this is a pretty damming analysis of traditional development practices, but it should be no surprise to those in the industry. For many years people have been aware of the problems and have been working on alternative approaches.  These approaches have become increasing popular with development professionals and are now reaching a level of maturing, both in terms of the methods themselves, their theoretical underpinnings and the tools that support them. These methods have recently been brought under the banner of Agile.

Agile

So how does Agile tackle our sources of competitive advantage?

Agile divides large projects into small batches of functionality, based on business priorities. These batches are then pushed through the development process one by one. Detailed work on the batches (requirements and design) is delayed until the batch is ready to go into production. The quality of each batch is based on fitness for purpose rather than conformance to specification and the batch is tested as soon as each function is available. When ready the batch is deployed live as soon as possible. Finally the order and content of the batches is constantly reviewed, changing in response to changing business priorities.

Dividing requirements into smaller batches improves the flow through the development process, increasing speed, efficiency, flexibility, quality and dependability.

Though the overall concept as described above is incredibly simple, the impact on the IS function and business community is non-trivial, analogous to some degree to the change in manufacturing from mass to lean production techniques. The way projects are conceived and funded changes. The relationship between the business and IT communities changes. The roles of the people involved change. Development and management tools and practices change. Processes designed specifically to handle the problems caused by traditional methods need to change. Finally there's a huge cultural and attitudinal change required from every level in the business.