NOTE: This document may be circulated or quoted from freely, as long as the copyright credit is included.
Two decades ago the "structured revolution" in systems analysis methodology offered an end to the most common obstacle to successful system development: the lack of a clear, complete, and unambiguous system specification1. Organizations that made a serious commitment to some version of structured analysis managed to bring their development projects under control and to tackle larger and more ambitious systems that they could have managed earlier.
Today, however, we're again seeing the most atrocious collections of so-called "requirements": incomplete, disorganized, ambiguous, and incomprehensible. The resulting projects are turning into major fiascos2.
How did this surprising and frustrating situation come about? Why have lessons learned in 1979 been forgotten in so many organizations? Let's look at four causes.
First, organizations have weak memories. Key people leave or retire, reorganizations change responsibilties, and anything not written down gets forgotten. Even when an organization carefully documents its standards and methodology, that documentation may grow obsolete over time and fall into disuse. In extreme cases, a new regime may deliberately throw out3 the standards and methodology established by its predecessor.
Second, the trend toward decentralization has eliminated the professional systems analyst role in a growing number of organizations. Even in the largest corporations some business units are attempting major system projects with no awareness of what systems analysis is or why rigorous specifications are needed. For them it's still 1964.
In the worst projects representatives of the user community and management are invited to contribute "requirements", which are then merged into an unstructured (but often numbered) list, which turns out to be a mixture of:
However, the list contains few, if any:
Such a list may be a helpful starting point for a systems analyst brought in after the project is underway, but it is of little value either to the developers or to the sponsoring users themselves. The people who prepared it have worked very hard and they feel a sense of accomplishment. They don't want to be told that essential requirements documentation work remains to be done.
Fewer and fewer companies are developing their own custom application software. Instead they're buying or leasing application packages. Some have gotten rid of their programmers altogether, while others have adopted a policy like this:
We will develop custom application software only when it is determined that no suitable software product exists. User organizations must be prepared to make compromises with their ideal requirements in order to exploit existing software.
Given the high cost of development and maintenance and the growing availability of good-quality application packages, that's a reasonable strategy for any organization that doesn't need to exploit a proprietary solution for competitive advantage.
The danger is that inexperienced managers may naively infer that once they've found an attractive software product, no real "system development project" is needed. The project team is simply given the assignment to install product X.
Unless the system is very small, skipping specifications leads to almost certain disappointment. For example:
The only aspects of application system development that buying packaged software eliminates are most5 of the internal design and programming.
A vigorous faction in the information systems profession actually asserts that formal specifications are bad! They've coined the disparaging term waterfall approach to suggest that, as water can't flow uphill, neither can specifications be reconsidered once "frozen". They propose instead incremental system development, an informal approach reinforced by some C.A.S.E.6 tools.
Of course, no enlightened advocate of rigorous specifications has ever proposed such inflexibility, and this is a non-issue. Like many other informal approaches, incremental development without rigorous specifications may work for a very small project but it breaks down in a larger major system project.
Naturally, we don't expect to get the specifications perfect, but we have to do the best we can. Then as the project proceeds, changing business conditions and new insights may suggest modifications, which can be considered rationally based on their impact on cost and schedule.
Some members of the systems analyst community itself have contributed to the requirements crisis through overzealous embracing of arcane and over-technical documentation methodologies. The so-called "Unified Modeling Language" (UML) is a current example of that trend.
The UML offers a repertoire of useful tools to help system designers7 work out the architecture of a new system. The UML can also help systems analysts refine their ideas and communicate with other systems analysts. For those purposes the UML has been a positive contribution and we endorse its use.
But what the UML doesn't do is to satisfy by itself the need for complete, correct, and understandable specifications. Those who try to use it as the sole or principal tool for documenting a system specification are only making the requirements crisis worse.
To assure the success of its system projects, whether the software is purchased or developed in-house, the organization must:
These issues are much too complicated to examine here. You may find some of the links to other pages on this web site helpful as a start.
An organization that tries to take on large-scale system projects without embracing these disciplines, will experience the frustrations, of intolerable delay, unusable applications, out-of-control costs, or worse.
Return to IDI Home Page