Mixed benefits from methodology breakthroughs . . .

If it ain't broke, fix it anyway

by Conrad Weisert
July 23, 2005, (revised May, 2006)
© 2005 Information Disciplines, Inc.


The failed promise of major dramatic breakthoughs

The past fifty years have seen a succession of major dramatic breakthroughs (MDB) in software development tools and methodology. Occurring roughly every three or four years, those MDBs have each promised an improvement in programming productivity ranging between twenty percent and an order of magnitude. If those expected benefits had all been realized, then either

Although today's application software is undoubtedly more complex than typical applications of the 1960s, few developers and even fewer users claim a factor of 1000. Besides, much of the growth lies in fancy user interfaces rather than in what the applications actually do. Organizations report that out-of-control projects wasting years of time and millions of dollars are as common today as they were a generation ago. Methodologists, trying to persuade us to adopt theit latest fad, concur.

Why?

It follows either that the promised benefits of each MDB were wildly exaggerated or that software-development organizations have consistently failed to adopt the new approaches in letter and in spirit. Both explanations are responsible for the ongoing disappointment.

Although both problems are hard to overcome, the first will eventually yield to staff education and management discipline. The second poses the greater danger, as methodology gurus intimidate practitioners into substituting the latest fad for techniques that have worked well. This article, therefore, will focus on the methodologies themselves.

Which MDBs?

Let's review some MDBs of the past decades, classifying them in two ways. For each major dramatic breakthrough, we'll note:

  1. Whether the new methodology component is

  2. Whether the MDB1
Year2 MDB Impact Benefits* Drawbacks*
1957FortranRevolutionary Enabled engineers & other non-programmers to solve their own problems.
Demonstrated practical code optimization.
Inhibited large-scale modular program organization.
Unnecessarily IBM-704-specific.
Facilitated loss-of-control bugs.
~1959Reusable componentsEvolutionary Lowered cost of development and maintenance, especially debugging; encouraged standardized approaches (See note 1)
Adoption very slow by business application developers
1961COBOLRevolutionary Large productivity gain in initial development over assembly languages Encouraged a monolithic programming style leading to high cost of future maintenance; inhibited reusable components and (later) structured programming
1964PL/I (& other Algol-like languages)Evolutionary Encouraged modular program structure; supported modern technology such as multithreading. Costly to implement; perceived as hard to master.
~1967System Development Life Cycle (SDLC)Revolutionary Limited risk of losing control; empowered sponsoring user organization (See note 1)
~1969Structured codingEvolutionary Lowered maintenance costs by making source code much easier to understand and modify (See note 1)
~1971Structured design / top-down refinementEvolutionary Facilitated modular program structure (See note 1)
~1976Online developmentEvolutionary Eliminated impact of turn-around time on testing Encouraged expedient short-cuts that compromise quality
~1978Structured analysisRevolutionary Addressed "requirements crisis";
clarified role of systems analyst;
eliminated confusion between analysis and design
(See note 1)
~1980Relational data modelEvolutionary Normal form helped simplify database structure;
Views helped separate program logic from database design
(See note 1)
~1984Non-procedural languagesEvolutionary Empowered end-users to bypass development services Some user orgs. became dependent on unsupported and undocumented programs
~1987Event-driven paradigm (with GUI)Revolutionary Facilitated consistent user interface Diverted emphasis from what an application does to how it looks
~1990Object-Oriented ProgrammingEvolutionary Enforced highly modular program structure; encouraged reusable components (See note 1)
1993UMLRevolutionary Standardized object-oriented graphical models Resurrected requirements crisis
1995Use casesRevolutionary Attempted to overcome UML requirements deficiencies Undermined structure in system specification
1997The Java PlatformRevolutionary Facilitated Internet application deployment Imposed complexity on some formerly straightforward programming functions; promoted distortions of object paradigm
2000Extreme Programming (and other so-called "agile methods")Revolutionary Fast development of single-program applications with heavy sponsor participation. Undermines packaged application software solutions

Note 1: A persuasive case can be made that those seven MDBs, when practiced by well-trained practitioners, impose no drawbacks and are still considered non-controversial enlightened good practice.

The discredited traditional approach

Veteran software developers have noticed a striking phenomenon: Each major dramatic breakthrough becomes tomorrow's traditional approach. I recall listening in the early 1970s to presentations contrasting the phased life cycle with "the traditional approach", and just a few years later learning how structured analysis would solve the requirements crisis that had characterized "the traditional approach". Today, it's rare to attend a presentation on UML or agile methods that doesn't contrast those new methods with "the traditional approach", by which they usually mean the phased life cycle and structured analysis.

Alas, in our profession tradition is often seen as a bad thing. When a methodologist uses the term, he or she implies a dismally unenlightened disorganized and undocumented mess. In a 1980 structured analysis course the instructor advised the students to begin the analysis phase by documenting a model of the current system. When I asked him how that step should be done if the current system was already well documented, he was astonished. He simply couldn't imagine a well documented system that pre-dated structured analysis. A quarter-century later, fad methodologists cannot imagine how a "legacy system" could possibly be well organized and coherently documented.

Growing intolerance and age discrimination

In the past dozen years, advocates of new methodologies have moved on from mildly disparaging characterizing of the current so-called "traditional approach" to rather mean-spirited condemnations of established processes and of the people who practice them. Procedural programming, they point out, was done in dinosaur languages most likely by dinosaur programmers. They sneeringly condemn the phased life cycle as a bureaucratic and hopelessly inflexible waterfall approach. People who insist on practicing it are characterized as dismally unenlightened if not "brain-dead".

We encounter those attitudes toward mature professionals not only in meeting forums, but even in articles and books written by fad methodologists. A nasty example is reviewed on this web site.

Those attitudes add much fuel to the age discrimination felt today not only by those over, say, 55, but increasingly even by 39-year-olds. Why would a manager want to hire a dinosaur who rejects or cannot learn modern methods? Recruiters are counseling job applicants to omit experience with COBOL and other pre-1985 technology from their résumés.

Ignorance

Not surprisingly, intolerance is often paired with ignorance. Some3 of the methodologists who specialize in the latest MDB are shockingly unaware of what came before and how the software development field has evolved. Among the misinformation I've heard in presentations by experts are the following:

Younger audience members including managers nod in agreement and sometimes take notes. Note that each of the above misstatements was about fact not opinion.

Bottom line advice for managers

Neither tradition nor innovation deserves unquestioned acceptance. Newer doesn't mean better; neither does tried and true.

The sensible course, then, for a modern software development organization, is to let your methodology evolve in the normal course of your projects. A participative approach in which a knowledgeable professsional staff proposes incremental changes to the organization's standards and processes will yield a good balance between stable and up-to-date methodology.

Here are a few recommendations:


1 -- The assessments in the table are my own opinion, and don't reflect quantitative measurement. They're based on a combination of experience, common sense, discussions with respected colleagues, and observations of actual projects in client organizations. If you strongly disagree, let me know, and I'll either revise my opinion or append your dissenting view to this article.

2 -- The years are approximate, reflecting the time at which a sizable body of professional and trade literature publicized the methodology as a major dramatic breakthrough. If you have a more exact date, let me know.

3 -- Of course, many of our methodology expert colleagues exercise sound judgment. My criticisms here are aimed at what I believe (and hope) is a small (but awfully loud) minority of methodology gurus.

Return to Management articles
IDI home page

Last modified October 6, 2010