The Case for an Applied Methodology


August 2002

The Case for an Applied Methodology

CIDMIconNewsletter Lynn Becker, Principal Program Manager, Process Engineering, Rockwell Collins

Single sourcing is a powerful concept but one that in implementation many interpret strictly in terms of the tool and technology. Single sourcing is, or should be, viewed as having four strong elements: process, standards, roles, and responsibilities of the people involved, as well as the tool selected to support the effort. Many organizations in their haste to derive the benefits of single sourcing have rushed into tool buying without looking at their processes and standards. In these organizations, the purchase of the tool may drive their processes and standards instead of the converse. In any single-sourcing equation, the processes, standards, and people should always drive the selection of the tool and technology; in other words, organizations should select the tool and technology to support the spirit, intent, and manner in which the product is designed and developed. Organizations should, therefore, focus on using an applied methodology to design and develop a single-sourcing solution. In a subsequent article, I will discuss a recommended implementation of the methodology.

Consequences of Failure

To begin, what are the possible consequences for not using an applied methodology to design and develop your single-sourcing solution? There are but three: complete success, partial success, or failure. While the odds look pretty good for some type of success, they are, in truth, not. Even projects that may be partially successful usually come at great expense. The days when a company could just launch a project with little initial analysis and planning are long gone. The tools and technology are too expensive and the cost of poor quality or failure is too great for the slim profit margins many organizations are dealing with today.

Project Classification

Resolution Type



Top 3 Factors Contributing to Success or Failure

Project Successful

Completed on-time and on-budget with all functions and features as originally designed

  • Excellent user involvement
  • Good executive management support
  • Clear statement of requirements

Project Challenged

Completed and operational but over budget, over time estimates, and have fewer functions and features than originally designed

  • Lack of user input
  • Incomplete requirements and specifications
  • Changing requirements and specifications

Project Impaired

Canceled during the development cycle

  • Incomplete requirements
  • Lack of user involvement
  • Lack of resources

In the short history of software-development projects, the statistics gathered during several surveys conducted from 1994 to 1998 by the Standish Group regarding project and project outcome were very depressing. The studies focused on the scope of software project failures, the major factors that caused software projects to fail, and the key ingredients that can reduce project failures. The numbers indicated that over 30% of software projects were canceled before they were completed. Of the projects that were completed, over 75% were late, over-budget, or were delivered with reduced functionality. For these projects, 52.7% experienced average cost overruns of 189%, and average time overruns were 222%.

The Standish Group estimates that in 1995 alone, American companies spent $81 billion for canceled software projects and these same companies paid an additional $59 billion for projects that were completed but exceeded original time estimates. Larger companies fared worse because their projects survived with only 42% of the original specifications while smaller companies finished 78.4% of their projects with at least 74% of the original features and functions. When projects such as these are considered to be a partial success, it is evident that it would not be in your best interest to lead or to be associated with the project.1

The respondents in these studies were IT executive managers. The sample size was 365 respondents whose departments had experience with an accumulated total of 8,380 applications or an average of 30 applications per respondent. The Standish Group also conducted four focus groups to provide qualitative context for the survey information. Projects were classified as follows in the table below.

Badly designed and difficult to use systems cause additional costs. Organizations often must provide about $3150 in technical support for every user (Gartner Group, 1997).2 Non-technical employees who help co-workers to resolve computer problems typically use 4%-10% of their time at a cost of $10,500 per year for each computer (Nolan Norton Institute, 1997).3 Unproductive activities with computers (support and housekeeping, response time delays, checks for accuracy, and so on) cost another $5590 per computer per year (Gartner Group, 1997).2 A study of 2,000 US companies revealed that about 40% of the implemented systems did not achieve the expected results (Bikson & Gutek, 1984).4

Cost of Poor Quality

One of the less obvious costs of doing business is the cost of rework. During a production process, errors cost exponentially more the farther along in the process they are before discovery. Shown in the figure on page 88 is a depiction of the cost involved for discovery of the same error throughout the existing software-development process. Errors found during the requirements phase cost the least to repair or rework. If an error is found during the design, it will cost five times as much to rework. During coding, that error will cost $10 for every $1 it would have cost, and so on. These ratios have been validated by a number of different sources besides Dean Leffingwell, one of them within my own organization.

The Cost of Rework10

The Cost of Poor Quality5

What Is Your Business Case for Your Single-Sourcing Solution?

Most organizations do not cavalierly make a decision to go out and buy a single-sourcing tool. Usually, there is a driver, most often an employee who learned about the tool through an article, from a friend, or at a conference, and who becomes sold on the benefits of using a single-source solution. This person is most often an influencer, but sometimes he or she is a decision-maker. In either case, influencers had to come up with some set of justifications for implementing a single-source solution, such as a business case. No doubt at least some, if not all, of the factors that helped to sell the proposal were centered around reducing costs, decreasing development time, training time, and revision time, while improving productivity and error rate. Hard to imagine attaining enough of these benefits to offset the increased costs detailed in the statistics previously cited, isn’t it?

As the Standish Group’s findings have shown, applying a defined methodology can provide some tangential benefits and ensure that you meet initial expectations. Tangential benefits are the reduced cost of future redesign of the architecture to make future versions of the product more usable, reduced staff turnover as a result of higher satisfaction and motivation with the system, reduced time spent by other staff providing assistance when users encounter difficulties, and reduced help line support. These benefits become a reality when they have been anticipated and planned for during the design of the solution.

Choosing a Methodology

There are any number of methodologies that can be used in planning and managing the single-source project. The methodology itself is not as important as the discipline and the structure it requires to adhere to it. You can adapt methodologies from a variety of fields for use in this process. The Software Development Life Cycle (SDLC) and the Software Capability Maturity Model are two examples from the software field. The training field offers instructional systems design methodologies, such as ADDIE (analysis, design, development, implementation, and evaluation). The engineering field offers rapid prototyping models that you can modify for use in any type of project. The growing project management field offers a number of different methodologies. Finally, you can adapt processes from the Six Sigma and ISO worlds for your use.

The important elements of choosing a methodology include the need for adequate analysis prior to design and development and the need for some type of evaluation process that provides feedback loops throughout the entire project and during the maintenance period. Another important aspect to consider when choosing a methodology is project or program management throughout the performance of the project. In fact while reviewing the different methodologies, you can see a great deal of synergy among them. At a high level, most of the phases can be broken into five distinct efforts: analysis, design, development, implementation, and evaluation. These five phases can be considered the basis for any single-source process methodology.


Prior to buying a tool, an organization should spend adequate time in assessing why it wants to implement single sourcing, what it hopes to gain from successful implementation of the solution, who will be using the tool and process, when the solution is needed, and where the most benefit will be gained. The analysis phase should include a needs assessment, risk assessment, task analysis, and a cost benefit analysis.

The needs assessment should identify user groups, recipient groups, and support groups along with concise descriptions of the environment in which each group will interact with the single-sourcing solution process. The process currently used should be analyzed for waste, efficiency, robustness, and scalability. You should ask questions, such as: Is there any waste in the current processes for development of an information product? Are all the steps used to develop the product necessary? The process should be mapped out from concept to customer delivery and aftermarket support. Each step of the process should be analyzed for value creation, for example: Does this step add value to the end product? Is the value added something customers would be willing to pay for if they knew of its existence? Added value can be considered to be decreased cycle time, decreased cost, increased quality, or better service, among other factors. The process must be examined from a systems point of view, and you should decide whether the process is acceptable, needs modification, or should be completely redesigned. Sometimes, a redesign is cheaper in the long run than a modified process that necessitates incremental improvement on a regular basis over time.

You should also conduct a careful analysis of the tasks that will need to be accomplished to implement the process. The task listing will enable a better estimate of time and effort for the project and will act as the basis for the work breakdown structure (WBS) once the decision has been made. Few processes can withstand such a close look without some kind of modification to the current process being recommended. The organization should not even consider selecting a tool until the analysis team has a systems understanding of the process and any proposed changes.

The risk assessment is an effort often overlooked but allows the design and development team to anticipate risk areas and plan for risk mitigation in those areas. Once the risk assessment is performed, the project manager and team should develop a risk and opportunity management plan to use through the course of the project. They should identify any symptoms that would indicate a risk is developing, estimate the likelihood of that risk occurring, and determine an appropriate strategy to mitigate the risk. If performed well, the risk assessment precludes ugly surprises from happening in the midst of the project.

A single-source project can be categorized by the following risks: organizational, process, and product. Organizational risks involve the different approaches and capabilities that an organization can bring to a single-source effort. Questions you need to ask about organizational risk include: How can the organization’s current infrastructure support this type of project? Are there conflicts between organizational objectives that must be addressed? Does an organization’s personnel work well together and across business unit lines? Does the project have staffing for all currently required skill sets?

Process risks involve all aspects of development within a single-source environment. Questions you need to ask about process risks include: Is the current subcontract administration appropriate and sufficient for the needs of this project? Have near-term milestones been defined to a level of detail sufficient for monitoring progress? Are the project objectives clear and feasible? Are the budget estimates stable, reasonable, and precedented? Are the schedule estimates stable, reasonable, and precedented?

Product risks are inherent within the single-source tool. Questions you need to ask about product risks include: Does the architecture allow for a modular approach to content? Will any planned decentralized database processing provide adequate and reliable performance? Does the run time computer environment provide adequate processing power?6

Production or developmental risks include product engineering, the development environment, and program constraints. Product engineering is considered to be the technical aspects of putting a working single-source environment in place. The development environment includes the methods, procedures, and tools used to produce deliverables. The program constraints are the contractual, organizational, and operational factors within which the deliverables are developed but that are generally outside the direct control of the local management. Questions you need to ask to assess these risks include: Are requirements missing or not completely specified? Is the implementation of the design difficult or impossible? Do requirements specify something no one has ever done before or that the organization has not done before?

Other general questions that should be answered prior to deciding to proceed include: Does management commitment exist? What are management’s expectations for the new process and tool? Who will champion the project? Are the requirements defined? Is the funding available? Will cultural changes be addressed? Will training be available? Is the correct staffing available to perform the project? Once the analysis is complete, the full breadth and depth of the project should have been defined.


Once the decision has been made to go ahead with the design phase of your single-source project, the project should be very easy to handle. The design of the new process or modifications to the old process should be completed because you’ve already created a blueprint for future work. Final decisions on what tool will support the design can now be made with complete assurance that it is the right tool for the job. Determinations on how to measure and what to measure to assess project success should be made, and tracking of project success should begin. These metrics should be tracked and maintained until the project is finished. The master schedule and the implementation plan should be written; part of the implementation plan should include sections on communication, change management, training, quality assurance, and definitions of roles and responsibilities.

All of the requirements that have been determined during the analysis phase should be incorporated into a design for the process and related tool. Database design should occur at this phase as well as user interface design if it is required to facilitate future use of the tool. In a future article, we will take a more detailed look at one recommended methodology to apply when implementing a single-source project.


Adopting a methodology appears to be a very labor-intensive task, and indeed it is. However, the level of effort it takes to implement such a methodology is offset by the decreased level of rework and redesign that occurs later in an unstructured project. You can argue that buying a tool should not require this kind of effort if your organization has already refined all of its processes and has a good program-management and project-management discipline in place. But I would have to argue that if your organization already has these processes and disciplines in place, you probably already use some kind of applied methodology to justify the purchase of the tools you wish to implement as your single-source solution. This discussion is intended for those organizations that still cowboy their way through business. But even if you are a member of such an organization, if you are given (or stuck with) the task of selecting your new tool, it would be wise for you to give some thought to using a defined methodology to facilitate your success. Good luck! CIDMIconNewsletter

About the Author



1 The Standish Group, The CHAOS Report (1994) and Unfinished Voyages (1996),

2 Gartner Group (1997). Cited in Gibbs, W.W. (1997) Taking computers to task. Scientific American, 7/97.

3 Nolan Norton Institute (1997). Cited in Gibbs, W.W. (1997) Taking computers to task. Scientific American, 7/97.

4 T.K. Bikson and B. Gutek, Rand Corp., Implementation of Office Automation, Santa Monica, CA, 1984.

5 Calculating Your Return on Investment from More Effective Requirements Management, Dean Leffingwell, Rational Software, Inc., 1996-1999.

6 Software Engineering Institute Technical Report CMU/SEI-93-TR6, Taxonomy-Based Risk Identification, June 1993.