Although the economy seems to be recovering, budgets remain tight. Even existing resource levels can require justification if, for example, a replacement is needed for an employee lost to attrition. To meet this challenge and to provide ready answers to management's questions about our department's performance, we developed a process for gathering performance statistics and using them to predict our future need for resources. In addition to providing the numbers that are so important to management, the process facilitates recognition of employee accomplishments and identification of problems.
Our department documents a web application delivery appliance that has frequent software upgrades to provide new features, and the past year has seen the introduction of new models incorporating new technologies. Following are some of the management questions that illustrate the need to develop a process for gathering metrics:
We therefore decided to focus our efforts on the development of a reliable, repeatable, flexible model for gathering productivity metrics. Quality metrics would also be useful, but they were not included in this project. Our basic strategy was to determine the rate at which existing resources can produce pages of output and to apply that data to calculate the level of resources required for keeping up with the release schedule.
In developing our metrics, we had to consider the teams developing the content, the nature of the content, the state of the product, and a few other factors, as follows:
To develop our metrics, we analyzed the data we had collected during work on previous deliverables and extrapolated to calculate our future need for resources. One complication was that our previous deliverables had been traditional guides and help topics, and we planned to convert that documentation to DITA for single sourcing. However, we had been preparing the writers for the transition, and we felt that they would only have to learn new tools, not a new way of writing.
But we could not base our metrics simply on our existing rate of producing new pages. Management might respond that we just need to work more efficiently. We first needed to define our standard processes, so that we would have documentation to justify our hours-per-page data. Additionally, a clear definition might reveal possible ways to make our processes more efficient.
To define our standard processes, we first defined our document development life cycle (DDLC), and then we used that definition as a framework for identifying the common types of work our writers have to perform. We also defined the tasks in terms of output. To make our metrics as accurate as possible in predicting the need for resources, we categorized the types of features that we are required to document.
Defining the DDLC
A writer does not simply sit at a computer and crank out pages of documentation. Document development must progress through the following stages:
Identifying Common Types of Work
Our writers have to find time for the following activities:
Categorizing the Features
Some features are more difficult to document than others. We defined three levels of complexity for the writing tasks.
A highly complex task has the following characteristics:
A task of average complexity typically has the following characteristics:
A low-complexity task has the following characteristics:
To provide a guideline for estimating future projects, we also categorized features and bugs by the size of the effort required:
To determine how much time is needed for writing a page, we analyzed the output of our writers and editors during the course of a seven-month project. For printed deliverables, we compared the number of pages produced by each writer to the number of hours logged by that writer. We did not analyze help topics, because the process of creating those topics will change completely when our single-sourcing project is completed.
Before we could compile meaningful data, we had to answer the question of what constitutes a page. Some updates require only a few changes to certain pages, and others require new pages written from scratch. We applied percentages. A change to a page can count as 25 percent, 50 percent, 75 percent, or 100 percent of a new page, with every page that requires any kind of change counted as at least 25 percent. We defined a full page as about 300 words.
Our analysis yielded the following figures for projects of high, average, and low complexity, written by junior writers and senior writers:
On average, a junior writer can write one page in 9 hours, and a senior writer can write one page in 8 hours.
We try to make time for a content edit of every writing task. After an edit, the editor and writer often have to send the file back and forth to resolve queries and ambiguities, and then the editor has to review the file before signing off. In our experience, these activities have required the following amounts of time:
When we looked for benchmark data with which to evaluate our metrics, we found that our figures are roughly in line with the rest of the industry. The following sources are representative of what we found:
We collected our data from the spreadsheets on which our writers maintain their individual plans. Each writer's plan has a separate sheet for each major type of work: new or updated features, bugs, special projects, and release notes. Each sheet includes columns in which the writer estimates the size and complexity of the effort, the number of new pages and modified pages, and the number of hours required. For example, the sheet for new or updated features includes the following columns (in addition to columns for scheduling and tracking the documentation).
For each feature, the editor verifies the entry in the Complexity column.
Our initial analysis revealed that we had spent an average of 9.21 hours per page on the combined writing and editing tasks, a figure that compares favorably with the benchmarks.
We then entered the data into spreadsheets that facilitated comparison of our actual work distribution to the manager's ideal distribution, which gave us a basis for projecting future needs for resources. For example, we entered the following data for a senior writer (Table 1).
The data clearly show that, despite working many more hours than should be expected, this writer did not have time to address bugs, attend to project-related tasks, or work on special projects. The "Addtn Hours" column shows that to work on all of the projects her manager would like her to undertake, she would have had to work an additional 296 hours. In percentage terms, this writer's time would be 183 percent occupied. Even factoring a 10-percent efficiency increase into the manager's projections, this writer would be 165 percent occupied.
Table 2 contains the data for a low performer and identifies a reason why the senior writer had to devote so much time to new features. This writer took 38 percent leave instead of 20 percent.
Table 3a shows the resources that were devoted to each type of work and the resources available for each type of work in a seven-month period if all writers work standard 40-hour weeks. Table 3b shows that 5 writers did the work of 6.4 writers.
Table 4 shows the number and type of features documented by each writer.
This data provides an answer to management's question and comment of "Where has your time gone? The last software release did not include very many new features." Clearly, management's focus on the larger features had resulted in substantial underestimation of the number of new features to be documented.
Table 5 provides useful data for determining development goals. The "Writing Page Efficiency" figure represents the average hours per page (6.58) divided by the writer's hours per page.
Table 6 summarizes the percentages for all the writers and the lead. The table clearly shows that we were understaffed by 1.72 resources. With this data to justify our request, management approved requisitions for two additional writers.
Our metrics project not only gave us the quantified information needed by upper management, but also enabled the writers to quantify their productivity. Now, when they say that they have too much work to do, they can prove it. And they find it easier to realistically scope their upcoming projects. The data also sharpened our focus on areas needing more resources. In particular, it demonstrated the urgency of our need to reduce the load on certain writers. We now have a methodology that is working for us, and we will continue to refine it.
Center for Information-Development Management
Voice: (303) 232-7586
Fax: (303) 232-0659