JoAnn Hackos, PhD
The U.S. government’s definition of productivity is based upon a manufacturing model. To measure productivity, you simply count the number of widgets produced and divide by the amount of time or cost it took to produce them. If you’re dealing with widgets, simple productivity measurements seem to make some sense in evaluating gains or losses in productivity.
For example, if it takes 100 hours of work to make 100 widgets, we have a productivity factor of 1. If we reduce the number of hours by half, now making 100 widgets in 50 hours, we have a productivity factor of 2. That means a 100 percent increase in productivity.
We can do a similar calculation using cost. If it costs us $100 dollars to make 100 widgets, reducing the cost of manufacturing by half gives us a productivity increase of 100 percent. In fact, that’s exactly what financial experts use to justify their decisions to move manufacturing to countries that provide cheap labor. Even if the same number of people are needed to perform the work, or even if more people are used, the result is an increase in productivity.
But what about the output of knowledge workers such as information developers? Is it that easy to measure productivity when the inputs and outputs are difficult to define?
Take the popular measure among information developers of counting hours per page. Teams divide the total number of documentation pages produced by the number of hours required to produce them. The measurement is very easy to make and quite accurate. However, it tells us virtually nothing about the productivity of the team.
Just compare this measure with the widget example. If any group of assembly line workers makes 100 defective, absolutely useless widgets in 50 hours, have they increased their productivity? Doesn’t the quality of the output have something to do with the quality of the measurements? The problem with quality points to the problem with such a simplistic productivity measure. Productivity experts suggest that we must take into account more than simplistic definitions of the work (inputs) and the products (outputs), especially when we consider the actions of knowledge workers.
In their June 1994 article, “Evaluating Knowledge Worker Productivity: Literature Review”, Beverly E. Thomas and John P. Barons provide a useful and thorough, if ultimately disappointing, account of the difficulty of measuring knowledge worker productivity. The article, although it is only a literature review, is ultimately disappointing because it does not provide a solution to the problem. The authors suggest that
“Quantifying knowledge work tasks is difficult. The literature suggests organizations categorize work by content, then select the most appropriate measurement technique based on implementation costs. Inaccuracies in productivity measurement are acceptable if the level of inaccuracy remains constant over time. The measures are most important for tracking trends, not quantifying empirical data.”
Thomas and Barons argue that the methods of evaluating productivity must be consistent, even if they are not particularly accurate, since the real point of productivity measures is to discover change. They recommend that many factors be used to determine the work effort of knowledge workers, including the quality of the output.
Let’s consider how we might measure quality. If customers have sufficient information to reach their goals with the help of the technical information and they express satisfaction with the experience in terms of ease of performance and timeliness, would we say we have been successful in producing high quality information? I suspect most of us would agree that this outcome means success.
However, how do we evaluate the value of this success? Is it worth one new product sale per year or 20 sales? We might consider what a happy and productive customer is worth. One way to measure the value is to consider how much a customer’s time is worth. If we are able to decrease learning time by 20 percent, meaning that our customer becomes productive in 20 percent less time than it took before, we have increased that individual’s productivity. We sell technology with the claim that it will increase productivity by making otherwise difficult tasks easier and faster to perform. If excellent information aids in this effort, we can take credit for customer productivity improvements. If a typical fully burdened knowledge worker’s salary is worth $150,000 USD, then by saving 20 percent, we will have saved $30,000. If we consider one month of this savings to be attributable to our efforts, we can divide by 12, which equals $2,500 per customer per year. If you have 1,000 customers using the product and being more productive, the gain will be $2,500,000. That is, through your efforts in producing quality information, you will have produced an output valued at that amount.
But we’re not done yet. We must look at the cost of the effort required to achieve this outcome. We must carefully evaluate the elements that drive costs and decide if they are worth the outcome. To produce information as successful with customers as the one I’ve described here will take some effort. The writer must not only be knowledgeable about the product but also knowledgeable about the customer. The writer must know how to produce minimalist information as well, which means just enough to meet the customer’s needs but not so much that time is wasted creating, reviewing, and publishing it. We might find, for example, that the amount of time the information development team required to produce a 100-page manual for this customer was 1,000 hours, a fairly generous number. This time included customer site visits and task analysis, plus usability testing of the results. Perhaps another 100 hours might have been spent outside the team, on reviews and discussions with the writers. The cost of the 1,100 hours was probably close to $82,500. In addition, we might consider the cost of printing and distributing the manuals to 1,000 customers, unless the information is being delivered through the Web. A generous $10 per manual would give us a publishing cost of $10,000. We are now getting close to an input cost of $100,000 to gain a value of $2,500,000 in customer productivity gains.
Now, you might look further into the input costs of the knowledge worker. We need to evaluate how many interactions of other team members are needed to make our knowledge worker productive beyond the development of one 100-page manual in about six months time. One worker is dependent on the time and effort of many others, including peers, managers, subject-matter experts, even customers. Knowledge work is most notable for its complexity. In general, we believe that the higher the quality of the output, the more complex the activities required to produce that quality.
The productivity researchers suggest, among other methods, the process of keeping diaries of activities among the team members. These diaries record by category what kind of work is being done and by whom. The evaluators then compare the amount of time taken to perform specific tasks compared to averages or best practices for the task. They also evaluate if the work task is commensurate with the skill level of the worker. For example, they might consider that a senior writer with many years of experience is being unproductive when he or she is struggling with desktop publishing problems, principally because this clerical task could be better performed by someone less costly.
The real issue, of course, is not, as the authors indicate, an absolutely accurate, one-time measure of productivity. The issue is improvement. In our hypothetical situation, how might we measure productivity improvements? If some of the tasks could be relegated to less experienced, less expensive workers, would that not result in a productivity gain? Would the work then be less costly to accomplish, leaving the experienced worker with more time for high-level efforts?
Let me ask you to offer your thoughts on measuring productivity. This is a discussion we will continue up until the October 18–20, 2004 Best Practices conference, since productivity increases is one part of our theme: Building Productivity through Innovation. If you have found ways to measure productivity improvements, I would like to hear about them. We can all learn from each other’s best experiences.
Contact firstname.lastname@example.org for questions or comments.