Measuring Information Quality: Who said it can’t be done!
Peter Fournier’s article in this edition of Best Practices demonstrates that information-development organizations are increasingly interested in measuring productivity. The increase is due, at least in part, to the investment in new technologies and new methods of developing content. Measurements of the percentage of reuse that an organization achieves become part of the message to senior management that the investments are producing a return.
Nonetheless, information developers must not forget that all the improvements in productivity and efficiency don’t necessarily translate into improvements in the quality of the information delivered to their customers. We need to ensure that our investments have a positive impact on customers by invoking measurements of the quality of our work.
David Skolnick’s article points to the importance of measuring quality, beginning with a definition of quality for the stakeholders in the end-to-end process. He points to seven characteristics of quality content: accuracy, completeness, readable, graphical, balanced, producible, and reusable. Each of the characteristics provides value to both internal and external customers of the content.
The measurements that Skolnick provides require an examination of the content before it is published. Such measurements are called leading indicators. That means that they attempt to predict the success of the content by examining factors that can be assessed and measured before the content reaches the customer.
Lagging Indicators Lead to Quality Improvements
In his keynote address to the Content Management Strategies conference in April, Gerry McGovern noted that one of the most important quality measurements is the customers’ success in reaching their goals. In his example, a customer trying to download software measures success negatively in relationship to the time, confusion, and frustration of the download process and its instructions. The more time, confusion, and frustration that the user experiences, the less successful the instruction and the lower the quality.
Customers are successful when they quickly and easily find the information they need and then use it to perform their tasks and reach their goals. Measuring their success means using lagging indicators.
Because these indicators measure the quality of content after it is released, they don’t affect how we develop that content until the next time around. If, for example, we learn that customers cannot find the content they are looking for in our print books, we might improve the table of contents and the index or re-title the topics so that they match customer queries. If we are delivering the content on a website, we might also provide better metadata to enhance searches or improve the specificity of short descriptions.
Many CIDM managers have recently pointed to another customer problem. The content they find does not provide the end-to-end solutions that they are seeking. As a result, these managers are working with customers to identify the problems they are trying to solve and asking writers to provide comprehensive solutions to the customer problems. They are asking for scenarios that fully describe the customers’ use of the software or hardware product so that writers can provide complete and accurate support for the scenarios. As a result, these organizations are moving away from an emphasis on features and functions to a new focus on the customers’ requirements.
In a recent conversation, one senior manager described his team’s work in developing comprehensive and meaningful examples that customers can apply to their own problems. His team is measuring how many examples they have been able to add to the content environment.
In each case, a lagging indicator, based on customer feedback, spurred actions to change and improve the quality of the delivered content.
Quality Improvements Require Customer Feedback
Perhaps the saddest, most discouraging comment I hear from information developers is this one:
“They won’t let us talk to customers.”
That comment usually means that others in the organization who have access to customers conclude that information developers have no need for feedback from customers or that the feedback can be secondhand.
Consider some examples of organizational barriers:
“The customers are too busy and don’t want to be bothered talking to technical writers.”
“Our company’s relationship with an important customer is too shaky to allow anyone except the sales representative to talk with them.”
“There is no room in the training courses for anyone besides the customers.”
“It is too costly to travel to customer locations.”
“Technical writers might say something that they are not supposed to say to the customer.”
In addition, we learn of impediments coming from the information-development organization itself when writers tell us:
“We are too busy getting the next release out to visit customers.”
“We don’t have enough staff to let anyone out of the office to talk to customers.”
“It’s too hard to find the customers who would be willing to talk to us. They’re too busy.”
The first set of objections can be overcome by presenting a sound plan and ensuring that staff is well trained in interviewing techniques. These objections also require that the manager communicate the importance of customer contacts to senior managers. One group I heard from paid their own way by visiting customers on staff vacation trips that were close to customer locations. Once they built a track record of valuable feedback, they got the funding they needed to continue.
The second set of objections is an issue for change management, including convincing information-development managers themselves that customer contact is essential to pursuing a goal of high-quality, valuable content.
Much of the hesitation, I believe, comes from fear. Staff members are unsure of their ability to meet customers face-to-face. They don’t want to appear awkward or clumsy in interacting with customers. They need training and a well-constructed plan.
So, here is my advice to your organization. Develop a plan with training to educate and prepare your staff for more customer contacts. Present your plan to your management by describing fully what you intend to do and what the outcome of the customer contacts will be to your organization and to the company. Stress the benefits that knowing the customer will have on customer satisfaction with your company’s products, not just the documentation itself. Indicate that it’s about time to look at the lagging indicators. The more positive customers are about their interaction with your product information, the more likely they are to buy more products and recommend your products to others.