First Steps in Implementing a Documentation Quality Program
A couple of years ago, the Information Design and Development (IDD) leadership in the BMC Software Enterprise Service Management (ESM) business unit initiated a documentation quality program that has enabled us to demonstrate the quality of our content and to track and communicate our improvements over time.
Implementing a documentation quality program is an important part of demonstrating the value that your IDD group brings to the organization. Implementing a quality program also demonstrates your group’s willingness to be accountable for contributing to the customer experience with your documentation.
When you decide to implement a documentation quality program in your group, plan for an initiative that will be an ongoing process that evolves over time. This article proposes initial steps that you can take—steps that are relatively easy to implement and that you can build on and expand.
Start to build your quality program with a checklist of activities that team members perform that contribute to documentation quality. Our IDD checklist was part of an R&D quality initiative to identify activities that would result in improved product quality and that were reported during product development lifecycle checkpoints.
The goals of our quality checklist were to
- Identify activities or KPIs (key performance indicators) that we believe lead to improved quality content.
- Measure how well we were doing when we started.
- Track and report progress over time—for ourselves, our cross-functional teams, and upper management.
- Integrate quality-focused activities as part of our IDD DNA and ensure quality activities complement or enhance other programs or initiatives.
In developing our quality program, we followed these steps:
- Form a Documentation Quality team.
- Define KPIs or checklist items.
- Provide guidelines to writers about how to accomplish KPIs.
- Determine when to measure or evaluate progress on quality KPIs.
- Decide how to report on KPIs and progress to IDD, cross-functional teams, and upper management.
- Identify focus areas for each evaluation period that will “move the needle” on our content quality goals.
- Adjust our quality goals as internal processes and business unit priorities change.
Forming the Documentation Quality Team
Our team consists of several senior and lead writers, an editor, and several IDD managers. Members have changed over time, and the percentage of member involvement has changed over time as product release requirements and timelines change.
When the group first started, we met every other week for an hour. Now that most of the quality KPIs and guidelines are in place, we meet monthly.
Defining KPIs or Checklist Items
We use a documentation checklist to identify our KPIs. We considered these factors when developing the checklist:
- Known issues in our documentation from comments made by our customers and by customer-facing groups, such as Customer Support, Product Management, and Professional Services.
- Challenges in cross-functional team communication and collaboration that can inhibit writers’ ability to obtain the information they need about documentation requirements—for example, requiring that writers communicate documentation plans and obtain stakeholder approval.
- Building continuous quality improvement in our documentation development process—for example, identifying a minimum set of required checklist items and gradually enlarging that set as activities are integrated into IDD’s DNA.
We decided not to use weighted checklist items because we were not trying to achieve a particular score. We want each writing team to be aware of activities that contribute to documentation quality, to plan for improvement, and to show how they’re accomplishing more checklist items over time. The measurement is simply the percentage of items accomplished each six months for the current product releases that each writing team supports.
KPIs are listed at the end of this article.
Providing Guidelines to Writers
Most of our checklist items involve guidelines for writers about the activities we expect to be completed. Many of these guidelines already existed and were developed by other groups. For example, in the checklist item for documenting the business value of product features and functionality, we linked to guidelines that were originally developed from agile development methodology and customized for our business unit processes.
Some checklist items didn’t have guidelines to which we could link. In those cases, the Documentation Quality team led discussions and finalized and publicized the guidelines. For example, the checklist item for performing usability testing for a product documentation web space required this kind of guideline development.
In addition to providing checklist guidelines, Documentation Quality team members periodically attend writing group staff meetings to address specific concerns, to help teams decide what to focus on, and to bring back topics for consideration to the Documentation Quality team.
Determining When to Measure Quality KPIs
Because we support a number of product lines and because many product releases occur at different times, we aligned our reporting of checklist status with the timing of BMC’s biannual performance evaluation and bonus period.
This cadence isn’t ideal, because it results in writing teams reporting their progress at different stages of their product releases, and a number of checklist items occur at different times during a release. For example, verification of compliance with trademark, copyright, third-party license information, and approved product names often occurs toward the end of a release, and the necessary information may not yet be available to writers. If a writing team has complied with this checklist item for previous releases, we wouldn’t include this checklist item in the list of activities to focus on—we would assume the team would complete that item for the current release.
Reporting Progress to IDD and Stakeholders
We use a number of communication avenues to report progress to IDD and to cross-functional teams:
- Regular IDD newsletter, the focus of which we alternate between internal IDD and cross-functional teams
- Regular IDD All Hands meetings, led by IDD leadership
- Regular IDD Information Exchange meetings, in which anyone in IDD can share information—for example, talk about a new idea, discuss how they overcame a challenge, or share how they improved a process
- Regular Lead ID meetings, in which lead writers and IDD leadership work on special projects or initiatives, the results of which are communicated to all the writing teams
- Regular cross-functional interlock meetings that IDD lead writers and managers have with stakeholder managers—for example, from Customer Support, Product Management, and Professional Services
Moving the Needle for the Future and Quality Goal Refinement
Your quality program should include a process for continual improvement and for demonstrating how you will act on what you learn. After compiling results across each writing team, we look for areas where teams struggle on a regular basis. At the beginning of each period, the Documentation Quality team works with IDD leadership to identify the top one to three focus areas for the next six months. For example, our initial reports indicated that a number of writing teams weren’t regularly reviewing legacy documentation to ensure the content still made sense in the context of new features and functionality. To close that gap, we recommended that testing of legacy content be one of the focus areas for this year.
Also, we adjust our checklist items as our internal processes change. For example, when R&D added a new quality activity for IDD and R&D to conduct a documentation-specific iteration focused on a targeted area of concern or a particular customer pain point, the Documentation Quality team added that checklist item and linked to information about how to conduct the documentation iteration.
Quality as Part of Our DNA
All the original quality checklist items are regularly completed by all writing teams, and now our focus has shifted to new checklist items that continue to improve our content. The quality program is no longer an initiative, but part of our IDD DNA.
It’s important to note that our IDD quality program started about the same time that we started a customer relationship program, where writers started proactively interacting directly with customers to obtain documentation feedback. The customer relationship program included two other first-time activities for IDD: implementing a customer survey only about documentation and customizing the regular R&D customer satisfaction survey results to highlight only the documentation feedback. With this new information, directly and indirectly from customers, we can measure customer satisfaction with documentation much better and validate that our improvements are making a difference.
Also, we started producing an IDD newsletter, which gave us a regular communication avenue to IDD, our cross-functional teams, and upper management. IDD’s quality goals, activities, and documentation improvements are regularly visible to us and our stakeholders.
When starting out with a documentation quality program at your organization, you don’t have to also start a customer relationship program or produce an IDD newsletter, but these activities can enhance your quality program.
BMC’s Key Performance Indicators (KPI)
- Ensure writers are adequately trained on products they’re documenting.
- Ensure IDD has access to VM, QA computer, or sandbox to run product.
- Provide documentation plan for major releases and obtain stakeholder approval—plan includes solution enablement, quality, and customer experience goals.
- Participate in R&D iteration planning and reviews, and ensure documentation updates are included in iteration demos, when applicable.
- Review and prioritize customer reported documentation defects and issues, customer reported RFEs (requests for enhancement), and docs.bmc.com customer online web comments. Determine trends, if applicable, and make plans to address in the backlog.
- Work with R&D to allocate time for a documentation sprint during a major or minor release for a targeted area of concern (legacy or new).
- Complete topic and user goal analysis of new features.
- Ensure documentation describes business value and features in context of the solution, suite, or value path, if applicable.
- Collaborate with writers on content across the solution.
- Describe limitations of a feature, if applicable.
- Conduct or participate in direct customer interaction, when available.
- Conduct or participate in regular discussions with Support, BMC IT, and Field (for example, Professional Service, Software Consulting, Customer Enablement). Look for Support knowledge base articles that can be incorporated into core documentation.
- Verify compliance with legal guidelines (trademark, copyright, and third-party license information).
- Verify all product names are approved.
- Conduct documentation reviews and validation of new feature content in context of use case impacted by new functionality.
- Conduct documentation testing for functionality and navigation.
- Conduct documentation usability testing, when applicable.
- Conduct procedure checks of existing/legacy content.
- Test online help using online help checklists.
- Perform quality checks on translated content using L10N checklists.
- Ensure cross-functional team (for example, QA, Dev, PM, Support) reviews and approves all content.
- Ensure QA tests and approves new feature/functionality content including installation, migration, and upgrade documentation.
- Ensure Interoperability lab, BMC IT, or Solution QA tests solution/integration documentation content.
- Complete edit of new deliverables and new content (editor preferred, peer acceptable).
- If producing PDFs (book PDFs or wiki), complete a PDF check of the final books using the appropriate checklist.