The Balanced Scorecard: Monitoring the Health of Your Organization
Your team is missing deadlines. Customers are complaining about the content you produce. Your writers are stressed out. Costs of producing content seem to be escalating. You want to adopt new strategies and best practices, but you’re stuck in a rut. Clearly, it doesn’t take an organizational thermometer to see that all’s not as it should be—there’s something affecting the overall health of your organization. But what can you do to diagnose the issues, and more importantly, monitor them in the future so that you can take preventive actions before you face an epidemic of these proportions?
Managers seem to be ever on a quest for the right metrics to determine organizational health. However, an overemphasis on one factor—such as the cost of producing a topic of content—frequently leads to treatment of a symptom in isolation. In effect, it’s as if we measure blood pressure, while ignoring cholesterol levels, and then wonder why our patient had a heart attack. We need to take a holistic approach to managing and monitoring the fitness of our team.
A balanced scorecard enables managers to examine elements of their organizations on a regular basis and helps flag problems before they impact the overall performance of the team. Although each organization may have different tolerances in various areas, use different scales to measure certain factors, or even use different key performance indicators (KPIs) to evaluate an objective, four critical dimensions seem to universally impact the well-being of information development teams:
- Planning—How well defined are the information products that the organization needs to create?
- Execution—How well does the organization perform against the plan?
- Quality—How well does the content produced meet the needs and expectations of the users?
- Growth—How much effort is put into improving and building core skills within the group?
To gain a complete view of your organization’s ability to meet the expectations of your company and your customers, you must consider all four dimensions. The sections that follow suggest critical objectives within each dimension that will provide a clear picture of the condition of your team and help you focus your efforts on critical skills that can improve your team’s prognosis. Each objective includes possible KPIs that range from simple yes/no answers to longer-term data gathering and analysis.
A healthy organization takes the time to identify the needs and scope of all its projects. It is deliberate in its actions rather than reactionary. It knows what is currently on the docket and what is on the horizon, and it is staffed to handle the workload.
|Objective:||Department staffing and budget based on projected workload.|
As each planned project begins, additional activities should confirm the allotted schedule and budget before work begins.
|Objective:||Project budget established before writing begins.|
Note that these measurements do not explicitly indicate what should be used to estimate the hours needed to complete a project. Plenty of debate and guidance exists regarding the appropriate methods—be they hours per topic, page, user story, or entire document. You can choose the measurement that works best for you. What is captured in the balanced scorecard, however, is that you have estimated the time required and that the estimate is rooted in the details of the project and your performance history, not on factors outside of your control, such as random engineering dates.
Similarly, your deliverable schedule should be based on the project scope and required budget.
|Objective:||Project schedule established before writing begins.|
The health of your department depends on realistic estimates and schedules. If the scope of the project calls for a larger budget or longer timeline than can be accommodated, scope must be adjusted downward, projects reprioritized, or team members reassigned, rather than crossing your fingers and hoping it works out.
|Objective:||Appropriate resources assigned and available.|
Beyond the overall budgeting and scheduling of the project, however, your balanced scorecard should track planning activities that ultimately affect the execution and quality of the end product. In addition to scoping the project to allow enough time and budget, activities must be in place to ensure that the right content is being produced.
|Objective:||Content plan created.|
A healthy organization meets its obligations according to the plans it has made. In fact, staying on budget and schedule is probably the most common of the dimensions regularly tracked.
|Objective:||Stay on budget and schedule.|
Clearly, this objective requires regular monitoring, which in turn requires methods for reporting, reviewing, and responding to actual data.
|Objective:||Monitor progress on regular basis.|
Note: Yellow KPIs indicate areas of concern to be investigated while red KPIs indicate areas that are in danger of causing the project to fail.
You might also want to use the balanced scorecard to track some of the critical factors that affect your ability to meet your budgets and schedules. You have likely factored certain assumptions about projects, such as amount of reuse or number of review cycles required, into your estimates. It’s a good idea to use your balanced scorecard to track these assumptions. For example, with a move to topic-based DITA authoring, you may have built a business case on avoiding duplicate work and achieving a certain level of reuse. Your balanced scorecard should track your performance on these goals.
In addition, there may be other assumptions that are factored into your project histories. However, if those assumptions change, it could have devastating and unexpected impact if you aren’t tracking them on your balanced scorecard. For example, budgets and schedules frequently get waylaid by excessive amounts of rework. The addition of new staff members may temporarily increase the amount of rework as they learn your standards and processes.
A healthy organization produces content that its customers need and use. Content is complete and accurate, easily accessible, and understandable to all users. If only efficiency is measured in your department, the quality of the information for end-users may suffer. If you measure only whether deadlines are met or how fast content is created, writers may be rewarded for turning in poorly written content that fails to meet user needs, contains inaccuracies, and violates established best practices and standards. A balanced scorecard must weigh not only the efficiencies, but the resulting quality associated with those efficiencies.
A clear indicator of quality in documentation is whether or not your customers can actually use the documentation to complete their tasks or solve their problems.
|Objective:||Customers use documentation to successfully complete their task.|
Note that these KPIs shy away from traditional measurements such as the number of hits or the time spent on a topic. “Hits” are not necessarily “finds,” and spending a lot of time on a topic could just as easily indicate that the content is difficult to decipher than indicating it contains the information users needed. The best measurement of this objective is direct user feedback—surveys, interviews, focus groups, and other customer studies. To effectively track and manage this part of the balanced scorecard, you must have an effective means of gathering user feedback and monitoring user interaction with your content.
Obviously, the corollaries to users being successful in completing a task using your documentation are that the task exists within the documentation, users can find it, it is correct, and it is understandable. By including objectives in these areas, you can shed more light on the reasons that users are, or are not, actually using your documentation. Increasing the scores in these areas should, over time, result in better scores on the first objective in the quality dimension as users learn they can rely on the documentation to have the information they need.
|Objective:||Documentation includes all user tasks required.|
|Objective:||Information can be easily located.|
|Objective:||Content is accurate.|
|Objective:||Content is understandable.|
A healthy organization does not simply rest on its laurels. It seeks to continually improve and build on its skills and performance. Part of that improvement includes investing in the professional development and growth of the individuals within the organization. The growth dimension of the balanced score card begins with a simple evaluation of whether or not staff members have personal development plans and are measured against them.
|Objective:||Individual staff members held accountable for personal development.|
This objective of the growth dimension is unlikely to change. If performance measurements are in place, they will likely stay in place. The rest of the objectives, however, address the data these measurements provide. For example, do you see improvement and growth from all team members year after year?
|Objective:||Staff shows improvement over time on all performance metrics.|
By including this objective, you measure that individuals, not just the group as a whole, improve their efficiency and quality metrics. Without this insight, you might observe improvements in these areas as a group, without realizing that the improvement comes from a few “star” performers who cover for the rest of the team. The loss of such a star, either through attrition or simple exhaustion, could send the department into a downward spiral, with no early warning. Your balanced scorecard reminds you to make certain that team members contribute equally to the success of the team.
The growth dimension should also reveal when individual skills are stagnant. Perhaps team members are solid employees with consistently good reviews, but their jobs are not evolving over time. They are better and faster at doing something that the industry no longer values or needs. A low score in this objective could be a red flag for your future organization. How will you address advancements and changes in the industry if the team is not aware of them and is not adding new skills?
|Objective:||Individuals add new skills to their toolkit each year.|
At an organizational level, growth occurs when innovation and change are part of the culture. It requires more than just putting a change management plan and process in place, but observing whether the process is ever used. Are team members encouraged and eager to innovate or content to do what they’ve always done?
|Objective:||The culture and environment foster innovation and change.|
You might also choose to use the growth dimension of the balanced scorecard to track progress on specific corporate or department initiatives. For example, as you move to topic-based development, away from books, you might want to measure department performance in collaboration and teaming skills.
|Objective:||Staff grows in collaboration and teaming skills.|
In the future, however, as these team skills mature, you might switch your emphasis and tracking to another area, such as content management or writing for reuse.
Forming the Big Picture: Dashboards and Stoplight Charts
Once you have selected the dimensions and objectives you will track, you need to determine how to score and evaluate the measures. In some cases, the KPIs are expressed as percentages that you can easily look up or calculate, but others might require that you define a quantitative score based on a qualitative answer. For example, are yes and no KPIs evaluated at 100 percent for yes and 0 percent for no, or do you need a sliding scale, such as the following?
|Individuals take ownership of team quality and efficiency goals and are motivated to help each other meet those goals.||100||Team members proactively offer assistance to others to meet project goals regardless of their own workload.|
|80||Team members provide assistance when asked by others and they have capacity.|
|50||Team members provide assistance when asked by management.|
|0||Team members work in isolation on individual assignments without worrying about overall project goals.|
Your balanced scorecard must also reflect the relative importance of those dimensions and objectives. The word “balanced” might imply equal distribution—all four dimensions are equally important to your success. On the other hand, just as a human can function with only one kidney, you might decide that one dimension is somewhat less important than the others. In the following example, growth was weighted much less than the other three dimensions. As a result, its low score, while visible at the dimension level, has much less impact to the overall performance measure. The department still has a healthy score of 90, while clearly needing improvement in growth.
Similarly, you can weigh the effect of each objective on the overall dimension score and even the effect of a single KPI on the objective’s score. In the following example, the objective is weighted at 20 percent of the overall dimension, and the first KPI is weighted as twice as important to the objective as the other three. In this case, the weighting of the KPIs results in an overall objective score of 86, and the weighting of the objective results in the 96 percent overall score in the example above.
|Monitor progress on regular basis.||86||20%||Team members report actual hours and progress on each assignment on a weekly basis.||40%||100|
|Team reviews actual hours and progress on a weekly basis.||20%||50|
|Team makes adjustments as needed to reverse the trend of all “yellow” KPIs.||20%||80|
|Team takes immediate action on all “red” KPIs to correct the issue.||20%||100|
It’s important to recognize that a “bad” score in a few KPIs, objectives, or even a dimension, does not necessarily mean your organization is suffering. It only highlights areas where you aren’t at your best. Just as you can do a good job with an earache, your organization can continue to perform well even when certain areas aren’t at peak performance. However, left unchecked, other areas may be affected or a critical mass of poorly met objectives can add up to poor performance overall. Use your balanced scorecard to identify those areas that can use some attention and that can be left to run as is.
To flag problem areas, you need to determine the acceptable thresholds within your organization. While some managers expect 90 percent or greater performance measures, you might be happy with only 80 percent. While some might require immediate attention to anything that falls below 75 percent, you might let some decline to 60 percent. Set thresholds on the values in your balanced scorecard to apply a “stoplight” chart interpretation to your scores. In the following example, the manager is not worried about anything with a score of 90 percent of above, is carefully monitoring objectives with scores of 75-90 percent, and has put action plans in place to improve anything with a score less than 75 percent.
|Project is progressing as planned.||greater than or equal to||90%|
|[Yellow]||Project is trending off plan.||greater than||75%|
|[Red]||Project is at risk and needs attention to put it back on track.||less than||75%|
Although it may take some effort to determine the specific objectives and KPIs that are meaningful to your organization and to ensure you are able to evaluate those KPIs, the balanced scorecard is an invaluable tool for managers of information-development groups. It transforms your goals and strategies into an easily interpreted action plan that can help keep your group functioning at its best.
About the Autor:
Comtech Services, Inc.
Dawn Stevens is a Senior Consultant specializing in information development, instructional design, and management consulting. With over 20 years of experience, including 10 years at Comtech, Dawn has practical experience in virtually every role within a documentation and training department, including project management, instructional design, writing, editing, and multimedia programming. With both engineering and technical communication degrees, Dawn combines a solid technical foundation with strong writing and design skills to identify and remove the challenges her clients face in producing usable, technical information and training. Dawn has a keen interest in metrics, editorial and design standards, user and task analysis, and single sourcing strategies. She enjoys helping our clients gain efficiencies through improved publication processes, minimalist writing techniques, and designing for reusability, with an emphasis on XML and DITA solutions.