Making Quality Explicit: The Leszek Method
Most technical writers try very hard to be perfect. But the notion that any person can ever be perfect can prevent us from looking for rigorous, repeatable, lightweight techniques for enforcing quality. The Director of Technical Publications at salesforce.com, Andrea Leszek, has designed an eight-point system for enforcing quality at every stage of a writer’s work. The system assumes we aren’t perfect and provides lightweight processes that are compatible with structured authoring and Agile environments.
Why Do Mistakes Happen?
To put it quite simply, to err really is human.
University of California San Francisco Chief of Medical Services, Dr. Robert M. Wachter, concludes in his book Internal Bleeding: “…despite an unacceptably high rate of medical errors, most mistakes were the result of bad systems, not bad people.”
If mistakes are inevitable for medical doctors, they are probably inevitable for technical writers, too. In eighteen years of dedication to producing the highest quality documents possible, I have released documents with missing topics, portions of a sentence missing, and worst of all, information that was just plain wrong. If you are still reading, I assume you have made a mistake or two in your day as well.
How Can We Stop Mistakes?
Over the course of several years, Andrea Leszek, Director of Publications at salesforce.com, developed a series of checkpoints and procedures designed to catch the most common technical writing mistakes, or to prevent them entirely. This is analogous to hospital checkpoint procedures such as asking the patient for name and date of birth before any interaction with hospital personnel.
The processes are lightweight and can be implemented by the busiest departments. These processes can be implemented in stages. Andrea created them in a waterfall software development environment. When we shifted to Agile, the system worked with few changes, because it already was Agile. Checking for errors every step of the way is one way to prevent “error debt” from building up.
The Leszek Method
Andrea’s system has eight components:
- Stop trying to be perfect. It just gets in the way of finding and fixing defects.
- Go after technical reviews like a pit bull.
- Edit every word.
- Review every check-in against a list of common errors, ideally with another writer.
- Test all check-ins and blitz all new material at key milestones.
- Put all documentation samples in functional tests.
- Ask for user feedback on every page (for web-delivered documents).
- Establish a culture of encouragement: If we’re doomed to make mistakes, then we can at least get really good at finding them before we ship them.
Some writers, including this author, believed that there wasn’t time for this kind of double-checking, that it just kept a writer from finishing a task. However, the third or fourth time these systems caught a really dumb error, like checking into the wrong branch or failing to update the copyright for a new year, all the writers became enthusiastic converts to the method.
Stop Trying to Be Perfect
Hopefully you are already convinced of this.
Go After Technical Reviews
Most technical writers are already adept at this, whether they imitate a pit bull, a shy suitor, or Grandma with cookies and chocolates. Unfortunately, this is where many documentation groups’ efforts at quality stop.
Edit Every Word
This technique yields direct, measurable, and powerful results. For any company with two or more writers, simply assign each writer an editor, and use writing tools that allow you to identify changes, called change-tracking. Even a single letter or number should be reviewed by the assigned editor or a substitute if the editor is unavailable. It’s remarkable how often we find both simple and subtle errors simply by having another set of eyes look at a file.
Sometimes writers start to slip. They check in small changes that haven’t been edited, and as a consequence, small errors start to creep in. Then the writer remembers why editors are assigned to each writer, and he or she goes back to following the process. The editor is considered as responsible for the content checked in as the writer. Each writer spends between one and four hours a week on editing. It’s a small time cost for a huge error-prevention benefit.
There’s another benefit—whenever a writer is playing the editor role, she or he automatically tunes up his own writing. Bad habits fall away before they can take root.
We are all tempted to skip this step when deadlines loom, but crunch times are when mistakes are most likely to happen.
Review Every Check-in with Another Writer
This process assumes you use source control or a content management tool to protect your files. If you don’t, you should before disaster strikes.
Once your editor has accepted your changes, build the doc and correct any errors, such as reports of broken links. Even if those errors aren’t in documents that you are primarily responsible for, fix the error or file a bug for the responsible writer to fix it. By preventing buildup of errors in the documentation, you are never in a state of having to untangle multiple problems that may mask each other.
Create a checklist for checking in, for example:
- Show the output from your test build and the build log to verify it is recent (and that, therefore, no one has checked in a change to the same file before you that clashes with your change).
- Check that the files you are checking in are from the correct branch of the codeline. For example, your company might have one branch for the current release, and another branch of the codeline for the next release. If you sometimes work in more than one branch at the same time, this check is a lifesaver!
- If you are deleting a file or some content in a single-source environment, verify that no other docs will break because of your change.
The check-in reviewer goes through each item with the writer. Both writer and reviewer verify the information. It never takes more than a minute or two, and it’s remarkable how often an error is caught at this stage. You can create your own checklist by thinking about the most common defects that occur in your group. You can add to the list or remove items that have been resolved by some other process change or other improvement.
What do you do if you are working until 2:00 AM, and there’s no one to verify your check-in? Don’t check it in! Writers often want desperately to feel “done” at the end of a marathon, but there’s almost never a business reason to take this risk.
If a writer is frequently working late at night with no one to help with check-ins, it’s likely that writer needs some kind of coaching or a smaller writing burden. Having this process in place can help highlight such issues so they can be resolved.
What to do if you are a lone writer? Make friends with a QA person or a developer, and offer to trade check-ins. You’ll never regret it.
Testing and Blitzes at Key Milestones
Most technical writers already test their documentation against alpha or beta versions of the software. The next step in testing is the blitz. We borrowed the word from “blitzkrieg.” During a blitz, everybody in the documentation group stops what they are doing for half a day and hunts as hard as they can for defects.
Writers and editors are not assigned their own documentation. All writers should participate.
The type of error we search for, and how much of the documentation we search in, depends on the milestone:
- At the end of every sprint (or, for waterfall environments, once a month), a manager collects a list of all documentation that has changed and assigns each writer a set of pages to review for content issues.
- For a “release sprint” (Agile) or during the end game (waterfall), every page of every document is briefly viewed, looking for formatting errors that your team has identified as frequently occurring. For example, you may have a lot of bad line wraps because sample code is too long. Schedule half a day for this activity.
By formalizing the blitz process, you acknowledge that errors do occur. Nobody is blamed for being less than perfect, and defects are found and fixed before a customer ever sees the document.
Blitzes should be scheduled a few days to a week or so before the key milestone, so that errors that are found can be corrected before the milestone passes.
An unexpected benefit of this process is that if the writers do start skipping processes, the errors show up in a blitz as a pattern. For example, if you change from 10 or so bugs per person to 20 or more, and they are trivial errors, then there’s probably something wrong with your editing process (oh, and you found dozens of errors before the customers did!). Analysis and correction are easier to do once you know a problem exists.
Blitzes do take a few hours to organize, and half a day from every writer on the team. But we always find at least 100 defects in each blitz, from formatting errors to incorrect information.
All Documentation Samples Tested
This testing was one of the last processes we implemented, after complaints about errors cropping up in samples. It took just a few hours for our quality assurance team to define a set of tags that we apply to all samples. During functional testing, which runs daily, the samples are run and if errors occur, a bug is filed and the writer can follow up before the document is released to users.
This technique requires cooperation from your quality assurance team, but they are usually very happy to help set up a system with you.
Add User Feedback to Every Page (Web Content)
It’s not that hard to add a feedback page to the bottom of every page of web content that you build. When we first added the forms, internal customers used them as often as external users, because it was easier to click the form than to send an email or file a bug. We have not found that any user abuses the form. We file a case for each valid issue (yes, sometimes people really have not read the manual before clicking the feedback link), and collect trends, such as customers asking for a type of sample we don’t provide.
We used about 40 hours of a development intern’s time to create the form, and we maintain it ourselves. We review email comments weekly, to avoid building up a backlog of defects.
Culture of Encouragement
When you foster a culture of encouragement, writers look harder for defects in their own work and in the work of others. Instead of feeling like hiding mistakes, because there is no blame (remember, bad processes, not bad people, are responsible for mistakes), writers feel safe to acknowledge defects and participate more fully in trying to find ways to prevent or catch them before a documentation release.
Implementing the Leszek Method
Some key points to remember when you implement the Leszek Method:
- You can implement one process at a time.
- Tailor each process for the typical defects in your group.
- Emphasize that these lightweight processes must be used rigorously to be effective.
- Add your own processes as you discover patterns of defects.
About the Author
Mysti Berry, M.F.A., is an award-winning technical writer with a B.A. in linguistics from UC Santa Cruz and an M.F.A. in writing from University of San Francisco. She has more than 18 years of experience designing and developing technical information for companies as diverse as Oracle and 20th Century Fox. Berry currently works for a cutting-edge software company in the SaaS (software as a service) space.
Robert M. Wachter and Kaveh Shojania
Internal Bleeding: The Truth Behind America’s Terrifying Epidemic of Medical Mistakes
2005, New York, NY