Technical Communications and Customer Support: Partnering to Publish What Customers Want to Know
Most customers do not provide direct feedback on product documentation. Instead, when documentation fails to provide the information that a customer needs to use a tool effectively, he or she calls Customer Support for advice. To find out what information was missing or incorrect in our product documentation, I analyzed the Cadence Customer Support call logs that pertained to my products to find out what questions customers ask most about each product. I then partnered with teams of applications engineers (AEs) to improve our documentation by answering common questions, both on the Web in FAQ documents and in product manuals.
Project Definition and Objectives
The purpose of this project was to create a system of mutual feedback and support between the Technical Publications and Customer Support organizations of Cadence Design Systems. By using this new system, the two organizations hoped to improve Cadence product documentation and lower support costs by tailoring the content of product documentation to answer common customer questions clearly and by giving customers easy access to the answers they requested most in the form of product FAQ documents.
Because each organization has a unique set of resources at its disposal, each organization had its own set of objectives in addition to the joint objective described above.
Customer Support Objectives
Within the new Support/Technical Publications partnership, the Support organization agreed to provide the following to the Technical Publications organization:
- Access to the Customer Support call logs.
Writers and editors can now use the data in the Customer Support database to determine the most common questions that customers have about their products. Writers can then address those problems in their product documentation and in separate product FAQs published on the Web, ensuring that customers always have quick access to the information they request most often.
- Answers to some of the most common customer questions.
After writers and editors analyze the data in the Customer Support database to find the most common customer questions, AEs provide the technical expertise required to answer those questions as completely as possible, both in product documentation and in product FAQs.
- Examples of product use for inclusion in documentation.
AEs traditionally write many examples for their own use in answering customer questions. Customers have told us repeatedly in various surveys that they want to see more examples in our documentation. Therefore, AEs agreed to let writers use AE examples in user manuals and online help.
Technical Publications Objectives
Within the new Support/Technical Publications partnership, the Technical Publications organization agreed to provide the following to the Support organization:
- Analysis of the most recent cases in the Customer Support database.
Finding out which questions customers ask most benefits AEs just as much as it does writers and editors. Our analysis of the Customer Support Call Database provides AEs with a list of topics on which they must have immediate information ready at all times.
- Creation and maintenance of product FAQ documents on Sourcelink, the Customer Support web site.
Analysis of the Customer Support Call Database results in a list of the most frequently asked questions for a specific time period for each product. We want to make the answers to FAQs easily accessible to customers, but although AEs have the knowledge required to answer these questions, they do not have the resources or writing skills to create and maintain documentation of product FAQs. Technical Publications therefore writes up the FAQ answers that AEs provide and publishes them on the Web, updating the pages with new questions (from new call analyses) every two months.
The First Step: Understanding the Customer Support Organization
Before we could move toward any common goals, the Technical Publications organization had to understand how the Support system functioned and how it gathered and organized data. When I first met with Support personnel, I found out that at Cadence, the Customer Support system has two basic functions:
- Answer customer questions and solve customer problems as quickly as possible.
Report bugs that customers discover to the appropriate R&D organization (Product Development, Technical Publications, or Educational Services), so that the problem can be addressed in the next product release.
The process by which Support achieves these two goals is similar to the one described by John Muehleisen in his paper on searching for usability data in a customer support database (1). The only difference between the Cadence Support system and the one at Muehleisen’s company is that our system has three separate tiers of applications engineers, who have the following functions:
- Tier 1: Monitor the Customer Support hotline. Log customer calls in the Customer Support database. Answer simple questions as quickly as possible and record those answers. File Product Change Requests (PCRs) to inform R&D of obvious problems with product functionality.
- Tier 2: Take responsibility for cases that require research and testing to identify and solve. Record solutions to customer problems in the Support database and file PCRs as necessary for the problems to which they are assigned.
- Tier 3: Act as Customer Support subject matter experts for specific products. Take responsibility for cases that Tier 2 AEs cannot solve on their own. Record solutions and file PCRs as necessary.
Despite the efficiency of this process in answering current customer questions and informing R&D when things do not work, this system is similar to the one at Muehleisen’s company in a negative way. The system does not identify the customer questions that indicate possible usability or documentation problems (1). In fact, our writers have no idea if a customer even reads the documentation before calling Support unless the customer specifically complains that the documentation is inaccurate or incomplete.
The Next Step:
Creating a Partnership
Once we knew how the Support organization functioned, the Technical Publications organization came up with a plan to utilize the AEs’ technical know-how and Customer Support database and the writers’ and editors’ organization and communication skills. In partnership with Support, we created two main projects for our joint organization:
- The creation of product FAQs that change bimonthly to answer the questions most commonly asked on the Customer Support Hotline.
- The inclusion of real-world examples created by AEs in product documentation.
Creating Product FAQs
To determine which information Cadence customers need most in their documentation, the editor for the documentation for each product group reads each case that AEs see over a two-month period and notes the topics that were discussed in the greatest number of cases.
Once the most common questions are listed for a two-month period, AEs apply their technical expertise to answer all of the questions in the FAQ list. The editor for that product group then writes the answers to these FAQs and publishes them on Sourcelink. Finally, the editor works with each writer in his or her group to add any new information on the FAQ pages into the appropriate sections of the product documentation for the next release.
The analysis of the cases in the Customer Support database, then, gives customers the advantage of getting all of the information that they most need on one web page. The analysis also ensures that the product documentation that customers receive is always updated to include the information in the FAQs-the information that customers specifically request.
Including AE Examples in Product Documentation
When a customer calls Support to ask a question, AEs often create real-world examples, usually using the design with which the customer reported seeing the problem, to demonstrate the answer to the customer’s question. These examples are stored in the Customer Support Call Database with all of the other call data, but most of the time, only the customer who makes a call gets to see the specific example that answers his or her question.
Meanwhile, in surveys that Technical Publications have conducted each year, customers have repeatedly asked for more real-world examples in our documentation. Until last year, however, our writers have always asked R&D to come up with examples if they felt examples were necessary. Because R&D has typically had little time to spare to create examples, our documentation has always been lacking the extra examples customers requested.
When I first began to analyze the Customer Support Call Database, I found all of the examples that AEs had been creating over the years. I immediately talked to the AEs for my products about the examples I saw in their call records, and they were more than happy to let my writers read and publish them in documentation. The AEs reasoned that if more general real-world examples appeared in documentation, then they would not have to create as many customer-specific examples to answer questions that came in, and they would not get as many calls asking for examples in the first place. This project would thereby save Support time on each call and lower call volume, and it would give Technical Publications the chance to give customers the examples they had requested.
Pilot Projects: Verilog Product FAQs and Verifault-SL Examples
Cadence Design Systems received 46,949 Customer Support Hotline calls in 1999 for a total of 20 different product families, each of which encompasses between two and six different products. Because this was obviously too much data to start analyzing at once, we decided to create pilot versions of both the FAQ project and the examples project described above.
FAQ Pilot: The Verilog Product FAQ Pages
To create the pilot version of the FAQ project, I decided to work with the AEs who handle calls regarding the Cadence Verilog product family and initially create FAQ pages only for those products. I chose to work with the Verilog products for two reasons:
- I edit or have edited the documentation for all six Verilog products: VPI, PLI, Affirma NC Verilog simulator, Verilog-XL simulator, SDF Annotator, and Affirma SimVision analysis environment. This means that I am familiar with the content and organization of all six product manuals, and I can work closely with the writers assigned to these manuals to document the answers to the most commonly asked questions in the product manuals.
- Verilog product users make up one of Cadence’s largest customer bases and therefore, the Verilog Product FAQs can help a large number of Cadence customers for a small amount of manpower.
Limiting the Number of Calls in Each Analysis
After I decided to create FAQs for the Verilog product family and met with the Verilog AEs to discuss the project, I quickly discovered that I had to limit the time period for which I would analyze the Verilog calls and update the Verilog FAQs. The Verilog Customer Support Call Database logs stretch back at least five years, and the Verilog product AEs get about 2,700 calls each year! I finally decided to analyze only the Verilog calls that came in over each two-month period. This created a good representation of the most current list of FAQs for each Verilog product. (New hotfixes for our products come out about that often.) By limiting the time period for each analysis in this way, I ensured that every two months I only had to analyze about 450 call logs to create and update the Verilog Product FAQs.
Analyzing the Customer Support Call Logs
The process by which I analyzed the calls in the Customer Support Call Database to create the current list of FAQs for each Verilog product was simple, but time-consuming. I first asked one Verilog AE to query the Customer Support Call Database to get a list of the Verilog cases that occurred over the past two months. These cases were listed by number. I then looked up the record associated with each case number in the Database and analyzed and organized the call logs according to their content, as follows:
1 I grouped all of the calls for each of the six Verilog products together. Each Verilog product would get one FAQ web page on Sourcelink.
2 I removed from consideration all cases that were about software bugs. These cases do not involve answers to questions. They involve solutions to problems, and they are addressed without by filing and fixing PCRs or by documenting them in the Known Problems and Solutions documents that we publish with each product release.
3 I summarized the basic question that each remaining case addressed. This part of the analysis required the most time and effort on my part, as often the problem that the customer reported was not the real question that the AE had to answer.
For example, a user can report that he or she keeps getting an error message when he or she runs a simulation of a design. As I read through the case history and see the AE’s answer to the complaint, I may find out that the error message is too cryptic for the user to understand, or I may discover that the error message happens because the user is specifying a command incorrectly because he or she did not understand the command syntax. So the same reported problem-an error message-can indicate a question about the message or the command. I have to read the entire case history to really understand it.
4 I grouped cases that explained one basic subject or answered one basic question. For example, I grouped all of the cases that asked questions about Verilog path delays and their syntax.
5 I counted all of the cases in each topic/question group. The questions that had the most cases associated with them were listed as questions for the Verilog Product FAQs.
Creating the Verilog Product FAQ Pages
After I finished my analysis of the Verilog Customer Support Call Logs for the past two months, I had a list of the most frequently asked questions for each of five Verilog products and for a general FAQ page that covered all of the Verilog products. (There were not enough Affirma SimVision analysis environment cases to justify a separate FAQ for that product.) I took this list of questions to the Verilog AE team meeting and let them answer them. Then I wrote up the answers, published them on Sourcelink, and connected the resulting pages to Web Trends, a program that monitors the use of specific Sourcelink pages and provides statistics such as number of visitors, average length of stay, and even names of visitors’ companies for a given time period. The results that Web Trends reported for the Verilog FAQs appear in the Results section below.
Examples Pilot: Verifault-XL Code Examples
When I met with the Cadence AEs to discuss the projects that Technical Publications wanted to work on with Support, I suggested Verilog as the pilot FAQ Project area. The AEs suggested Verifault-XL as the pilot Examples Project area.
At this time, Verifault-XL has a brand new utility that comes with a new set of commands, new system tasks to add to design code, and even a new use model for the Verifault product as a whole. Given the complexity of this new functionality, the chapter in the Verifault-XL User Guide that describes the new utility really needs examples for each of the new system tasks and commands that come with the new utility and a comprehensive example that illustrates the new use model for the product.
Unfortunately, when the new Verifault-XL utility was first released last year, R&D did not have time to create new examples, so none were added to the documentation. Consequently, after the product was released with the new utility, the Verifault-XL call volume greatly increased. Customers did not understand the new utility because there were no examples that showed how to use it.
To save time when handling all of the calls about this new utility, the Tier 3 Verifault-XL AE wrote a comprehensive set of code examples to show the new functionality. The AE recorded this set of examples in the Customer Support Call Database and AEs used it when answering calls on the new utility. I found out about the example when I talked to the AEs about adding their examples to our documentation. Though it had existed for two months, only customers who called Support asking about the utility had access to it.
After my initial meeting with the AEs, the Verifault-XL writer worked with the Tier 3 Verifault-XL AE to include the examples in the product manual. Consequently, customers got the examples that they had been requesting from Support without calling Support, thus lowering Verifault-XL Support call volume.
Results: Customers Get Information by Request
As a result of the Cadence Technical Publications and Customer Support Partnership, Verilog customers can now get the information that they want the most. This is because every time a customer calls Support, he or she contributes to three improvements in the information to which he or she has access.
- An FAQ page now exists for each Verilog product for the Verilog product family as a whole. Every two months, the cases that have been recorded during that time period are analyzed, and each FAQ page is updated with a new set of common questions.
- The information on each Verilog FAQ page has been used to update the documentation for each Verilog product.
- AEs have begun to create the examples that customers request, and writers have begun to include those examples in their product documentation.
Results of the Verilog FAQ Project:
Questions Answered by Request
Because of the new Verilog FAQ pages, Verilog customers can now get the information that they used to request from Support without calling Support. Both AEs and customers save time and money because customers now have quick, free access to the answers that they need most.
Because the FAQ pages represent only a small piece of the new documentation for the Verilog products, call volume analysis cannot truly show the effect that the Verilog FAQs have on customer call volume.
However, customers have expressed their approval of the new Verilog Product FAQs in the simplest possible way-they use the pages! According to our Web Trends statistics, despite a complete lack of advertising for the new documents, in their first month of operation, the Verilog FAQs got 283 visits averaging about three minutes per visit. And these visitors were not Cadence AEs, who would be expected to visit the pages to see the results of their labors. The Web Trends reports state that for Q4 1999, the top five visitors to the FAQ pages were (in descending order): NEC Japan, Motorola, Sony, Hitachi, and IBM-five of Cadence’s most important customers.
Now, in June 2000, it is clear that the Verilog FAQ pages are going to remain a popular feature on Sourcelink. The pages have now received about 300 visits per quarter for the past three quarters, and the top five visitors for Q2 2000 are still important Cadence customers. The top five visitors in Q2 2000 are (in descending order): LSI Logic Corp., IBM, Mitsubishi, Sonoma Interconnect, and National Semiconductor Corp., and at the beginning of June 2000 (four weeks before the end of Q2 2000), we had had about 200 visitors during Q2.
Clearly, our customers remain pleased with our Verilog FAQ pages, as we continue to revise and add information to the pages. In addition to reading the FAQ pages that we publish today, customers have requested new FAQ topics using the “Suggest a Question” links on each page, and our subsequent revisions of the FAQs have included some of those questions and topics.
Results of Examples Project:
Examples Published by Request
While the Verilog Product FAQs represent a small piece of the new documentation for the Verilog products, the new Verifault-XL examples, released in August and September of 1999, represent the only change that Cadence made to either the documentation or the product itself for that release. Therefore, unlike the FAQ Project, you can clearly see the effects of the Examples Project on the Verifault-XL call volume data.
In June of 1999, before the release of the new examples described above, the Verifault-XL call volume was 22 calls per month. By October, the month after the last customer shipment of the new release, the call volume was only 10 calls per month. The November and December call volumes were 11 and 9, respectively. Because the new examples represented the only change that was made to the Verifault-XL product or to its documentation in 1999, it seems safe to say that the only thing that could have affected the Verifault-XL call volume this dramatically is the only change in the product-the examples. The Verifault-XL examples, then, have apparently lowered the call volume for that product by 50 percent-a significant improvement.
Going Forward: FAQs and Examples for All Cadence Products
Because the Verilog and Verifault-XL pilot projects have gone so well, Cadence has decided to expand the Technical Publications/Support Partnership effort across all product groups. The Customer Support Call Log Analysis Process is now being presented to all of our writing groups, and new FAQs are planned for 2000, starting with FAQs for those product groups that have the highest call volumes. The AEs have also begun to suggest new uses for their examples, and these suggestions are being communicated to our writers for publication in the next release.
Overall, our documentation is being improved throughout Cadence, and our customers are being given the tools to answer their more common questions independently, without the help of the Support Hotline. Support personnel will soon be able to focus on more unique and challenging questions and customers will be able to access the most necessary information immediately, without waiting for Support to respond to their questions. In short, Cadence AEs and customers alike will be able to get the information that they need and get back to work more quickly than ever before.