Results of the 2012 Best Practices Research Needs Survey

Home/Publications/CIDM Matters / eNews/Information Management News 04.13/Results of the 2012 Best Practices Research Needs Survey

Rebekka Andersen, University of California, Davis

This article reports the results of the 2012 Best Practices Conference research needs survey and describes a new CIDM research initiative. You can participate in the initiative by engaging in the activities detailed below and by completing the follow-up research priorities survey available here. Look for “The Value of a Reciprocal Relationship Between Research and Practice” in next month’s May e-newsletter for a discussion of research approaches and design possibilities and strategies for overcoming empirical research challenges.


What are the research needs of the information development community and how might academic researchers and professional communicators collaborate to better meet those needs?

This was a central question at the 2012 Best Practices Conference, which introduced the panel, “The Researchers, Educators, and Communicators Roundtable: Forming a New Habit of Collaboration.” The panel brought together researchers, educators, and communicators to discuss various challenges information development organizations are facing and how new research data could help managers better address those challenges.1

Pat Burrows of EMC opened the panel by identifying some of the challenges of today’s organizations, including:

  • Managing continually changing requirements and technologies
  • Moving from simple, file-based development to complex, single-sourced, multiple outputs in a technical infrastructure
  • Publishing to multiple formats and managing real-time updates and dynamic deliverables
  • Understanding customer requirements and issues
  • Developing useful metrics and analytics
  • Training information developers to transition into new roles requiring different skill sets
  • Finding qualified candidates
  • Managing global and remote teams

The problem with how we have been meeting these challenges, she said, is that we have been largely following our usual habits. We gather limited customer input when possible, follow social media, review white papers, react to demands that come down from management, and read extensively on any number of tools and/or methodologies. These habits often result in compiling lots of information, sometimes anecdotal or subjective, and relying on processes that require planning and re-planning, training and re-training, and organizing and reorganizing. Because sound research data to guide decision making has been difficult to find, Burrows pointed out, we tend to rely on old habits to address today’s information development challenges, even if those habits are from a different publishing era.

One outcome of the Best Practices research panel discussion was an informal survey distributed by JoAnn Hackos that asked conference participants to respond to the question, “What topics would you appreciate having research data for?” Forty eight completed surveys were received. It is important to note that survey responses were limited to those who attended Best Practices.

Sid Benavente of Microsoft Corporation has developed a follow-up survey that seeks input from the greater CIDM community on the kinds of research projects that would add value to your organizations. You can complete the survey here.

Below, I report the results of the Best Practices research survey and describe the academic/industry collaboration research initiative that the CIDM has established to continue the conversation begun at the Best Practices conference and to support new research projects.

Survey Results

The informal survey distributed to Best Practices participants asked one open-ended question: What topics would you appreciate having research data for? Participants submitted completed surveys to the CIDM conference coordinators, who then compiled responses in their raw form into one document.

Table 1 lists the research topics that emerged from my review of survey responses, and it presents a sampling of the kinds of research questions respondents would like to see new research projects address. Topics are listed in order of respondent interest; those receiving the most attention are presented first. Alongside each topic is a sample of research questions that respondents would like to see new research studies address. For ease of reading and understanding, I edited questions and responses that were not phrased as complete ideas.

Survey results point to the need for studies that fall into the following categories: user behavior, process/practice, content strategy, metrics/measurement, technical communicator roles, training, value proposition, and tool comparison. Other categories that were mentioned but received less attention include information developer behavior and the future of technical communication (See Table 1).

Research Topic Associated Research Questions
User Behavior Studies
  • We write to people, not machines. We need more research on the human side. How are people using the information? How do they feel about it? What is useful and not useful?
  • What are users thinking when they read the content? Why are they accessing it? Is it useful? Why or why not? Why are people accessing content two or more times? Why are they leaving pages? Did they leave these pages more often because their questions were answered well or because the content caused them to give up?
  • What information do users actually use? What information is not needed? We need to get away from the idea that everything needs to be complete. We don’t have any studies that tell us what info can be cut. Lots of guesses. Do we need to explain the OK button four times?
  • How do end users react to problems and frustrations? What are their habits—pick up the phone, read documentation, Google for an answer? How are their habits changing? And how can we use this knowledge to intervene in and reduce or eliminate customer pain?
  • What output types (video, document, wizard) work best for given user groups? What type of information/content would be more appropriate in a portal than an e-book? When is video documentation preferred over traditional text-based documentation?
  • How do generation differences in users influence how they access and use information?
  • How does a story contribute to the effectiveness of technical content? Sometimes a coherent narrative gets lost in the pursuit of minimalism.
  • What makes users notice warning and caution statements? These are required by the FDA in medical device instructions, yet the only studies published are for tractor manuals. How can we write useful warnings and ensure that users notice them?
  • How effective are videos in user assistance and what percent of people prefer video for instructions and for learning?
  • How do software developers and users really interact with organizations using social media? Can this type of collaboration support improved quality of content? Can this type of collaboration scare or work with small pubs teams?
Process/Practice Studies Agile Development

  • What tenets of Agile are working well, and what tenets are wasteful? How can we best integrate information development into an Agile environment?
  • What is the difference in required amount of time a writer spends on writing content for a feature in a traditional waterfall development process vs. an Agile environment?
  • What changes do we need to make in the entire organization to succeed in producing quality documentation in an Agile environment?
  • How can content development processes be adapted to efficiently integrate with Agile development processes

Impact of Cultures/Industries

  • How do information development processes differ in different industries, such as software, hardware, medical, and regulated industries? How do these processes differ among industries with publicly available documentation vs. those with highly proprietary documentation with controlled access?
  • What are the variations in department practices and behaviors region by region and/or industry by industry? Are there variations in practices attributable to where a department sits in the organization (engineering, support, shared services)?
  • How do we promote and help establish the technical communication profession in new geographies, such as China and India? What is the state of the profession in those regions? What challenges do information development teams in new geographies experience?
  • What are the effects of different corporate cultures on information development processes?
  • How do changes in information delivery by large organizations impact strategic planning for small companies?
  • What changes in organizational behavior result when the organization is not necessarily the authoritative voice on its products anymore?
  • In what kinds of contexts has enterprise-wide adoption of DITA been effective? What factors contribute to adoption success?
  • How do changes in information delivery by large organizations impact strategic planning for small companies?

Best Practices

  • What are best practices for writing for shrinking mobile devices? What are best practices for developing on-product help for touch screen products?
  • What information needs to go in the product and what information needs to go in the documentation? What redundancies can be cut out? What information is most effective as embedded content and what information is most effective as documentation?
  • How can we best write for users with limited literacy? How can we best write for different age groups?
  • What are the best strategies for implementing scenario-based content?
  • What are best practices for authoring multiple versions of the same product concurrently in a content management system?
  • How do we best stream media as part of information development?
  • How can information development teams effectively support developer ecosystems (code samples, developer guides, levels of expertise)?
  • What quality assurance practices are most effective?
  • What strategies for getting documentation team members involved in software interface design (moving to the front of process from the end of process) have proven most effective?
Content Strategy Studies
  • What strategies have organizations adopted for implementing content management processes that provide optimal balance between facilitating reuse and accommodating the need to version content to support multiple concurrent releases?
  • What strategies have organizations adopted for implementing a continuous update model without drastically increasing localization cost and process complexity?
  • What are best practices for transitioning the translation review process in an XML/DITA environment?
  • What media are best for what kinds of user situations?
  • What will be the next generation content architecture using scenario-based content, OLH, and user experience (UI) design? What will be the most effective combination?
  • What strategies work best for building an information architecture for a large documentation set?
  • Where does content management (CM) breakdown and why? How do we best plan for complexity management—for managing users, authors, content managers, and CM solution architects? What impact do these management decisions have on usability and sustainability?
  • What are best practices for integrating content from companies, customers, and communities? For organizations just getting started, what are the best places to start? For example, do we start first by listening for unmet customer needs and/or determining what types of information are the most valuable to users? What activities should be outsourced? When does it make sense to outsource activities vs. keep them in-house?
Metrics/Measurement Studies
  • How do departments evaluate their own success? How do their companies evaluate the success of information development departments?
  • What metrics have proven useful for measuring optimal resource levels for content development teams?
  • Are there quantifiable differences in information quality as a result of Agile development?
  • How has the shift to collaborative authoring affected the quality of content?
  • How can we quantify the value added by technical content?
  • How can we best measure the productivity and morale of remote writers/teams vs onsite writers/teams?
  • How can we quantify the value of an editor? How can we best measure editing activities (style and usage guideline development, terminology management)?
  • How many topics can a writer effectively and accurately develop per day? How many words? What percentage of reuse can we measure?
  • What is the optimum ratio of product developers to information developers?
  • What metrics have proven useful for measuring content quality and accuracy?
  • What metrics can we use for identifying un-used information?
  • How can we measure content reuse in our organizations (chapters, topics/pages, sections, tables, lists, figures, paragraphs, in line elements)?
  • How can we measure the production impact of technology and tools, such as XML, DITA, a content management system, standardization?
  • Is there a good method for measuring the deflection of support calls?
Technical Communicator Roles Studies
  • How significant is the gap between technical writing graduates and entry-level content developer requirements? How do we best handle this gap?
  • What are the different roles of today’s technical communicators and what new job classifications best represent these roles? How can we get organizations such as Radford to adopt these classifications?
  • How can we best develop new job descriptions based on the different roles of today’s technical communicators (production specialist, DITA/XML architects, CMS administrator, content developer)? How can we ensure wide adoption of these descriptions?
  • How have different organizations redefined team roles and job titles? What factors influenced the creation of these new roles and titles?
  • What are best practices for staffing information development? Are ratios for staffing or resource planning reliable and/or valuable?
  • What does the work environment of today’s technical communicator look like? Are technical communicators generally happy in working in these environments? What is the turnover rate for technical communicators in an organization and what accounts for this rate?
  • What is the effect of having a well-defined, formal, information architect role?
  • How much time do content developers actually spend writing content?
Training Studies
  • What skills and knowledge are we teaching that are actually helping students get hired and that are actually useful on the job?
  • What are effective training strategies for transitioning to topic-based, task-based documentation?
  • How do technical communicators learn? Does a specific educational background determine learning behaviors/processes?
  • Can we apply a more scientific educational methodology to our field?
Value Proposition Studies
  • How can we best understand our value and articulate that value to stakeholders? What does our value proposition need to reflect?
  • How can we translate the value technical publication organizations bring to their organization into the return on investment language that will gain the attention of corporate decision makers?
Tool Comparison Studies
  • How do different content technologies compare (content management systems, translation management systems, authoring tools, and output production options)? What are the potential advantages and disadvantages of the different features and capabilities of these technologies?
  • What kinds of quantitative and qualitative data exist that could help organizations make decisions at different stages of the tool evaluation, implementation, and adoption process? For example, for an organization that needs to select a translation management system, what data is available on product options, requirements, impacts on staff and process, cost/benefit, balancing, and return on investment?
  • What are best practices for evaluating content technologies? What criteria tests and independent benchmarks are available?

Table 1: Summary of Survey Results

The research questions raised by survey respondents speak to the need for large quantitative and qualitative studies that examine specific practices and behaviors across a wide range of users, organizations, and industries. These studies might be supported by one or many organizations, possibly through crowdfunding, and might be conducted in one or many organizations by a research team or individual.

To add value to the larger information development community and to technical communication scholarship, study findings would need to be disseminated through both practitioner and academic venues. Possible venues for researchers include whitepapers, conference presentations, best practices publications, peer-reviewed articles, and book chapters. To add value to individual organizations directly supporting the research, study findings would ideally result in local problem solving and improved local practice. Researchers might work with organizations to offer workshops, consulting services, training, and other opportunities for learning. Any large quantitative or qualitative study, too, would need to be approved by the institutional review board of the researcher; organizations offering site access for data collection might also require the researcher to complete a non-disclosure agreement. These measures would ensure the confidentiality of proprietary information and protect the identity of research subjects, including organizations and individuals.

Participating in the CIDM Research Initiative

You can participate in the CIDM research initiative by engaging in any of the following activities:

  • Joining the LinkedIn subgroup to the CIDM group called Academic/Industry Collaboration. This online community, set up by JoAnn Hackos, provides communication professionals and academics a space to dialogue about research priorities and possibilities, hiring needs and challenges, and other topics for which dialogue could lead to problem solving and productive action. Sample discussion points might include how best to successfully connect researchers with organizations interested in supporting a research project, how best to make a business case for a research project to upper management, or how best to disseminate research results in ways that benefit both communication professionals and academics. If you are interested in contributing to this conversation, please request to join.
  • Completing a follow-up research needs survey. Sid Benavente of Microsoft Corporation developed a follow-up research survey based on the results of the initial research needs survey distributed at Best Practices. You can complete the survey here. The survey seeks your input on research projects and questions of most interest to you.
  • Announcing requests for research proposals on the CIDM website. If your organization has a need for research and is interested in bringing in a researcher to conduct research, write up a request for research proposals and send it to JoAnn Hackos. The CIDM will be creating a new research section on the CIDM website to announce research projects and solicit research proposals. Details on how to coordinate the RFP process and the proposal submission process will be discussed in the LinkedIn subgroup, so please do join the conversation and share your ideas.
  • Contributing to the CIDM’s crowdfunding effort. The CIDM will be announcing research projects that will be funded through a mechanism called crowdfunding. If your organization is interested in contributing to a general fund to support a specific research project of value to the industry, not just to one organization, contact JoAnn; as she states in her March e-newsletter article, project sponsors “get to help determine the direction of the research project and get early and personal access to the results.”

1 Panel members included Pat Burrows of Global Information Management at EMC, Carolyn Rude of Virginia Tech, Dave Clark of the University of Wisconsin-Milwaukee, Bill Hart-Davidson of Michigan State University, and Rebekka Andersen of the University of California, Davis