October 1, 2022
(Multilingual) Content Quality Is No Coincidence!
The current trend of content marketing has had a huge impact on technical content creation. Not only has the ease with which end users can find your content become more important, but companies are increasingly discovering that technical content is just as important a pillar of the overall content strategy as pre-sales “smarketing” content. You therefore need to ensure that customers and prospects are satisfied with the quality of your content. In this article we describe a proven approach for strategic content quality with a special focus on multilingual challenges.
- What is quality?
- Define expectations!
- Cover the basics: terminology and style guides
- Enable collaboration
- Review & evaluate
- Use KPIs for strategic management
What is quality?
We all perceive quality from our own point of view, depending on our context and expectations. More objective criteria can be found in various industrial norms and standards. The ISO 9001 quality standard defines quality as “the degree to which a set of inherent characteristics of an object fulfills requirements”. As we are working in a globalized world, it might be useful to look at a framework for describing and defining custom translation quality metrics, e.g., the Multidimensional Quality Metrics (MQM). According to this model, “a quality translation demonstrates required accuracy and fluency for the audience and purpose and complies with all other negotiated specifications, taking into account end-user needs.”
A strategic and sustainable content quality process, therefore, needs to start with defining expectations.
Quality is a process, not a coincidence
When you set out to define quality expectations, you need to keep in mind that different audiences have different requirements. We recommend:
- Setting up content profiles for each target group, impact, relevance, risk, channel, etc.
- Defining quality criteria for each content profile (ideally between 7 and 11 categories)
To clarify these steps, let us look at a sample content profile for technical documentation. Of course, technical information needs to be correct, so accuracy is key: with three subcategories to be more precise. Each of these is assigned a high penalty score if an error of this kind occurs. Terminology is naturally the starting point as a fundamental prerequisite for quality content. Style, on the other hand, is not as important: flow and fluency are “nice to haves” and thus only receive a minimal penalty.
If we contrast this with marketing content, the importance of style is underscored with three selected subcategories for more detail. Terminology is important as well, not least since potential SEO keywords must be managed – otherwise, you run the risk that your content just does not get found online. Fluency, cultural adaptations, and “wrong” translations are also important. The document structure is less important.
In the next step, it is important to weigh the impact of these quality criteria against the quality of the output, depending on the content profile. You can also divide the quality criteria into “severity” levels, e.g., minor, major, or critical.
Once you have agreed upon these content profiles with your various stakeholders, such as the infamous in-country reviewers, you send content for “quality evaluation” rather than “correction”. This approach forces your stakeholders to think about what they are looking for in the documents, and they evaluate each “change” against the list of quality categories. For example, there might be three terminology issues and two mistranslations in a document of 1500 words.
Our system can now calculate a quality score for each document based on the underlying quality metric. This is done by adding together all the “penalty” points for each error and then weighing the total penalty score against the size of the document. With a sophisticated algorithm, we then convert these points into more “human-readable” percentages, resulting in a quality score of say 89% for a given document.
You should also consider and define:
- Quality thresholds. What percentage is still “good enough” for your intended target audience? Because as we all know, we will rarely achieve 100% quality.
- Steps to take in the event that quality thresholds are not met.
Do you already have a heavy workload? Then here comes some good news: you do not have to check the entire document. Instead, a cleverly generated sample of the text will suffice. As TAUS states in their “Best practices on Sampling”, a 20% sample size provides an acceptable statistical sampling error within a 99.9% level of confidence (for example, a document of 5,000 words and a 1,000 words sample has a sample error of <5% with a 99.9% level of confidence. If the sample contains five errors, then there are 25 errors in the total content plus or minus two errors).
The goal is not to check the entire text, but to determine a quality score, define a pass/fail value, and then sample the text. Only if the quality assessment of the sample fails should the entire file be checked (or other measures taken).
To be on the safe side, you can start with larger samples and track the results over time. Once you get a clear picture about the overall performance of your translation process, you can start to reduce the sample sizes for those languages, content profiles, or suppliers with consistently high quality.
You can also choose between random or parameter-based sampling. We recommend using the Dynamic Quality Framework of TAUS and the Multidimensional Quality Metrics of the EU-sponsored QTLaunchPad project.
Cover the basics: terminology and style guides
In order to start introducing a metrics-based quality assurance system, particularly when also using sampling, you need to cover a few basics. Since you will no longer review the entire content being translated, you need to make sure the outcome is governed by rules which can be adapted and extended as you go.
A translation style guide, therefore, is an important basic requirement. It helps translators with tone of voice, phrasing choices, and grammar decisions. If you draw these up together with your translators and reviewers, then instead of your reviewers constantly having to re-write content, they are helping you by maintaining the style guide and providing a steady stream of feedback.
Choosing the right words is, of course, crucial for any content. The key to using the right words throughout the entire company (and by partner companies such as vendors or marketing agencies) is terminology. Managing terminology can therefore be a major competitive advantage; even more so if you manage terminology in all the languages of your content and not just in the source language. And just like with style guides, if you can involve your translators and reviewers in the definition of your basic terminology, you will not only save a lot of time during review, but you also make them focus their energy on the words that really matter: the corporate terminology.
Agreeing upon the “right words”, in other words, a consistent terminology database with preferred and rejected terms, is not an easy process. It involves many stakeholders and sometimes requires lengthy discussions. Our platform is therefore particularly strong in collaborative terminology processes, ranging from requesting new terms or changes, through collaborative discussion and editing rounds, all the way to final approval by whichever stakeholders need to be involved for certain concepts.
When switching to strategic quality management, particularly with sampling of the reviewed content, you need to make sure any potential issues get addressed and not just “hope” that ambiguous texts are interpreted correctly. This involves ambiguities or mistakes in the source content, as well as decisions affecting the target languages. The best way to do this is to actively encourage translators to ask questions, and then follow up on these questions. After all, no one reads a document as intensely as a translator. And if we share the information gathered from answered translator queries with the team and the content creators, everyone can benefit from this “localization intelligence”.
The same holds true for terminology management. Including every stakeholder in the localization process to suggest, validate, and approve terminology is a fantastic way to make sure terminology is not only complete but also correct and accepted by all stakeholders.
Review & evaluate
With all these safeguards in place, you can safely make the paradigm shift in content reviews: away from “correcting” texts and towards a systematic and sample-based approach. One thing to keep in mind for this process is that as reviewers are normally not translators themselves, the review should be performed using an intuitive, simple tool for non-translation professionals. The tool should display visual or contextual information and it should be possible to change the reference language. Furthermore, the tool should both incorporate terminology databases and be integrated into the translation process itself. And it should allow the evaluator to easily change text or leave comments, and most importantly, assign the changes to the quality categories defined in the content profiles.
Once the review has been completed, the ideal process should see the amended file looped back to the translator to not only provide feedback but also to make sure the reviewer did not inadvertently introduce any errors into the translation. The translator should therefore be able to log in and immediately identify and cross-check the changes made by the reviewer via a convenient view or filter. They can confirm the changes or suggest a new translation. And since this all takes place inside the “bitext” world of CAT tools, the tedious process of incorporating changes is completely eliminated.
After a file has been fully reviewed, a final error score is calculated based on the evaluation. If only a sample was checked, the result is extrapolated to generate a score for the entire document. These scores then allow you to assess the quality of the document and decide how to proceed. More importantly, however, these scores are collected and tracked over time to give us key insights.
Use KPIs for strategic management
You can – and should – track these quality scores in many ways, e.g., per project, content profile, language, translator, and so forth. Tracked over a longer period, these scores (aka KPIs) add up to objective, significant, and valid data that enable you to:
- Strategically manage quality
- Identify trends
- Intervene pro-actively
This step of using KPIs concludes our content quality cycle and is also the foundation for any further steps you should wish to take to ensure your content is a perfect match for your target audience.
Establishing a quality process for your content is beneficial in several ways.
You gain not only from high-quality content that adheres to your company’s approved terminology but is also precisely tailored to suit its intended readership.
To establish a quality process for your content, you need to keep the following steps in mind:
- Define content profiles and quality metrics
- Make sure you resolve any ambiguities in the source and target languages
- Track quality per language, vendor, and content profile
- If the quality is stable, reduce sample sizes
- But always keep tracking to identify trends
- Do not forget to integrate terminology
- Do not forget to integrate into the translation process
- Select user-friendly, intuitive, yet powerful software to support your stakeholders
If you would like more information on this or any other related topic, please do not hesitate to contact us. It will surely come as no surprise to you that we also provide the leading software solutions for all the challenges described above. Our Kalcium Content Quality Platform takes care of all your content quality needs!
Contact: [email protected], +43 (0)1253 5352