All things associated with measurement was the theme of this year’s Best Practices conference last month in Austin, Texas. Doug Hubbard, the keynote, kicked off the conference by challenging us to rethink our perceptions of the concept of measurement, the object of measurement and the method of measurement. He started by defining measurement as an expressed reduction in uncertainty based on observation.
Doug said, if you think something seems like an immeasurable “intangible”, you probably need to re-define it. Ask yourself:
- What do you see when you see more of it?
- Why do you care? (what decision could depend on the outcome of the measurement)
- How much do you know about it now?
- At what point will the value make a difference?
- How much is additional information worth?
Doug suggested that we should make the following measurement assumptions when we are defining our measuring strategy:
- It’s been measured before.
- You have more data than you think.
- You need less data than you think.
He then put on his economist hat and told us several measuring statistical methods including, The Rule of Five, i.e., there is a 93.75% chance that the median of any population is between the smallest and largest value in a random sample of five. CIDM staff tested this theory by asking attendees to guess the number of Legos in a large mason jar. Sure enough, the median of our lowest and highest guesses was just shy of the exact count of 459!
There were many great presentations on metrics throughout the conference, everything from the politics of metrics, exploring how to measure the immeasurable using minimal viable indicators, measuring quality, and even the results of the CIDM member survey on Taxonomy and Metadata Strategies. Some nuggets of wisdom captured by our attendees:
- Metrics without action is just trivia.
- Choose one metric that you can control, measure, and improve to make your project justification.
- Rather than asking how far a rock moves in a year, perhaps we should start with whether or not the rock moves at all.
- Tell people the behavior you want, not what you don’t. Forbidden fruit is the sweetest.
- The danger of cost avoidance is that it becomes a race to the bottom.
- Quantitative data helps identify that there is a problem. Qualitative data helps inform what the problem is.
- The formula for the value of information is the cost of being wrong times the chance of being wrong.
- The economic value of measuring something is inversely proposal to the measurement attention it gets.
- When there is a lot of uncertainty, you only need a couple of data points to greatly reduce that uncertainty.
- Don’t measure to ensure you are better than someone else. Measure to determine if you are meeting user needs.
The presentations alone aren’t the only reason attendees come to the conference; it’s all about sharing ideas. We provided plenty of opportunities to do this including the return of our Management Therapy Session. This year there were two types of therapy offered; attendees could opt for one-on-one sessions where they could get advice on their own management challenge or they could attend the new group therapy session where participants voted on which topics to discuss and then broke into small groups for several mini therapy sessions. The group therapy sections included: the value of documentation, working in Agile environments, preparing your team for the future, collaborating with other departments, gathering user feedback, and managing difficult people and change management.
In the change management session, someone referenced the Knoster, Villa, and Thousand “A Framework for Thinking About System Change” — The five things you need to change:
- Vision for the change (without vision you have confusion)
- Skills to make the change (without skills, you have anxiety)
- Incentives for making the change (without incentives, you have resistance)
- Resources to make the change (without resources, you have frustration)
- Action plan to get to the change (without an action plan, you have many false starts)
Of course, in the evenings there was networking and fun to be had by all. A group of puzzle solvers had a chance to collaborate on a space-station themed escape-room which was included as part of the conference. Unfortunately, the game was a bit more challenging than expected and all teams “ran out of air” before the allotted time-period. Regardless, two teams continued to solve the puzzle and did indeed finally escape from their space stations. Attendees also enjoyed exploring Austin, visiting the music scene on famous 6th Street and watching the evening emergence of 1.5 million bats from the downtown bridge.
One of our members sent us an email after the conference stating:
“It’s a serendipitous experience whenever I attend a Best Practices conference. Whatever stage my company is at in our structured authoring journey, Best Practices brings together the best minds to explore and innovate to resolve the pain points we’re experiencing as our doc sets and our department matures.”
Thanks to all who participated in this year’s conference. Next year’s conference will be in Baltimore, MD where we hope you can have your own serendipitous experience!