Methodology: The systematic analysis of the methods, concepts, paradigms, models, phases and quantitative or qualitative techniques used in a field of study. In short – a frameset for repeating success.
Did I get your attention?
The sad reality is that analysts are so busy they barely ever take time to think about how they work – what is the process of analysis? At best, some agencies and practitioners attempt to define their own methodology but this leads to wasted energy and a unique, unrepeatable, often undocumented approach. Yet, our research on organizational analytics maturity clearly indicates that the analytics process and continuous improvement methodology is the weakest of six areas outlined in the Online Analytics Maturity Model. You can do your own self-assessment to find out how you score and how you rank against the benchmark.
Like in fairy tales, repeat “DMAIC” three times and the magic will happen. DMAIC is the acronym for the Lean Six Sigma Define-Measure-Analyze-Improve-Control continuous improvement process I’ve been advocating for years. This simple 5-step approach is a tight and efficient way to improve “velocity:” reducing the time between the moment you identify a problem and point out the root cause – or state a hypothesis and prove it right or wrong. The net result is better insights, faster decisions, and quicker and bigger returns!
Let’s look briefly at each step.
State the hypothesis or problem to be analyzed, define what is expected and what success would look like. At this early stage, you can map out the actual process, the scope of work and the relationship with other aspects of the business; identify what is critical to quality and any stakeholders involved (managers, web designer, content editor, IT?). You should be able to fill the blanks of a SMART statement: “By (when), (who) will do (what) in order to impact (metric) by (how much)” or an hypothesis such as “If (this), then (that happens), therefore (the impacting metric) should result in (outcome)” or some variation thereof. As Jim Sterne rightly said, “How you measure success depends how you define success” – if this can’t be articulated clearly, even the most experienced analyst will fail miserably.
Identify data sources (anything missing or inaccessible?); determine the timeframe; conduct audits and cleansing to ensure data quality. At this stage, you can do preliminary, simple analysis on the data and uncover valuable elements without investing a lot of effort and energy. By the end of this step, the problem (or opportunity) should be clear and understood by all stakeholders.
Combining data with knowledge and experience is what separates true insight from mere reporting. Business and functional requirements and constraints are thoroughly understood; potential positive/negative impacts are identified; affected processes and resources are known. Empowered with all of this you can seek correlations, patterns, and plausible explanations to craft a thoughtful recommendation. You should be able to map this initiative on the analytical landscape encompassing audience (personas) > persuasion scenario (actions and goals) > business outcomes. Don’t forget you might uncover new metrics requirements or there might be impacts on existing KPIs and dashboards.
We want to demonstrate we can solve the problem (or capture the opportunity). The success criteria are revisited and confirmed one last time. Risks are uncovered and mitigation factors are elaborated. Here, you might conduct an A/B or multivariate tests to confirm your approach. Don’t forget about change management, training, and communicating the good news!
To quote Jim Sterne again, “It’s not about getting the right answer, it’s about asking the right questions and providing insight.” The Control step revisits the outcomes of your analysis and recommendation and measures actual performance against stated objectives. You want to keep an eye on your changes and get ready to enter the improvement cycle again.
Be nimble about this approach – it doesn’t have to be complicated and remembering the DMAIC steps will get you a long way. There are numerous online resources, books and training opportunities available.
If you want to learn more about applying this methodology in the digital analytics realm, enroll in the eMetrics Boston hands-on workshop “Mastering the digital analytics process” on Sunday, September 29, 2013.
By: Stéphane Hamel, Director Strategic Services, Cardinal Path
Re-edited from original article published at http://online-behavior.com/analytics/lean-six-sigma
 |
 |
 |
 |
Data Driven Business © 2014 • 100 Waverly Street • Ashland, MA 01721
