12 years ago
eMetrics: The Measurement Curve

How much implementation?  How much reporting?  How many man hours?  Although it’s hard to come up with a number for each of these dimensions as it relates to getting measurement strategy off the ground, these are the essentials whether one realizes it or not.

Too little implementation and you’ll be spinning pivots in everything from Excel to Business Objects just to deliver rudimentary reporting.  Too much time on the implementation and you’ll never generate a single report, you’ll all too often be saying “I’ll get that to you next week” to key stakeholders in your business – and nobody wants that.

The most critical, and yet most often misunderstood question in digital analytics is…”what are your KPIs?”  The KPIs should be tied to revenue, and the key to any good implementation is to ensure that KPI’s can be implemented and tested before product launch and the required development cycles.

If there are too many stakeholders, the list of KPIs may actually go far afield from what is required for the business, and could potentially put the entire project at risk.  In some cases it’s important to identify certain “linchpin” variables such as a Customer or Asset ID that can then be leveraged in a supplementary post-processing exercise to deliver on less critical KPIs.  I should stress that post-processing should be a good compliment to a sound web-based implementation, not a substitute.

The bottom line is this, the more implementation the greater amount (volume and variety) of reporting we can do.  However, the key is for any good analyst is to find the equilibrium.

Below is a chart that I’ve created that uses the same principles as a Supply & Demand curve, where “price” is replaced with “hours”

The “RIE” curve is the mount of Reporting Implementation Effort, and the “RRT” curve represents the Required Reporting Time.  At a reporting volume of zero, the required implementation effort is also at zero, and the amount of time that’s need to complete the report can be infinite…in other words you can take forever to generate zero report, utilizing zero effort.  However at the other end, if an infinite number of reports were required, it would take an infinite amount of implementation work and you would need to turnaround each report in literally “no time” at all.

The goal of a good implementation is to find the equilibrium between the competing demands of each dimension, a thorough enough of an implementation to meet the required amount of reporting (volume and variety) within the required amount of time.

For example, if you’ve got data architects in your organization that want to delay the implementation so that you can fit all of the fields they have available into your web beacon because it “might be useful” someday…I would probably push back because it’s an added risk to both initial and recurring reporting time.  Also, if someone suggests we can go to product launch with a web measurement beacon that contains near zero customization, with the thought of running post-processing queries against a data warehouse for daily reporting, I would also say no to that because it will take too long to generate each report – and it will be you the analyst holding the bag.

In short, search for the equilibrium and expect it last for a good 24-36 months.  After that point your implementation as a matter of good practice should be revisited anyway.

Blandon Casenave is currently Vice President of Digital Media Research for NBCUniversal. In his current role, Blandon is responsible for metrics and analysis for NBC.com and NBC Sports. He is also responsible for the measurement of centralized products and services across all brands within NBCUniversal as they relate to video and mobile. His team also provides general consulting for affiliated brands within the company. During his tenure at NBCUniversal, Blandon has overseen the digital measurement and reporting of two Olympics, led the first direct cross-platform measurement of TV & Online via a partnership with Arbitron and developed a patent pending method for measuring unique visitors and visits on the mobile web. Prior to landing at NBCUniversal Blandon worked at MTV.com and CourtTV, totaling over a decade in digital media with about half of that time spent in TV ratings research as well.