The Three R’s of Predictive Analytics: Reliable, Repeatable, Relatable

For marketers, the value of data is immeasurable. From the ability to rapidly scale marketing campaigns and target valuable customers to identifying defection patterns in order to prevent churn, data-driven insights can have a real impact to your company’s bottom line. But with the seemingly endless menu of Big Data and predictive analytics tools, technology and models available, it’s easy to lose sight of the true business purpose this data is intended to serve.

Whether you’re using traditional regression techniques or the latest machine-learning algorithms, there are some simple rules to keep in mind when adopting predictive analytics to ensure you are achieving measureable and meaningful business results.

I call these rules of thumb the three “Rs” of predictive analytics – Reliable, Repeatable, Relatable.

Reliable refers to the accuracy of your predictive model. A predictive model doesn’t have to be perfect, but it does need to be accurate enough to have a business impact. The art of analytics is knowing when a model is “good enough,” so you don’t burn cycles searching for the ultimate model.

This is especially true when it comes to marketing or customer analytics, where most modeling is comparative in nature and outcomes are measured by their likelihood or rate relative to the average or random outcome, for example:

• Wireless “Customer X” is 50% more likely to attrite than average based on her demographics, usage and payment history;
• A retail customer is more likely to purchase a particular product because his shopping pattern is similar to that of others who purchased that product.

Repeatable refers to repeatable results and processes. Models need to be able to replicate their results across customers, time periods, and markets to be most useful to the business.  Poorly built models may look good initially, but could be susceptible to small changes in data and won’t hold up over future applications and measures.

Having a repeatable process means building a predictive analytics framework that can be applied to different business problems. At a high level, we use a “Define – Diagnose – Predict” framework where you define your objectives through the right data and KPI’s, mine the data to explore and diagnose the problem, and then build a predictive model that can be deployed going forward. Such a process turns analytics from a one-off project to a critical, ongoing business function.

Finally, predictive analytics must be relatable to the business users. Simply put, this means being able to present and explain analytics in business, not statistical, terms. This is key to convincing your users to trust and believe in your models enough to put them into practice.  Unfortunately, while this might be the most important of the three R’s, it’s often the least appreciated when it comes to analytics. The bottom line is that no matter how good a model is, if your main audience can’t relate to it, the model will be either ignored or underutilized.

Two short examples illustrate this point:

In the first example, the head of analytics for a Fortune 500 entertainment company was frustrated by his CMO’s desire to “understand” his models. The analytics leader lamented that his boss will “never understand” his model, since he wasn’t a trained expert in modeling and statistics. What he failed to realize was that the CMO wanted to believe the model, not understand it. The analytics leader’s failure to make his model relatable to the CMO led to the model sitting on the shelf to this day and has limited his organization’s adoption of predictive modeling in general.

In a contrasting example, a global, integrated supply-chain company was being challenged by its client on the accuracy of its forecasts. The client, an international fast-food retailer, was relying on its supply-chain partner for daily and weekly demand forecasts to optimize inventory at each restaurant. The modeler took different items, such as hamburger patties, potatoes, etc., and translated the average error rates into actual orders or units of inventory. In other words, if patties can only be purchased in packages of 10 with 10 packages per case, he calculated the percentage of time when the forecast was off enough to order more or fewer cases. By defining the model’s performance in the language of the restaurant manager, rather than the language of the statistician, the modeler was able to show that the error was rarely large enough to change the order. The restaurant managers have now become advocates for the modeling solution in the field.

The challenge with this third “R” is that it is rarely taught as part of any analytics curriculum. Modelers typically learn this the hard way as they watch perfectly good models go unused because the business “didn’t get it.”

While the world of big data and predictive analytics seems to be ever changing, it’s important to keep these simple “three Rs” in mind when building an analytic enterprise. By focusing on making predictive analytics reliable, repeatable, and relatable, you will be able to turn your business insight into foresight that creates real business value.

Joe DeCosmo ([email protected]) is a director of advanced analytics and insights at West Monroe Partners.