Executive Edge

What predictive analytics “is not”

By Piyanka Jain

Predictive analytics is a powerful method used by leading business organizations to predict future events and behavior in order to optimize current marketing, product, operations and sales actions. The prediction is based on a basic fundamental: “past behavior predicts future behavior.”

As an example, let’s take an e-commerce company selling furniture online. The online site has about a 10 percent contact rate, i.e., of the 100 people who come to the website, about 10 people contact customer service within 24 hours. The VP of Customer Service Operations is looking to reduce the contact rate as each contact costs the company incremental dollars in operations. Let’s say a 1 percent decrease in contact rate would amount to a savings of $1 million a year for the company. The operations team decides to use predictive analytics to understand the drivers of contact.

By using certain predictive analytics techniques on historical data, a relationship is identified between help page visits and visitors making a phone call to the customer service department. Specifically, 50 percent of visitors, after having gone through three distinct help pages, call customer service. This is a very helpful clue. If the visitor can be intercepted before he hits the third help page either by providing better help content or a live chat or a clarification window based on what he is browsing, contact rate can likely be reduced. I have seen organizations save and make millions by understanding this kind of pivotal relationships between behaviors and events.

On the flip side, predictive analytics, in spite of being a powerful optimization technique, is often left to the devices of data miners and data scientists, and thus is often misunderstood and misused by businesses. Having a firm hold on what predictive analytics is not will make predictive analytics a more useful tool for businesses.

1. Predictive analytics is not new. Some news items such as the Feb. 16 New York Times report on retail company Target’s prediction of teenage pregnancy implies that predictive analytics is a new found technique. But in fact predictive analytics is not new. Fischer and Durand, founders of the Econometric society, built one of the first credit scoring models 80 years ago. But predictive modeling techniques go back thousands of years –Indian astrological charts used to arrange marriages are one such example.

2. Predictive analytics does not produce perfect predictions. Often while building the model, it is clear to all that model prediction has a probability associated with it, but upon successful use, there is often a misplaced sense of perfectness in the scores. As in the case of our e-commerce company, 50 percent of visitors after visiting three help pages are going to call, but the other 50 percent won’t call. The model predicts by maximizing the likelihood, and a certain degree of misclassification always exists. By using other predictors, these odds can be improved, but the prediction will still not be a 100-percent accurate.

3. A good software tool does not mean a good model. With tremendous development on the software tool front with better GUI as well as higher automation, people new to the field often mistakenly believe that a good model can be automatically built by pressing a “build model” button. A good model requires proper technical skills and a proper model-building process. Surprisingly, sometimes even proper skills and a proper process does not deliver a good model with decent lift and low misclassification.

4. A good model does not always mean better business results. This is one of those highly prevalent myths that even experienced analysts fall for, leaving them frustrated when nobody in the business seems to care for the amazing model they built. A good model generates business impact only when the right stakeholders are brought into the analytics process at the right time, thus building proper alignment toward actionability using a framework. In our e-commerce example, if instead of building a contact rate prediction model, we had built a model to predict visitors most likely to do live-chat, would the VP of Operations care for the model and the results? Probably not. Unless we can show the relationship between what we (predictive model/team) are trying to do and his business goal (contact rate reduction), the VP of Operations is not going to use the model.

5. Models can’t be built and forgotten. Models become stale over time. If not maintained, they often stop delivering the incremental value it started with. As organizations embark on the journey of competing on analytics, they need to be aware that it is not a one-time investment. You can’t hire external consultants, get the model built and leave it at that. Model needs to be tested, tweaked and then maintained to continue delivering the incremental benefit.

Piyanka Jain (piyanka@aryng.com) founder, president and CEO of analytics training company Aryng, speaks regularly at business and analytics conferences on data-driven decision-making in an organization. Her prior roles include head of NA Business Analytics at PayPal and senior marketing analytics position with Adobe. She is an INFORMS partner.