via HBS Working Knowledge

Businesses rely on solid marketing strategies to boost sales—yet the tools used to evaluate these strategies often provide misleading results, leaving managers with the inability to accurately measure how they can get the best bang for their marketing buck.

"Companies really need to pay attention to the effectiveness of their marketing instruments"

Thomas J. Steenburgh, an associate professor in the Marketing Unit at Harvard Business School, has developed a new analytical tool that more accurately measures the effectiveness of various marketing efforts. He created the model with Qiang Liu, an assistant professor of marketing at Purdue University, and Sachin Gupta, the Henrietta Johnson Louis Professor of Management and professor of marketing at Cornell University.

Steenburgh believes that the model could help brand managers determine which marketing strategies work best to invest in.

"Companies really need to pay attention to the effectiveness of their marketing instruments," Steenburgh says. "They need to look at whether they're creating new customers or whether they're just drawing customers away from competitors. It's a fundamental question in the field, and this model helps measure that."

The ideal mix

When planning marketing campaigns, brand managers have a wide portfolio of weapons to draw on, including in-store merchandising, advertising, coupons and sweepstakes, trade promotions, prices, and deployment of a direct sales force. The key is crafting the right mix between them—the ideal brew needed to achieve sales and market share goals.

The trick is that each marketing effort affects consumer behavior in different ways, and also prompts different types of responses from competitors. Some activities result in expanding demand across an entire category of products. Take for example the "Got Milk" advertising campaign, which is intended to increase demand for a category of products, milk. In contrast, an advertisement that points out how one brand is better than a competitor's brand has the goal of encouraging consumers to switch products within a particular category.

If a business seeks to grow demand for a category of products, the effort may not elicit much of a reaction from its competitors; after all, if the entire category grows the rising tide lifts all boats. But a competitor's reaction is typically quite different when a company attempts to move in on its market share, perhaps by offering price discounts. Since this strategy is viewed as more threatening, the competitor can be expected to retaliate with prejudice—often by firing off a campaign to win back many more customers than it lost.

"We know that retaliation happens and that companies worry about that," Steenburgh says. "But nobody benefits when both companies are retaliating. One effort just offsets the other."

Measuring the different effects of these marketing strategies can help brand managers make the right decisions about which strategies to use in their marketing mix. Steenburgh, Liu, and Gupta argue that the tools that have been used in the past to analyze the effectiveness of different marketing activities—called discrete choice models—can skew the results and misguide brand managers.

Traditional discrete choice models—logit, nested logit, and probit, for example—are flawed because they make it appear as if all marketing activities produce the same results, the researchers contend. In reality, differences between various marketing instruments are often significant. The cause of these flawed results comes from what is called the Invariant Proportion of Substitution (IPS) property, which implies that the proportion of demand generated by taking business away from a competitor is the same, no matter which marketing activity is used.

"These models get run all the time in academics," Steenburgh says. "There has been some talk at conferences where there seems to be an understanding that these models are too restrictive."

Widening the view

So the professors created a new discrete choice model called Flexible Substitution Logit (FSL), described in their working paper The Flexible Substitution Logit: Uncovering Category Expansion and Share Impacts of Marketing Instruments. The model relaxes the IPS property and allows a wider variety of results to be analyzed when studying the effects of different marketing instruments. By doing so, "the FSL allows a wider variety of individual-level choice behavior to be recovered from the data," according to the researchers.

The team tested the new model by looking at the marketing of prescription drugs, namely, statins, used to lower cholesterol levels in people at risk for cardiovascular disease. Using data from 2002 to 2004, they studied the three primary ways these drugs were marketed by Pfizer, Merk, Bristol-Myers Squibb, and AstraZeneca: "detailing," in which drug firm representatives personally visit physicians to sell the drug; at professional meetings and events (M&E) sponsored by the pharmaceutical firms; and by using direct-to-consumer advertising (DTCA).

First, they employed the complex mathematical formulas of traditional models to study different marketing strategies used by the drug companies. They found that the IPS property created counterintuitive estimates of demand gains attributable to these marketing investments. Although logically the researchers expected detailing to generate greater demand for the products than either direct-to-consumer advertising or meetings and events, the traditional models would not allow them to discover this because of the IPS.

When they applied their FSL model, however, the results provided much greater detail about the potential effects of different marketing investments. For example, the model predicted that sales gains from DTCA and M&E would come primarily through category expansion (87.4 percent and 70.2 percent, respectively), whereas gains from detailing would come at the expense of competing drugs (84 percent). By contrast, the random coefficient logit model predicted that gains from DTCA, M&E, and detailing would come largely from competing drugs.

"The FSL model is very useful if you want to predict consumer demand," Steenburgh says. "This model gives you a better way to do that."

Figuring in payback

With results that provide a better analysis of how different marketing instruments work, brand managers can now decide how to best invest their marketing dollars. For example, if a brand manager is concerned about retaliation from competitors, the best decision may be to limit investment in detailing and instead put more emphasis on direct-to-consumer advertising or on sponsoring meetings and events, both of which are more likely to expand the category.

Steenburgh notes that future research is needed to find alternative models that overcome the IPS, and he hopes that the FSL model will be applied in other studies that examine the effectiveness of marketing instruments.

"It would be interesting to apply the FSL model in a lot of other situations to see which ones expand the pie and which ones threaten other actions," he says.