How to augment market research and glean customer insights with AI

Surveys and focus groups are the go-to methods for gathering customer insights to drive marketing strategy. However, they have major flaws like inherent biases, poor predictive power, high costs and responder fatigue. It’s time to move beyond these outdated tactics. 

Today, AI-powered tools like data mining and sentiment analysis offer a powerful way to augment and improve customer research. By tapping into customer data and feedback, AI can provide deeper, more accurate insights with less bias and better predictive capabilities than surveys alone.

This article explores two key use cases for how AI can enhance customer understanding more efficiently and effectively.

Using AI to increase the predictive value and decrease the size of customer surveys

Two major issues associated with surveys are dubious predictive value and responder fatigue due to size. Surveys have poor predictive value because they often present responders with choices or ask responders to identify pain points in isolation from the larger context of their lives. As a result, the survey findings often mismatch with actual customer behavior and preferences. In addition, response credibility decreases as the number of questions increases. 

Fortunately, customer interaction histories can be mined to better understand actual behaviors and preferences. Traditionally, marketing analysts have used data mining techniques on structured customer data to identify behavioral patterns and build predictive models. AI lowers the requirement for structuring customer data and improves the speed at which insights can be delivered. 

While our experience tells us that AI still requires significant human supervision and direction, using AI we can evaluate a broader range of behaviors and scenarios in a shorter time. As a result, the insights generated have both predictive and explanatory powers.

A survey will still help to identify underlying drivers, needs and motivations. Customer data-driven segmentation and insights can help focus survey questions on observed behaviors, customer profitability, key demographics and other valuable dimensions. Furthermore, the survey can be shortened to address problems or opportunities identified specifically during the customer data mining stage. 

Removing biases inherent in surveys 

Surveys are significantly susceptible to biases. The very design of a study and the questions in the survey often reflect the company’s agenda. 

Take the scenario of an innovative engineering-focused consumer products company looking to develop a new brand proposition for the market. Seeing themselves as innovative, the company will likely survey customers’ thoughts about innovation, and most would respond, “It’s great.” If you further ask them whether innovation is essential to them, they will likely respond, “Of course.” 

However, when the customer goes to make a purchase decision, they are unlikely to consider innovation because that is not transparent or obvious. Instead, they may evaluate a product or service based on features and benefits that reflect a sense of innovation and relevance to their lifestyle. 

This is just one example of a bias injected into market research projects based on what a company may believe is important to them rather than what is essential to customers. While it seems obvious in hindsight, in my experience, these biases (and others) are very difficult to detect and prevent. 

An alternative, less biased way to understand what customers value is to evaluate minimally prompted feedback. This could be information on social media, chats, or simple free-form responses to open-ended questions such as, “How do you like the product?”

This information has been challenging to mine because text mining and sentiment analysis capabilities have been limited. With AI, we can evaluate large volumes of open-ended responses and identify critical perceptions, attitudes and needs. Once these AI-driven needs are revealed, a more targeted and less biased market research project can be designed to yield deeper insights and support market strategies.

Unleashing the power of AI in customer insights

The two use cases above are limited examples of using AI to create powerful insights at lower costs with less bias and better predictive powers. There are many more use cases for AI in market research. The challenge for marketing science is understanding how AI can augment and improve research methods that desperately need revamping.