How written product reviews affect consumer impressions of star ratings

When shopping online, most consumers are attracted to products with high star ratings. Star ratings are not only an easy way for previous buyers to rate purchases, but they also provide a visual indicator that helps potential buyers quickly sort through various options. Quantitative star ratings have long been known to positively influence consumer demand for a product, but less is known about qualitative (written) text reviews. New research from Vanderbilt Business explores the relationships between quantitative star ratings and qualitative writing reviews.

“Reading Between the Stars: Understanding the Effects of Online Customer Reviews on Product Demand” explores the relationship between quantitative star ratings, qualitative text reviews, and product demand. The study is co-authored by Hallie Cho, Assistant Professor of Business Administration at Vanderbilt Business, and Sameer Hasija and Manuel Sosa of INSEAD.

While previous research has focused on the effects of star ratings on product demand, this study is the first to explicitly examine whether (or not) star ratings and text reviews convey the same information. The study also explores the interaction between these two modes of review and how they affect product demand.

To study these variables, the authors obtained purchase data and customer reviews for 416 car models sold in the US car market from 2002-2013. The features of each model were fed into a shopping model to simulate the decision-making process. The authors then measured market size by dividing the number of models sold each year by the number of US households

The authors then extracted star ratings and text reviews for these car models from a popular third-party review site. They then completed a text analysis of all written reviews using a supervised machine learning algorithm to quantify the sentiment expressed. Finally, the authors compared how star ratings and text review sentiment tracked with sales of different new vehicle models over time.

“One of the most interesting findings is that sentiment expressed through review texts is not fully represented in star ratings,” says Cho.

The study finds that text revisions help increase product demand and affect consumer demand in several ways. First of all, positive text reviews provide supporting evidence for high star ratings, both of which drive consumer demand. Positive text reviews also help counteract the tendency of consumers to discount exceptionally high star ratings as sponsored or fake. These positive effects tend to diminish as ratings get higher, so they have the greatest effect on products that initially have a lower star rating. However, if the text reviews are negative, they reinforce the tendency of consumers to discount the high star rating.

Research also shows that these two types of ratings capture two slightly different types of ratings, although the information is still complementary. The authors liken this to Daniel Kahneman’s ‘System 1’ and ‘System 2’ framework model of human cognition, popularized in his best-selling book Thinking, Fast and Slow. The quick and easy nature of star ratings means they tend to be more automatic, intuitive, and emotional (Thinking System 1). Meanwhile, the longer and more in-depth nature of text reviews enables a more deliberate and logical evaluation (System Thinking 2).

“Star ratings capture a reviewer’s gut reaction—a quick process—whereas text reviews capture the reviewer’s conclusion at the end of a slow thought process,” says Cho. “An even more interesting finding is that the two don’t always agree!”

When shoppers are making complicated and expensive purchases, like a new vehicle, extremely high star ratings prompt them to look for evidence, like text reviews. If text reviews are less enthusiastic than intuitive reviews, potential buyers may take a step back and look for alternative purchases.

The study, “Reading Between the Stars: Understanding the Effects of Online Customer Reviews on Product Demand,” is published in Management of Production Operations and Services.