Why AI Chatbots Don't Work (For Shopping)

The Rise of AI in E-commerce

With the release of advanced AI technologies like ChatGPT, many e-commerce shops have considered adding chatbots onto their websites as virtual sales assistants. The chatbot would function like a real-life shopping assistant, answering any questions and guiding consumers to the right product.

The Reality Check

Here’s the problem: it doesn’t work.

At alby, we spent a year experimenting with different ways to embed ChatGPT-powered chatbots into shopping experiences. We used extensive A/B testing to measure incremental sales and surveyed consumers about their experiences. 

What we found is that ChatGPT-powered chatbots do a good job solving support problems, but are just not that good at selling. In fact, in many cases, chatbots drove negative incremental sales.

The Core Issues

Why are chatbot ineffective sales assistants? The problem is twofold:

• People don’t click on chatbots when they have product questions. Consumers tend to avoid chatbots for product-related inquiries, associating chatbots with IT help.
• People often don’t know the right question to ask. About 95% of the questions people have are “latent”, meaning they will not think of the question on their own but still want answers.

As a result, we concluded that the open-ended chatbot is not the right experience.

A Better Alternative: Embedded Generative Q&A

alby found a more effective solution by: 

• Embedding the conversational experience directly in the product page (no pop-ups), signaling to the user that the tool is here to answer product questions. 
• Predicting the right questions that an individual customer may have and displaying a set of those questions in a conversational interface.

We found that this approach generated over 10x the engagement rate, driving significant lift in customer satisfaction and incremental revenue. We used AI combined with a proprietary dataset to power the tool. Generative Q&A continuously self-learns which questions drive actual sales. Shoppers found the embedded Generative Q&A experience far less intrusive than the standard chatbot pop-up and engaged far more with the predicted questions than they did with the open-ended textbox.

Conclusion

At alby, the customer’s experience is our priority. New AI technology requires creativity and testing to find which experience actually helps customers. Our findings indicate that traditional chatbots are currently not the most effective tool for online shoppers. Instead, embedded AI experiences that help guide customers to product answers continue to outperform, proving more engaging, leading to increasing satisfaction, and ultimately, more sales