Amex bets on AI and NLP for customer service

The customer draws the AI roadmap at American Express (Amex), at least according to two of the company’s top AI leaders. When describing their latest project, Josh Pizzaro, the company’s director of AI, and Cong Liu, the VP of natural language processing and conversational AI, couldn’t stress this enough.

“We were looking to apply machine learning and advanced analytics to create frictionless and seamless customer experiences. And so when we looked across the enterprise, we looked for opportunities to inject machine learning, and we found one such opportunity in search,” Pizzaro told VentureBeat.

Contextual search is rising as a use case for natural language processing (NLP), which is booming overall. This year, Amex will debut a contextual and predictive search capability inside its app. Trained on an NLP model initially intended for the company’s customer service chatbots, the feature will “understand” various scenarios and, if all goes right, predict what customers need before they type anything at all. If a customer opens search while en route to the airport, for example, the system (equipped with their transaction and previous search data) might predict they’re looking for the lounge finder. Or in the case of a user opening search after noticing duplicate transactions, it can determine they’re likely interested in disputing a charge.

The company started the project in early 2020 and recently launched a U.K. pilot for the elevated search function, with a U.S. launch set to follow later this year. To learn more about the problem they were trying to solve, challenges they encountered, and the technology’s potential impact, VentureBeat spoke with Liu and Pizzaro.

VentureBeat: What was the impetus for creating this? What problem were you trying to solve? 

Cong Liu: For this specific capability, what we really wanted to do is anticipate a customer’s need at any given point.

Josh Pizzaro: And I would say, from a more agnostic perspective, we started building the model because if you think about where the world was, it was in a place where we would ask our card members how they’re feeling and what they wanted. And now today, in the machine learning era, we just need to know, and we do know based on the data that we have. And so we look across the different services that we provide and try to reduce the burden on the customer, and in this case, search and present things in that contextual and fast way so they get what they want faster. Because ultimately, great customer experience is about speed.

VentureBeat: Why did you lean into AI, specifically a deep neural net? What was the decision process?

Liu: We started this journey [of leveraging AI] long before we applied machine learning to some other more mature use cases, including our fraud models and some credit risk models. And in the past couple years, especially in the past five years or so, we started to see with certainty that deep neural network models started to outperform almost every other machine learning model when it comes to high dimensional data and highly unstructured data. We not only deal with some of the traditional fields, like customer transactions, but also there are tax consequences and volume history data. Neural network models can effectively deal with all of that.

VentureBeat: What internal challenges, perceived opportunities, or other factors did you consider when launching this search project? Was there anything in particular that tipped the scale for whether or not to do this, or how to approach it?

Pizzaro: First, I think it’s really about recognizing patterns. And if you look at certain use cases where you have customer behavior that’s being repeated and you can expedite that behavior, then that tends to be a real sweet spot for machine learning capabilities. The other thing I would add is we take the decision to apply machine learning techniques quite seriously. We have an entire AI governance board that cross-checks all the models that we build for bias and privacy concerns. So even taking the approach of AI, we have to justify to a number of internal teams why it makes sense.

VentureBeat: The NLP model used to train this neural network was originally developed to advance your chatbots. What was the process of extending its use? And what did you learn about applying models created for a specific purpose to a new use case?

Liu: When we started developing this model, we started with tags and focused on improving the personalization of the data and making the bot smarter. Later, we identified it could be power search as well because both in search and in chat, the goal is to help the customers with better and more proactive services. So from a data science perspective, it’s kind of a natural extension.

Pizzaro: For what we learned, I need to take a step back and say we developed an in-house annotation team that retagged data where our models went wrong. It was all American Express customer service experts. And a lot of other folks, you know, farm this out to different companies. And what we realized is that by actually having the customer service experts tag the data, accuracy is just so much higher. So it’s an investment, but it’s an investment in accuracy and progress.

VentureBeat: So you think that’s your real differentiator?

Pizzaro: We absolutely do. It’s been key to the success of the accuracy of our models.

Liu: Sometimes people overlook the effort they need to spend on the simple tasks, such as labeling. But without accurate data, you’re not going anywhere. You’re not going to build an accurate model.

VentureBeat: So that’s worked well for you. But I know you feel that building this type of one-to-one search capability is more difficult than it sounds. What was the biggest challenge you ran into along the way, and how did you overcome it?

Liu: I think the biggest challenge for this particular capability is that, in general, when you open a browser and do a search, you’re looking at 10 or 20 different links and have to find what you want. We really wanted to build a one-shot journey. If the customer searches and is already happy with what we provided, that’s great. But otherwise, we’d love to get it right with as few inputs as possible. So that’s the challenge: How do you get the model right with very limited input?

VentureBeat: Are you finding any limitations with your current model or approach?

Pizzaro: One of the things we have not done today is create generative models. And so that’s something we know is a technology we’re capable of working with and creating, but it’s not something we feel is in our customers’ best interest at this time. And so we haven’t explored it much in production.

Liu: And another thing I want to add here is that when you talk about limitations of machine learning models, there’s one common limitation, or I would say, an opportunity. How do you keep improving the model? Because as long as it’s a machine learning model, it’s not 100% accurate.

VentureBeat: Let’s talk about the impact. What’s the most significant result you’re seeing?

Pizzaro: Search just launched as a pilot in the U.K., and we’ll be launching later this year in the U.S., but we can speak to how the predictive machine learning capability is working in chat. Over the past six to eight months, we’ve seen our RTS scores, which is essentially a proxy for NPS scores for the bot experience, go up significantly. And so obviously there’s a number of things that we’ve done in order to move some of those results, but we do believe that some of these advanced machine learning models are helping that score.

We’re also seeing higher engagement with the responses that we send back to our customers, which refers to them clicking on a link or the information that we’re providing. It’s greatly improved. Our chat function is a bot-human hybrid, and so we’ve been reducing some of the chat handling time on the agent side. We’ve also seen more fully automated experiences.