How Etsy Is Upskilling its Workforce With GenAI and Machine Learning
At a recent Axios event in New York City, Esty CEO and tech entrepreneur Josh Silverman shared the ways in which his team is using GenAI and machine learning, experimenting with customer service upgrades, fielding customer feedback and training employees on AI. Following are six insights from the discussion.
- While GenAI tools are important, the data processing capabilities, technical understanding and talent required to implement artificial intelligence tools are equally paramount.
“Generative tools are amazing, and it is another important capability, but there’s a lot of other forms of AI and machine learning that are also really important. So we shouldn’t lose sight of that,” Silverman said. “There’s a lot of data processing capabilities that are required. There’s a lot of technical understanding, having the right talent on the team, and then having a lot of the right infrastructure.”
Another key component is having the ability to test in front of audiences to gauge their level of interest and tolerance for AI tools. “I think right now, where the capability is is often further ahead than where consumer comfort is. So, trying to find out what people are actually expecting and meet them where they are is going to be a big part of the journey now.”
- Large Language Models are used by product sellers and within Etsy as a business, but the level of usage differs.
Many sellers on the platform are generating content in collaboration with a machine, Silverman said, and a large amount of new item listings are generated in collaboration with GenAI. But given that Etsy is a platform for handmade goods, the boundary between man-made and machine-made required some clarification.
“We had some discussions about, is that handmade or not? Does that belong on Etsy? And we decided, ultimately, that it was—in the same way that electronic dance music, the most popular form of music in the world right now, is collaborations between humans and synthesizers,” said Silverman. “I think this collaboration between humans and machines to generate art is actually going to be the next frontier.”
Within Etsy the business, “translator technology” is the core capability. Here’s an example: When a user enters the search query “cocktail attire for men,” the platform shows the user sport coats—even though nothing in the listing says “cocktail” or “attire.” “The translator technology understands not what you said, but what you meant,” Silverman said. “And that’s been a very important part of our journey for quite some time.”
- Consumer comfort level with AI chatbots is evolving—but they’re not there yet.
Citing an example of how the technology’s capability is surpassing consumer comfort levels, Silverman shared how unpopular a recent chatbot sales assistant was with users. The problem it was solving for—narrowing down a hundred million sales items to 30 search results appearing on the site’s first page—was “literally impossible.” But an extra chatty chatbot—however well-meaning and thorough—came across as annoying to time-strapped customers.
The chatbot has “very, very low customer satisfaction scores now,” Silverman admitted. “Customers are going to get there. They’re going to get used to interacting with OpenAI, and that’s going to become a more natural thing. I think over time that’s going to be great, but it’s not where they are right now.”
- GenAI works well as a creative idea generator, but still requires human curation.
A more successful initiative for Etsy, called “gift mode,” uses GenAI to recommend groups of products related to one specific thing about the person that is being shopped for—such as being a cat lover or a mixologist. And then under each idea, the team picks three or four of the best listings to show the user.
“We start to learn from that,” Silverman said, “what are they browsing on and what are they dwelling over? Those very implicit cues that tell us, here’s what we’re getting it right, here’s what we’re getting wrong. We did that across hundreds of persona—and that would’ve been very difficult before.”
But despite the improvements resulting from AI efficiencies, humans play an important role in the gift mode process. “Our mission is keeping commerce human. We’re all about humanity, but we talk about becoming super human by partnering with machines with people,” says Silverman. “We still need people in gift mode. We still have human curators that look at the ideas that the machines come up with and then pick the very best. And that feedback also helps the machines get better, too.”
- Customer feedback is critical to the successful integration of AI into ecommerce.
To gauge consumer satisfaction with its services, Etsy performs focus groups and surveys, as many businesses do. But the majority of feedback comes from learnings from A/B tests. The main focus at Etsy internally is “speed to learn,” which determines how long it took to learn something, Silverman explained. “Every team at Etsy is organized into a small squad—eight to 10 people. Those squads typically ship twice a week. So, they come up with an idea and they launch it. They’re not designing a PowerPoint deck and making an argument for why they should and asking for permission. If they have an idea, they code it and they ship it.”
That alacrity results in exponential learnings. “Within seven days, we usually have statistically significant results on whether it generated more sales or not. Eighty percent of the time it doesn’t, but the cost of failure is so low that that velocity really pays off,” he said. “We learned a tremendous amount by just putting ideas in front of customers and understanding how they actually interact in the wild. And it’s very often counterintuitive. It’s not what they say. They do something very different.”
- With data scientists in demand and scarce, upskilling employees internally with machine learning techniques is critical.
One of Etsy’s goals is to democratize machine learning, according to Silverman. “Today, for the most part, there’s a relatively small group of people who are commonly called data scientists. And the cost to hire a data scientist goes infinite pretty quick—because there aren’t that many of them, and everyone’s trying to hire them,” he said.
To fill that gap, Etsy has turned to upskilling employees. “Can we create paved paths that allow any full stack engineer at Etsy to be able to use the basic machine learning techniques and technologies?… The idea is, how do we take the techniques and technologies that you need for AI and build enough tooling internally that someone who’s a competent full stack engineer can do 80% of that themselves with a certain amount of training… We’re making a lot of progress there,” he said.
Moreover, in certain areas of Etsy’s technology stack, relating to trust and safety, for example, the company has been focusing on tools that allow its full stack engineers to launch AI around trust and safety without needing to tap the specialist data scientists, and then broadening it to more parts of the platform.
“I think that’s where the puck is going,” Silverman said. “Those kinds of infrastructure investments make the entire team at Etsy a lot more productive. And over time, I think that’s what’s going to need to happen across the industry, because honestly, there’s hardly a team at Etsy that isn’t begging for AI to help make their work more productive.”
