How Boll & Branch Spotted Scammers Attempting Generative AI Fraud

Scammers are now using generative AI to commit return fraud, and luxury bedding brand Boll & Branch was almost a victim.

In February, a scammer using generative AI submitted a fabricated photo of a ripped Boll & Branch sheet to customer service, claiming it arrived damaged. Fortunately, the luxury bedding retailer quickly identified the picture as fake. (The AI watermark was a large clue.)

CEO and co-founder Scott Tannen shared the attempt on LinkedIn to alert other retailers of this new form of return fraud. The topic resonated with his audience, generating more than 2,000 reactions, 309 comments and 120 reposts.

Boll & Branch launched in 2014 as an online-only bedding retailer, and today it generates more than $300 million in annual revenue. The brand operates its own stores and sells in Nordstrom, Bloomingdales, among other retailers, and has been profitable for 10 years, Tannen said. This success, unfortunately, comes with being a criminal target, he said.

Tannen shared details of this story in the below Question & Answer with Chief Marketer.

Chief Marketer: So what happened?

Tannen: Someone on my CX team just flagged it. It was so obvious to us that it was AI. It was the first time that we’d seen it. We got two in one day from different users. Of course, all with the same story that they bought in a retail store, so they don’t have a receipt or they’re not in our system. And one actually had a watermark of AI on it and the other didn’t. But knowing our product as we know it, it’s not how our product would rip.

It shouldn’t have been a surprise, but it was a surprise that all of a sudden people are trying to find ways to exploit using AI. You used to need real Photoshop skills to do something like that, and now all of a sudden you just need to type a prompt and you are done.

Chief Marketer: How big is your CX team and are they trained on how a product would rip?

Tannen: We don’t have any outsourced CX. We have seven people on the team. They’re all right in headquarters, and so they know the product intimately well.

You don’t know what gets through. People might ask, ‘Well, how many times have people scammed you?’ ‘I don’t know. How would I know? Because they scammed us.’

But we try to think about time to resolution as a key metric. If somebody has the rare case where we do have a defect, we’ve historically just asked for photo proof, and we are going to overnight a replacement and off they go. What’s commonplace for us is to ask somebody just to send a picture. But now, can you trust that anymore? Because these tools are only going to get better.

So do you move to FaceTiming with somebody? You need to see it real, but then could you add a filter to that? We don’t want to inconvenience the 99.9% of customers who just want the product. They want what they’ve paid for and they deserve it. But [it’s] how you don’t let that sort of fraction of a percent cause everybody else to have to jump through too many hoops.

We just have to be better at understanding our data and understanding our customer’s history. So, if you’re a customer that’s bought five or six times from us and we have a relationship with you, we’re going to trust if you reach out and you have a problem. Whereas somebody that has no history, has never ordered, is unwilling to give us a lot of information, the ‘Spidey Sense’ starts perking up.

Chief Marketer: Are you guys in the middle of revamping how you train or implementing a new policy?

Tannen: It’s still early days. But this is something else to think about. Our first step is actually asking the CX team, and that’s what we’ve done is asking, ‘What do you guys think we should do? How do you think we should keep our eyes on this and look at it?’ To me, it’s less about policy. It’s more about starting by just being aware. If you’re unaware, things are going to get past you. If you start becoming aware, you can then start identifying how big of a problem it becomes.

Chief Marketer: How often are your products defected or what percent of the CX calls are about it?

Tannen: Extremely low. We’re known for our manufacturing standards. So anytime someone claims a defect in a product, it gets immediately escalated. I know about it, Missy (co-founder and Chief Designer Missy Tannen) knows about it, so it doesn’t happen very often.

Let’s say this had been routine, and let me give you an example. Sometimes someone will open a cardboard box and the scissor digs in, and it’ll cut something.

Chief Marketer: Which is what this looked like.

Tannen: It’s what it looked like unless you know the type of box that we ship that product in, and how the fold would be on the front and that the scissor can’t actually touch the product because it has a double barrier layer of cardboard there.

Again, those are the things that someone claims a defect that immediately comes up and all six or seven of us that look at this say, ‘Well, no, that’s not a defect that can’t be because it would have to be this, this or this. It has to have been ripped somewhere else.’ And in that case, we know, okay, our product cotton doesn’t rip like that. It’s all about intelligence and evaluation.

Even in our category, our return rate is very, very, very low. You don’t generally have an issue of your sheets fitting. Everybody kind of knows what size bed they have, unlike apparel where it’s like, “Well, I’m in XL in this company. I’m a medium in this company.’ So our returns are usually purposeful for a reason. All right, I got the blue and I just really didn’t like the blue or I want to light a color and that kind of thing. When it falls outside of that, it’s something that we look at as a company really closely because we just don’t get that many.

Chief Marketer: What was your goal behind posting about this on LinkedIn and what do you think of all the chatter now?

Tannen: My goal in posting it in general is always just trying to be transparent and sharing what I’m experiencing as an operator, knowing that somebody else is going to experience the same thing.

Sometimes we all just have to pick our heads up and realize what’s going on. And we can’t do that if we’re not sharing. I had so many companies reach out. I’ve had two groups of people reach out. One would be people that are service providers that are like, ‘Oh, we can help you detect this.’ And then a lot of other operators that are like, ‘Oh geez, now I’ve got to worry about that.’

Chief Marketer: For the service providers, were any of them viable options?

Tannen: Not really. Nothing’s better than your eyes. The last thing I want to do is fix an automation problem with more automation. At some point, you need human intervention. The volume’s not there at the moment for us to say, ‘We’re overwhelmed by this. We can’t keep up.’ At the point that you become overwhelmed by it, that’s when you might look to a third-party solution.

We have a very bespoke customer experience system, and so we’re not at the point where you can’t tell yet, but I think at the point that you can’t tell, what are we going to do? At the end of the day, we’ll probably just get duped by people and send them an extra fitted sheet. I could spend a bunch of money on a software product or I could just end up losing a little bit of money on sending free product to people that really want to bilk us for a fitted sheet. Good luck. What are you going to do? Sell it on eBay?

Chief Marketer: Any advice for other brands?

Tannen: My advice for other brands is that when you see something, share something because the only way we’re all going to really have a sense of how these tools can be exploited against us is to understand what’s happening.