4 A/B Tests Every SMS Marketer Should Be Running

Most marketers are at least familiar with the concept of A/B testing. But, putting it into practice—especially on a consistent basis—is easier said than done. Planning and producing all of your brand’s ad, email, and SMS campaigns is a tall order on its own, so piling a testing program on top of that might seem downright impossible.

Here’s the thing, though: if you’re not testing the effectiveness of your messaging, then you’ll never know whether you’re getting the highest possible returns for your efforts.

Here’s the other thing: A/B testing doesn’t have to be a comprehensive, grandiose endeavor. It’s 100% okay to start small and build on your learnings. The key is simply to start.

Many brands start testing campaigns first—and that’s perfectly okay. This will allow you to practice thinking strategically and to get results quickly—which may translate into a more immediate impact on revenue. (If you’re interested in A/B testing your campaigns in Postscript, follow the steps outlined here.)

But what we’ll focus on in this article—and where the long-term revenue impact lies—is automations. Specifically, your:

• Abandonment automations (browse abandonment and cart abandonment)

• Welcome series automation

• New subscriber/no order automation

• Post-purchase automation  

These automations are always on and running, so any results you gather from A/B tests can be applied immediately and continuously. This allows you to fine-tune some of your most valuable messaging to increase its value even more. (Did someone say “maximized revenue?”)

Let’s take a closer look at the automations above and the A/B tests we recommend for each.

Of all the SMS automations you have up and running for your brand, your abandonment automations are the most financially lucrative. These recipients are extremely high-intent—right on the brink of completing an order.

Testing these automations should be at the top of your priority list—not only because of the potential revenue boost, but also because legally, you can only send one message per abandonment event (and it must be delivered within 48 hours of abandonment). That means you have one shot to convert—so you’ve gotta make it count! 

When testing messaging for folks at this stage of the buyer’s journey, you’ll want to think about two things:

• What’s holding them back

• What could push them forward

These factors can differ widely depending on your audience and your products. For example, shoppers interested in higher-ticket items may need more reassurance about the quality and effectiveness of your products—or the ability to return the item at no cost if it doesn’t meet their expectations.

For less expensive products, shoppers may be concerned about shipping costs. If the subtotal is less than $20, then an $8 shipping charge can be a tough pill to swallow—so a free shipping offer could be enough to seal the deal.

Here are a few more testing ideas for each of these automations. 

According to Wassim Dakik—Customer Success Team Lead here at Postscript—abandoned cart automations should be one of the most active A/B testing grounds for ecommerce brands. Many stores default to a generic discount offer (e.g., 10%, 15%, or 20% off) for every single abandoned cart. But in doing so, they’re missing out on a huge opportunity to increase conversions.

That’s because a discount only addresses one barrier to purchase: price. But that’s not always the barrier you need to overcome. Different offers may better address different objections—but you’ll never know until you test them.

“Your value props are the low-hanging fruit [for testing],” Dakik said. “What’s going to work best for your potential customers at this stage? Is it a discount, a free shipping offer, buy now/pay later?”

Dakik recommends selecting three to four of your best value props to test on your abandoned cart automation. “You’re already offering these things, so why not test to see if they resonate better than a blanket 15% discount?” Dakik said.

Here are some examples to get the ideas flowing:

• For carts above a certain value (e.g., $100) test a free shipping offer against a 20% discount offer.

• For carts containing high-ticket items (e.g., SKUs with prices above $150) test a social proof message—for example, one featuring a customer-submitted review or photo—against a 15% discount offer. Or, test the same discount against a buy now, pay later offer (if your store allows customers to pay in installments via a service like Afterpay, for example).

• For carts containing clothing items, test a “what happens if it doesn’t fit” message highlighting free, easy returns and exchanges against a social proof/customer review message specific to each item. (You could also test an MMS featuring customer-submitted photos against an SMS featuring the customer’s review only.)

• For discount offers, test expiration windows (e.g., “offer expires in 48 hours” versus “offer good for one week”).

Keep in mind that while Postscript allows you to create up to five different test groups per automation, we’d recommend that those new to A/B testing start simple with two variants. The length of time per test will vary depending on your list size, but you’ll want to wait until you've sent 500-5,000 messages total before you look at the performance data and declare a winner.   

Browse abandonment messaging might be similar to cart abandonment in terms of offers, but one additional factor to think about is timing. In Dakik’s words, “When do you nudge them?”

Someone who is casually shopping around might not be ready to convert on any type of offer, so you probably don’t want to fire off a message minutes after they view a product on your site.

You may also want to consider the number of products viewed. In some cases, a shopper who looked at several products on your site may be higher-intent than someone who only viewed one product in a 24-hour period—so you may want to test sending the former a more enticing offer (e.g., a higher discount).

According to Dakik, welcome series automations are ripe for testing because they typically have high unsubscribe rates. So, there are two ways to go about A/B testing:

• Finding the approach that yields the fewest unsubscribes

• Finding the approach that produces the most orders

If you’re more interested in retaining subscribers for the long haul, then you may want to test delaying your welcome series until after a customer purchases from you.

“One question to answer is, should the welcome series go to everyone who subscribes, or just purchasers?” Dakik said. “I tend to think it should come after purchase, because people feel more connected to your brand after they purchase from you—more like family. But test that!”

On the other hand, if you’re more interested in using your welcome series to build brand loyalty prior to purchase—and possibly help drive that initial sale—then you should test the order and content of your messages as well as when you extend your offer (if you choose to extend one at all).

For example, you could test the following approaches against one another for pre-purchase subscribers.

Message 1: Brief welcome with discount offer

Message 2: Personal message from the founder explaining brand mission and history

Message 3: User-generated content and CTA to reply with questions

Message 1: Personal welcome from the founder with information about the brand mission and history

Message 2: User-generated content and CTA to reply with questions

Message 3: Discount offer

You could also test different offers—free shipping, free gift with first purchase, various discount percentages—against one another to see which one yields the best conversion rate. Or, you could test a series with no discount offer against a series with a discount offer to see if the discount is actually effective in converting new subscribers to customers.

In fact, brands that don’t want to offer discounts as part of the welcome series can still use this automation to drive plenty of value. “This is a great place to incorporate keywords to drive engagement and learn more about your subscribers,” Dakik said.

For example, one message in the series might invite subscribers to reply with the keyword that best represents their interests, their personal style, or a problem they are interested in solving. You can then bucket those subscribers into segments based on their responses and deploy targeted automations for each. It’s a great way to give a more valuable, personalized experience—and as a result, retain subscribers longer.

Here’s an example from skincare brand Glow Recipe:

Once the subscriber replies back with their skincare concerns, they could be placed in an automation (or even future campaigns) specific to that concern. For example, those who replied “FINELINES” might receive an offer with a deal on a sleeping mask—while those who replied “BREAKOUTS” might get an offer on a cleanser.

If a subscriber opts into your SMS list and does not convert on any of your messaging within the first 30 days, then that subscriber is at a high risk of churning out of your list. Why? According to Dakik, there’s a good chance they’ve gotten distracted and pulled out of the purchase mindset.

“In today’s day and age, consumers have so many distractions,” Dakik said. “It’s crucial to bring them back in.”

Getting new subscribers to complete that initial purchase is critical to retaining them for the long haul. Once they’ve experienced the value of your products, they are more likely to see the value in staying engaged with your brand.

But at this point, getting them to that sale likely won’t be easy—which makes it even more important to test what works. They’re playing hard to get, and you’ve gotta figure out how to speak their love language.

Dakik advises that you go back to the previously discussed value props. Think about what you’ve offered thus far. Have you missed anything? Could you reframe an offer in a way that better addresses whatever barrier or objection is holding the subscriber back?

For example, if your subscriber welcome series already mentioned free shipping and returns for orders over $50, could you expand on that and expressly link “free returns” to “zero risk?” This sort of message might read something like: Remember, all orders over $50 qualify for free shipping and returns, so if you don’t love your product as much as you think (and we know!) you will, you can easily return or exchange it at no cost to you.

This may help overcome a subscriber’s fear that if an item doesn’t fit or function the way they want, they’ll be stuck with a product they won’t use.

Another approach to overcoming this barrier is assuring the subscriber that the product has worked for a lot of other people (i.e., in the form of ratings and reviews). To find out which of these two approaches work best for your audience, test them against each other!

To do that, you’d set up a split test in which the “A” group receives the message above, and the “B” group receives something like: We know you’re going to love our products because we receive dozens of rave reviews from customers just like you every single day! Follow this link to see what they’re saying: link.here

You could even include an image or GIF displaying some of your most convincing reviews. This strategy can be especially effective for products with high price points.

We saved the best for last—and we say “best” because there are so many ways to test your post-purchase messaging. Your strategy should be unique to your brand and, more importantly, your products.

There is no one-size-fits-all approach to post-purchase follow-up, and Dakik urges merchants to be thoughtful about their post-purchase automations and A/B tests to ensure they get the most out of them.

The first factor to think about is timing. This is particularly important for consumables. “How long until a customer runs out of the product they ordered?” Dakik said. “You’ll see a lot of merchants putting in arbitrary times, but that’s not strategic.”

Let’s say, for example, that your store sells vitamins, and a particular customer ordered a 30-day supply. Sending a re-order reminder one week after their initial order ships is probably too early—but sending it 30 days later is probably too late. To find that sweet spot, factor in shipping times and the customer’s feeling of urgency. When will they start to notice that their supply is running low? You may want to test two weeks post-shipping against three weeks—or test an SMS version against an MMS version containing a photo of a half-empty vitamin bottle.

For products people only purchase once per year—or even less frequently—a re-order reminder wouldn’t make sense. Instead, these orders are ripe for upsell and cross-sell tests. For example, say a customer ordered a pair of skis. They’re probably not going to buy another pair of skis for a while—but what product might they logically purchase next? What might perform better—a follow-up discount offer on a helmet or a ski jacket? Test it!

And of course, cross-sell and upsell automations shouldn’t be reserved only for large, infrequent purchases. If you have multiple products, then you have multiple opportunities to test cross-sell and upsell follow-ups—and hopefully, discover some trends around product interests and purchase paths.

Once you’ve discovered a particularly effective cross-sell or upsell pairing, test an MMS version of the message (i.e., one that includes an image or GIF) against a plain SMS version. Or, test a discount offer against a free gift offer. As Dakik notes, “The opportunities are endless!”

Automations to prioritize when planning out your A/B tests include:

• Cart Abandonment

• Browse Abandonment

• Welcome Series

• New Subscriber/No Order 

• Post Purchase 

Be thoughtful in your approaches. Subscribers are used to blanket discount incentives, so this is your chance to find out if other value props that resonate better.

Use these automations to test different offers, messaging creative, and delivery timing.

Okay. Time to maximize that revenue, y’all. 

Follow the steps outlined in this help article to set up your tests and review their results. Questions? Hit us up using the purple chat icon at the bottom of this page, or email support@postscript.io.