The Marine Mammal Center (TMMC) is a marine mammal hospital and science research facility in Sausalito, CA. Since 1975, they have rescued nearly 18,000 seals, sea lions, dolphins, otters and other marine mammals from along 600 miles of California coast, with the aim of giving those in need a second chance at life in the wild.
The Marine Mammal Center has a strong retail component. As any facility with visitors should, it has a gift shop which sells intangible support items (Adopt-a-Seal) as well as very tangible ones, like warm socks with various marine mammals on them. It also has a basic e-retail store (housed and managed within their Convio website)
For two years the marketing team has experimented with featuring e-retail at various points during the calendar year, or with various audiences. Some discoveries we’ve made historically include:
- People on lists we’ve acquired seem more willing to buy something than to give a cash donation. Knowing this little nugget, gives us a perfect way to ease them into supporting us further down the line. (Also, marine mammal socks are cool.)
- There are only three times a year where we can generate a surge of orders! We spent a whole year trying to pitch our audience e-retail in connection with every conceivable holiday, like Labor Day, with hilarious and often disappointing results but we found pretty conclusively that besides Hannukah and Christmas, the effective hooks for e-commerce purchases are Valentine’s Day and Mother’s Day, and every so often a random ocean /conservation related e-retail does pretty well, i.e. Earth Day, World Ocean’s Day, Sea Otter Awareness Week.
- We can maintain a steady stream of e-retail orders by adding one item to a normal e-news alert or latest news email. Making the commitment to include and promote one item in at least two emails per month is s a cost-efficient and effort-efficient strategy.
As we approached Valentine’s Day 2013, we decided to try an experiment with our last Valentine’s Day e-retail email before the shipping deadline. We split our list in two (we use Convio Luminate) and sent two emails with different leads.
The original and the variant
Our original format had an animal story at the top with a charismatic photo of the patient.
The variant instead substituted a product as the lead:
The results
Both emails were sent as part of an A/B test to half of our housefile. They got almost the exact same open rate (15% - on the low to average end of our e-retail emails). Of those that opened either version, about 11% clicked through to various places on the website. The difference comes in results. The original generated 8 purchases in our e-retail store, whereas the variant with the product at the top generated 14 purchases and 2 donations, for about double the revenue vs. the original.
When we computed statistical significance we found that statistically it only reaches a 90% confidence level and we generally look for 95% confidence in A/B tests. However, it’s a strong indicator, so we plan to repeat the experiment with a different product and a new animal story to see if we get similar results.
There’s one weird quirk to all of this as well: the original version with the animal at the top got almost 150 more forwards than the one with the product at the top. And the original, once forwarded, generated 31 extra clicks from friends-of-friends onto e-retail product pages.
At first we were excited about this, and then we realized that for all this forwarding, it still did not generate enough additional revenue to make the original as compelling as the variant – it didn’t motivate people to buy or give at that point in their e-retail experience.
As a rule, we try and put an A/B test into every single piece of large-segment email we send out. Tell us in the comments what sorts of A/B tests you’ve run on your e-retail emails and what results you’ve seen.
PDFs of each full email can be seen here and here.
Kate Harle and Dale Anania are the core marketing staff at The Marine Mammal Center, and Shabbir Imber Safdar (of Safdar Analytics) is their analytics consultant.
COMMENTS