November 15, 2018
This article is part of our ongoing Mobile Optimization Initiative series designed to help Magento merchants close the mobile revenue gap.
At Something Digital, we spend our days helping our clients boost their sales, improve revenue per visitor (RVP), and increase average order value (AOV). It’s why we’re such strong advocates for A/B testing.
When Magento asked us to participate in its Mobile eCommerce Optimization Initiative we jumped at the chance. As a Magento Community Initiative, this gives us the opportunity to pool our resources and to test a wide variety of factors that have an impact on mobile commerce.
The initiative is ongoing, and so are our results. That said, this is a great opportunity to show our clients how the Mobile eCommerce Optimization Initiative works, the kinds of things we’re testing, and a preview of some of the results.
Treatments We Tested
Working with TheInsoleStore.com, we A/B tested five new treatments (i.e. changes to the existing web pages) to see what, if any, impact they had on conversions, RPV, and AOV. We partnered with PayPal to track the value of orders placed, and HiConversion, a customer experience analytics, and optimization company, to give us the data about their sites and visitors—regardless of device—which allowed us to act on what we learned in real time.
We also wanted to test another hypothesis: If you improve mobile, you improve the other channels. For this reason, we opted to A/B test these changes on desktop as well as mobile phones and tablet.
We’ve tested several treatments. For the purposes of this blog post, we will report on:
Addition of a security lock icon on the checkout page
Cart header suppression
Checkout coupon collapse
We have the most data on TheInsoleStore.com, so these results focus on that retailer.
Security Lock Icon On Checkout Page
Treatment: We added an icon to the checkout page in order to impart a sense of security.
The goal of this treatment was to impart a sense of security in the checkout among site visitors which would then lead to an overall higher conversion rate, which was true for TheInsoleStore.com.
Overall, traffic from desktop and mobile constituted the majority of visits which caused RPV to be positive for this treatment.
We learned that safety and security is a value for TheInsoleStore.com customers and this can be tested elsewhere throughout the site.
Cart Header Suppression
Treatment: Eliminated the navigation options in the header of the checkout process. The theory is that if users don’t see these options, they’ll have less temptation to abandon the cart.
Performed well on tablet. Tablets are often an outlier in test results. Users tend to be older, which may or may not have been a driver.
AOV was down for desktop, perhaps because users were unable to return to shopping to add more items to their carts.
Checkout/Cart Coupon Collapse
Treatment: We hid the coupon field, based on the theory that seeing it tells users there are coupons available, which encourages them to leave the checkout process to search for one. Or, seeing the coupon field leads them to believe they’re not getting the best possible deal, causing them to abandon the cart. Note: the coupon field was still available it was positioned lower on the page so it was less noticeable.
Desktop and tablet results make a compelling case to roll out the coupon collapse more broadly
Mobile phone results are inconclusive in that the coupon field isn’t as prominent on page load as it is on desktop and tablet. This means it makes it difficult to compare the treatment’s impact on mobile to desktop and tablet.
Mobile optimization focuses on conversions, revenue, and order size. Tweaks here and there can boost your revenue, but it’s important to note that customers will buy from you once you understand how they want to engage with your brand and which problems you’re solving for them. Take the time to understand your customers, and you will know how to tweak your site to win their business.
Keep in mind that when a test fails—i.e. a treatment fails to deliver a positive gain in conversion, RPV, or AOV—you’re learning something about your customers. Therefore, that test isn’t a failure, it’s a learning tool.
Additionally, as we look at results across retailers, not just TheInsoleStore.com, we see that there is no “one best experience” that works across all brands. Every brand is unique and should test various hypothesis with their own customers.
About Caitlin Mekita & Something Digital
Something Digital is an interactive agency headquartered in NYC with additional offices in West Palm Beach, Los Angeles, and Seattle. We are a full-service agency that does everything from discovery, design, development, deployment, ongoing maintenance, digital and email marketing with eCommerce sites. We differentiate ourselves with creativity, business acumen, and a keen awareness of eCommerce best practices. Furthermore, we pride ourselves in delivering and maintaining outstanding sites with unique brand identities and simplified user experiences. Caitlin Mekita is a certified Solution Specialist and Strategic Engagement Manager for Something Digital’s Strategic Engagement Group. She takes a hands-on approach in helping clients grow and maintain their Magento stores in collaboration with a talented team of developers at SD. Caitlin comes from a creative background and is passionate about building a competitive edge through customer experience.