Global Homepage Banner Boosted Email Sign-Ups
Role: Product Designer
Client: Thrifty Traveler
Problem: Visitors were not signing up for the Daily Beat email list during their browsing session, leading to missed engagement opportunities.
Solution: Increase email signups by making the email signup visible and accessible during the browsing experience.
The A/B Testing Process
Why We Decided to Test a Global Homepage Banner
This test is part of an ongoing effort to iterate and optimize email sign-ups, a key strategy for increasing user engagement and conversions. In 2023 and 2024, Google significantly updated its SEO algorithms with a series of core updates, emphasizing user intent and high-quality content. These changes have reshaped how users discover and interact with websites. As a result, the client aims to capitalize on this evolving landscape by boosting subscriptions and engagement through targeted email sign-ups. We’ve been exploring ways to make the sign-up process more visible and compelling, and the pencil banner with a clear CTA represents our latest approach to improve visibility and drive higher sign-up rates.
Phase 1. Test Plan & Hypothesis Development
To address the ongoing challenge of low email signups, we identified an opportunity to make the sign-up process more visible and appealing. After conducting user research and aligning with the client on this key focus area, we refined our hypothesis and began crafting a detailed test plan.
Hypothesis: We believe, that adding a prominent pencil banner across the website with a clear CTA ("Subscribe for Cheap Travel Tips") will increase Daily Beat email signups.
The Test Plan also detailed the following:
Metrics to Measure Success: We defined the key performance indicators (KPIs) such as Daily Beat Signup Events, Subscribe Form CTA Clicks, and Average Engagement Time.
Tracking Plan: We worked with our analytics team to set up tracking systems to monitor user interactions and ensure data accuracy.
Audience Segmentation: We clarified the segments of the audience that would be included in the test to ensure we were targeting the right users and keeping results statistically valid. Specifically, we segmented the audience into new vs. returning users and mobile vs. desktop users to evaluate how different groups interacted with the banner and how device types influenced sign-up behavior.
Phase 2. Design & Development
Design Decision: The design of the banner was kept simple yet effective. We chose a pencil banner and paired it with a straightforward message: "Get cheap travel tips in your inbox." Next to the text, there was a clear and simple "Subscribe" button with an email input field.
The design needed to be versatile, so we created two versions:
Desktop Version: The banner was positioned at the top of the page, spanning across the full width of the site to catch users' attention as soon as they landed on the homepage.
Mobile Version: We designed a mobile-friendly version that adapted to smaller screens. Given the limited real estate on mobile devices, we focused on ensuring the banner was still prominent without interfering with the user experience.
Desktop Email Banner
Mobile Email Banner
Phase 3: Testing
To properly assess the effect of the banner, we split the homepage traffic into two groups:
Variant 1: This version introduced the pencil banner with the message, “Get cheap travel tips in your inbox.” and a simple "Subscribe" button, paired with an email input field.
Control: This was the original homepage, which had no banner or prominent call to action for email sign-ups.
We set clear success metrics: We wanted to measure the conversion rate of visitors who signed up through the banner compared to those who interacted with the original homepage layout. To do this we tracked key metrics such as CTA clicks, average engagement time, and daily beat sign-up events to assess overall user engagement and the banner’s effectiveness in driving conversions.
Desktop Control
Desktop Variant: Email Banner
Mobile Control
Mobile Variant: Email Banner
Phase 4: Analysis & Results
Analysis: After the test concluded, we analyzed the data to compare the performance of the banner variant against the control. Key metrics, including Daily Beat Signup Events, CTA Clicks, and Average Engagement Time, were evaluated to determine the impact on conversions and user interaction. The results revealed valuable insights into user behavior, highlighting the effectiveness of the banner in driving sign-ups and increasing engagement across different segments.
Results:
141% increase in Daily Beat sign-ups across all visitors with the pencil banner.
Desktop visitors were 216% more likely to sign up than control, while mobile visitors saw a 76% increase.
Additional Insights:
The banner increased interest in the Premium subscription by 7.3% among desktop users compared to mobile users.
Why? Desktop users showed higher engagement with the banner, likely due to its clearer visibility on larger screens. The banner's placement was more prominent on desktop screens, making it easier for users to notice and interact with, leading to a higher conversion rate compared to mobile users who faced visibility issues behind the hamburger menu.
Key Takeaway: The test was successful, leading to higher Daily Beat sign-ups. Further recommendations include exploring optimizations for mobile visibility.
Conclusion & Next Steps
Conclusion: The pencil banner significantly increased sign-ups for the Daily Beat, proving the hypothesis that visibility and a direct CTA would encourage user engagement.
Learnings: Design solutions should be tailored to device-specific user behaviors to maximize impact.
Future Testing Recommendations:
Implement a sticky banner for mobile users.
Test additional content teasers (e.g., what type of content users can expect when signing up for a specific email list) to increase engagement and email subscriptions.