When you hear the word “optimization,” what do you think of? Possibly optimizing your sleep schedule so that you feel more rested, or adjusting your diet to lose weight. All these examples clearly have cause and effect making it simple to see results. But what about optimizing something more complex like product development or marketing campaigns? How do you measure improvement, when there are thousands of possible factors at play? Before you say that it’s impossible, let me show you some techniques used by marketers to achieve measurable results from Conversion Rate Optimization.
Conversion Rate Optimization, What is it?
Let’s review what Conversion Rate Optimization (CRO) is. As put simply by my Professor and Digital Marketing expert Mark Staton, “It’s all about having a goal in mind for your site or app, and then making it easier for people to achieve it.” For example, let’s say you are running an email campaign, with the goal of leading people to your site so they can see your product. CRO is making changes to that email so the process becomes smoother for the target audience. You change the size of the “shop now” button and notice that all the sudden clicks increase from 10% of users to 15% percent. This is an instance of CRO done right!
Measuring CRO with A/B Testing
Now that we have a better understanding of what Conversion Rate Optimization is, let’s finally look at how we can measure it. While there are many techniques to do this, today we will be discussing A/B testing. This method compares two different versions of what you’re testing to see which one performs better. Revisiting our “shop now” button from earlier, to see if the change in button size really was the cause of the 5% conversion rate increase, lets design an A/B test.
- Start with the original email (Email 1: The Control)
- Make a copy and size up the button (Email 2: The Treatment)
- You should have your original email (Email 1) as well as a copy with a sized up button (Email 2)
- Randomly divide your mailing list so that half are presented email 1, while the other half are presented email 2
After a week if email 2 is experiencing higher clicks on the shop now button, then congratulations, you have successfully optimized an aspect of your promotion! Now let’s take a look at this process through another lens. This time we are going to be testing the “add to cart” button on an e-commerce site.
Each listing on this site has an “add to cart” button next to the item description. Although your boss insists it’s perfect the way it is, you think that adding another button next to the “add to cart” that says “buy now” will increase sales. After finally convincing them to let you test it, you set it up and it goes as follows:
- You open the site (The Control)
- You make a copy and add the “buy now” button (The Treatment)
- You program the website so that half the visitors are presented with the original site (The Control) and the other half are presented with your updated site (The Treatment)
- After a week of running the sites you see that your site (The Treatment) has a 7% increase in conversion rate than your bosses (the Control)
You see this, calmly walk into their office, show the changes, they congratulate you, give you a raise, then you walk out of the building, get to the end of the parking lot, then pump your fist in the air like Bender at the end of the Breakfast Club.
As you can see, it’s a simple way to ensure that changes you make are actually having an effect, however there are some things you need to look out for to make sure it remains accurate. Let’s take a look at those now:
- Time Period: Always perform A/B tests simultaneously, comparing Christmas sales to mid January sales will obviously give biased results.
- Statistical significance: Try and keep changes between versions small. Ideally no more that 5% should be changed to best understand what is causing the differences.
Not Just In The Text Books: A/B Testing in Practice
By this point those of you just learning about A/B testing may be thinking “wow, that’s cool and all, but does anyone successful actually use this”, or “Is this just another dated strategy textbooks repeat year after year”. Instead of trying to convince you otherwise, let’s take a look at some large companies that utilize this very testing, all while experiencing strong performance.
- Automakers like Toyota, Porsche, and many more use A/B testing for many aspects of their vehicles ranging from body design all the way to the internals of an engine (Rozange, 2025).
- Using A/B testing, Amazon found that adding a “continue as guest” option at checkout increased sales by 45%, or $300 million in revenue in the first year (Fulton, 2025)
Wrapping it up
By now, we see the importance in Conversion Rate Optimization and it’s a measurable approach to understanding what really works and what doesn’t. Whether it’s testing the size of a button or changing the suspension in a car, CRO helps marketers form insights to turn small changes into meaningful results.
If you enjoyed this post and want to learn more about marketing strategies and optimization techniques, check out my website for more blog posts!
Leave a Reply