One of the most common questions people have when the topic of SEO testing arises is, “can’t you just use a tool like VWO/Google Optimize/Optimizely, etc. to run SEO experiments?”
TL;DR – No. Because, Google.
It seems simple in theory, right?
If you want to run an A/B test for SEO, just do the following:
- Head over to your CRO tool of choice, be it Google Optimize, VWO, or Crazy Egg.
- Set up a variant for one of your most prized pieces of SEO content.
- Launch the experiment and see which page variation performs better in Google Search.
Easy peasy!
Here’s why it’s a little more complicated than that.
Reason #1 – CRO and SEO aren’t the same
Breaking news to no one.
The first thing to keep in mind here is the most fundamental difference between what we want to measure in an SEO experiment versus what we want to measure in a CRO experiment.
When you measure a CRO experiment, you’re going to be focused on conversions, and you’ll be measuring accordingly.
In SEO, although we can sometimes measure success down to the conversion level, the way that we’re driving those conversions is by generating organic traffic growth.
CRO tools don’t measure organic search traffic, but it’s more than that.
Reason #2 – URL parameters
Just in case you as the reader aren’t already familiar with the way that CRO tools work, here’s a very basic explanation of the process.
- You add the CRO tool’s script to your website and identify a page with already-substantial traffic levels that you want to run an experiment on.
- The CRO program creates a URL parameter so that it can display two separate versions of your web page (the control and the variant)
- Example: http://www.yourwebsite.com/v1/?test=1
- or http://www.yourwebsite.com/v2/?test=2
- As traffic flows into your site, the tools send 50% of the users to the parameterized control page, and the other 50% of users see the parameterized variant page. (these settings can be adjusted)
The problem?
Google doesn’t play very nicely with URL parameters.
Much could be said about the challenges that parameterized URLs face with Google, but the primary problem here is that parameterized URLs tend to generate multiple versions of the same page, whilst Google needs to identify a single canonical URL in order to give preference to the correct, or “main,” piece of content.
So, even though Google is capable of handling the parameterized URLs, they are often seen as duplicate.
Reason #3 – No way to control for equal sample sizes
Now, let’s assume that somehow, some way, we did manage to achieve the highly unlikely task of getting both our A versions and our B versions to rank on the first page of Google so that both iterations can acquire search traffic. Even though this is highly unlikely, it is theoretically still possible, so let’s entertain ourselves for a moment.
If, we were to achieve this, then the two pages would either appear as indented search results:
Or, they would appear in separate ranking positions.
Either way we slice it, the traffic volumes that would come through each of the two page variations would be unequal.
And that’s not accounting for the differences in each page variation’s long-tail traffic.
Reason #4 – CRO tools can’t measure Organic Search performance
The last shortcoming that CRO tools present when it comes to SEO split testing is that they lack the ability to segment organic search performance apart from other traffic sources, and they don’t have the measurement frameworks needed to account for the contrasting differences in process that SEO’s use to run experiments.
I will cover some of these differences in future posts. Some of the testing measurement techniques that SEOs employ include processes such as:
- Time-based forecasting & measuring
- Daily average clicks gap measuring
- Inferred causal impact
Solution – SEO split testing tools & techniques
To run viable A/B tests for SEO (A.k.a. SEO split tests), we need more URLs than just one.
A LOT more URLs.
And, we need those URLs to be as identical as possible, minus a few page variables. These are known as programmatic pages, or templatized pages.
With websites that have programmatic pages, SEO split testing takes each group of pages and “splits” them into equally-yoked control groups, and variant groups so that both groups can be indexed and analyzed somewhat equally.
Now, you’re probably wondering about SEO testing tools right about now, aren’t you?
These tools are indeed out there, and available to the savvy SEO scientist.
They are, of course, more niche and specialized than your average CRO tool, which is exactly why SEO testing is in a league of its own.
