The 6 Main SEO Testing Techniques for Organic Growth

SEO testing. Seems straightforward doesn’t it? You start with an idea. You test that idea. Then, you see if your idea worked or didn’t work.

But as with so many things in SEO, things are not always as they seem.

SEO testing can take on many different forms. Faces, if you will. Each face is a testing method designed for different environments, different hypotheses, and different goals.

This article is an attempt to make sense of SEO testing methods, so that you can hone in on the types of experiments that suit your objectives.

The 6 SEO testing techniques that lead to better rankings are:

Individual time-based testing. (Also called, “single-URL experiments.”)
Group testing.
SEO A/B testing. (Also known as “split testing.”)
Bulk-URL testing.
Fake keyword experiments.
Custom experiments.

How to conduct a simple SEO test

Identify an SEO opportunity for testing on a primary keyword.
Analyze the search engine results page (SERP) for your primary keyword.
Build a clear & measurable hypothesis for your experiment.
Benchmark current performance data.
Back up a copy of your page before implementing the test hypothesis.
Implement your experiment hypothesis.
Publish the variant page.
Resubmit the URL in Google Search Console.
Allow the test to run for at least 2-3 weeks.
Analyze your results by comparing the benchmarked data against the new data collected during the SEO test.

What SEO testing is not…

Before we get into the most-common SEO testing methods, we need to address one great big elephant in the room.

SEO testing is NOT the same thing as a CRO-esque A/B test.

You can’t run a concurrent A/B split test on just one individual SEO page.
You can’t use the same A/B testing software that a CRO professional might use (Google Optimize, VWO, Hotjar, etc.)
Plus, many SEO experiments are more difficult to build controlled environments for, which means that some amount of ambiguity can influence your confidence intervals.

So this…

Doesn’t really work the same way for SEO as it does for CRO.

So, can you still run A/B tests for SEO?

Yes. Yes you can.

You just can’t run two variations concurrently on a single page.

Why not?

Because in SEO, we’re testing to see how Google responds and we can’t force Google to index two landing page variations at the same time for the same keyword sets.

In other words, Google is our only user, which means we can’t send 50% of the search engine to one page and 50% of the search engine to another page.

So, if you’re aiming to run an A/B test for SEO purposes, there is a distinct method that I’ll cover for you here in method #3, A/B split tests for SEO.

Technique #1: Individual time-based testing. (Single-URL)

This is the first, and most common method that people tend to start with when running their first SEO experiments. I cover this process in-depth here in my SEO title testing guide.

Important to know about single-URL experiments is that these are generally more difficult to control, which means that our confidence scores are going to be less reliable than they would be in a more controlled testing environment.

That means that our results can sometimes err more on the side of correlation than causation.

Some SEOs might say that single-URL tests aren’t valid experiments due to the lack of control and the unreliability of your data when so many variables can influence the results for a single-URL test. But take the following scenarios. Are these not worth running?

A single-URL experiment moves your primary keyword position from page 8 to page 1.
A single-URL experiment increases your organic traffic from 12 clicks per day to 100 clicks per day.

In my experience, these kinds of tests have been ROI-positive, despite their lower confidence intervals. So how does a single-URL test work?

The oversimplified process for single-URL experiments is this:

(Read the full process here)

You identify an opportunity to improve a page’s rankings and traffic.
You analyze the SERPs and build a hypothesis to help that page achieve the desired outcome.
You launch a new version of the page that reflects your hypothesis.
Then, you measure the page’s performance using a time-based method that compares the data before the optimization to the data after the optimization.

These kinds of SEO tests may not always be the most scientific way to experiment, but they are still a very powerful technique for traffic growth and they’re a great place to begin learning how to conduct SEO experiments.

Technique #2: Group experiments

Moving up in our SEO testing repertoire, you might come across group experiments.

Group experiments follow the same basic process that we use for single-URL experiments, except that they produce better confidence intervals because you can run your hypothesis on a large group of templatized web pages.

More pages = more data = higher confidence interval.

The requirement for running a group test is that the pages in your group must be templatized (usually programmatically-generated). So you can’t run a group test on the company blog, for example, where each page is unique.

Better use-cases for group testing could include product pages on an eComm site, resort listing pages on a travel website, or any other group of programmatic web pages that have been built on the same uniform page template.

Group tests are good, but they could be a lot better if you’re able to deploy this next testing method.

Technique #3: A/B Split Tests

At last, we come to the A/B testing technique that actually works for SEO.

SEO A/B testing (a.k.a. “SEO split testing” or just “split testing”) is the grand-master technique that the pros use to achieve better confidence intervals, more statistically significant data, and a higher probability of causation.

So, why aren’t all SEO experiments run this way if the A/B split testing technique is so amazing?

Mainly because split testing can only be executed in the right environments.

In order to run SEO split tests, you’re going to need:

A good number of programmatic (templatized) pages that are already generating substantial organic traffic volumes.
A specific CMS configuration or tool that allows you to group and edit some of the pages independently from the rest of the pages (to launch a B group).
A program to collect and measure the data from both the A group, and the B group over time.
A trained professional who is capable of the setup and measurement process.

Without these crucial pieces it’s going to be very difficult to attempt SEO split testing, which is why it can’t be performed on most websites, even though it has my vote for the coolest experimentation technique. 😎

Now, assuming that you’ve got all the pieces in place, here’s the SEO split testing process in a nutshell.

You’ll come up with a strong hypothesis that can be applied across your programmatic pages.
Split the templatized URLs into two page groups based on traffic levels.
- Critical note: you should not split the page groups right down the middle, based on the number of URLs. If you have 1,000 URLs to test with, you might assume that the “A group” should have 500, and the “B group” should also have 500, but that would be an error because the metric that you use to measure results with (traffic) is probably not going to be evenly split down the middle. Instead, make sure to balance your A and B groups so that traffic is split evenly across the middle as a starting-off point.
Set up your measurement tool(s) so that the A and B groups are accurately distinguished and connected to the data source.
Apply the changes to all URLs in the “B group.”
Allow the experiment to run until it reaches statistical significance.
Measure and report on your results before applying the winning results to all URLs.

Technique #4: Bulk-URL testing

This method is much lesser-known, but it’s the one that I’ve been dedicated to building out in my continuous SEO testing program.

The premise of bulk SEO testing is simple, but it requires specialized tool sets to pull off, which is why it hasn’t reached mainstream popularity yet.

Bulk testing allows us to run dozens, or hundreds of single-URL experiments across a large array of non-templatized URLs.

It’s different from group testing and split testing in that bulk testing doesn’t require the pages to be programmatic, or templatized, so it can be executed on any website that generates substantial amounts of revenue and traffic from organic search.

Technique #5: Fake keyword experiments

This one is really cool in my opinion, and it’s one that I first learned about from Kyle Roof.

Although these experiments are not designed for direct traffic gains like the first 4 methods, fake keyword experiments are a powerful way to see how Google’s Algorithms respond to specific hypotheses and ideas.

If your main goal is to isolate variables and learn exactly how Google’s algorithms will respond to a hypothesis, running a fake keyword experiment is your best bet. From there, you may walk away with a substantial-enough takeaway that you can apply to your SEO strategy.

The basic idea for running fake keyword experiments looks like this

You have an idea that you want to test out.
You acquire a domain that’s based on a totally-fake / made-up keyword that no one else is searching or ranking for. (This is how we control the experiment and remove external variables, like competitor rankings from the equation).
You add pages to the fake-keyword domain using Lorum Ipsum.
Index the nonsense pages by submitting them to Google Search Console.
Begin testing your hypotheses.

Kyle also provides a more detailed breakdown of this process here.

Technique #6: Custom experiments

Last, but certainly not least, there are a sh*t ton of custom experiment techniques that you can design for testing SEO hypotheses.

This post covers what I believe are the most common and highest-impact experiment techniques in SEO today, but I’ve also come across some very innovative and informative experiments that don’t fit neatly into any one category because the hypothesis required a more customized way to control environmental factors and/or measure results toward a confident conclusion.

If I’ve missed any techniques that you’d like me to add, please drop me a message and reach out to me so that I can add them to this article.

Honorable Mention: Technical SEO QA testing

I didn’t include this form of “SEO testing” in the above list because I believe it’s really a whole separate classification that should be distinctly separate from growth experiments.

Technical testing, or QA testing, is an automated form of QA’ing SEO performance issues and/or metrics on a site.

Technical SEO testing software can be installed on a website and automatically crawl, or detect issues related to site speed, 4XX errors, internal linking issues, malware, missing meta tags, image issues, etc.

Due to the fact that this form of “SEO testing” is more of an automated QA activity, technical testing is less impactful for new traffic acquisition, but it can be very valuable in preventing traffic loss from site issues. I suppose the confusion here is just a pitfall that comes with a dual-meaning word, like “testing.”