Note: This is a guide for beginners. More advanced SEO testing content can be found in the resources at the end of this post.
Intro
If you’ve found this resource, there’s a reasonably good chance you’re ready to level-up your SEO game.
Maybe you’re tired of the same old-hat tactics that everyone keeps talking about: internal linking, content, technical, offsite SEO, and the like. Or maybe you’re still driving success in these areas, but you’re working on a website with diminishing returns and you’re looking for a new competitive edge.
Welcome to the exciting world of SEO testing.
The not-so-secret (but vastly underutilized) technique that my teams have found for that extra competitive edge.
Why we test
In short, most SEO testing is designed for performance enhancement. Whether it be traffic, keyword rankings, or conversion activity, performance is often the primary aim of SEO experiments.
But SEO testing is about far more than performance.
In every SEO test, there’s a hypothesis behind the experiment. And where there’s a hypothesis, there are unknowns.
SEO testing helps us illuminate those unknowns so that we can iterate toward smarter strategies.
Thus, developing a clearer understanding of what Google’s algorithms are / aren’t rewarding makes you a better SEO professional.
The NOS analogy
My favorite way to think about SEO testing is with a little Fast & Furious analogy. Remember when Paul Walker hit that little red button to send his Toyota Supra Mark IV into warp speed? For anyone who spent the 90s under a rock and didn’t see the movie, that little red button was a NOS injector. Hitting the NOS button adds nitrous oxide to the car’s fuel system, which supercharges the car’s top speeds above the max speed of its core combustion engine.
If your content and links are fuel to the SEO engine, SEO testing is your NOS.
Ever had a page or post get stuck on page 2 or bottom of page 1 for high-value search terms? Most often, you’ll hear SEOs recommend more links, but taking the link building approach can be expensive and risky.
What my teams have found in our own SEO testing efforts is that we can take our content’s baseline performance (main combustion engine) and we can supercharge it with SEO testing (NOS) to surpass the top-end performance potential and crack our way into the top 3, 5, or 10 positions.
And we could do it with less time and resources than any of the link building campaigns we’d tried in the past.
SEO testing basics
If you’ve been in the field of digital marketing for any amount of time, you’re likely familiar with CRO (Conversion Rate Optimization), and you probably have a fundamental understanding of the A/B testing process. Or, if you’re a seasoned SEO, then you may even have run your own time-based SEO tests.
We’ll cover more advanced SEO testing concepts that go beyond the basics, so if you’re a high-skill SEO professional, you may want to check out some of my other material. For now, this article will stick to the fundamentals.
Getting started with your first SEO test, the simple way
There are some digital marketing pros out there who would have you believe that there’s no point in running an experiment “the wrong way.”
This is gobbledygook.
All that’s really required to run your first SEO experiment is an innate sense of curiosity, and a desire to figure out which optimizations will / will not work. Well, that and you should probably have a website with basic analytics capabilities.
If your experiment isn’t perfectly-executed the first time, you’ll learn how to make it better the next time!
The point here is just start trying things, which is the very undertone of the word, “testing.” So, for your first experiment, here’s the most basic process that you can possibly run.
Step 1 – Establish a clear goal & attach it to a primary KPI.
To run a good, solid SEO experiment, it’s fundamentally important to establish a clear goal that maps to a primary KPI.
Ask yourself any number of the following questions:
Am I trying to increase rankings?
Am I trying to grow traffic?
Am I just trying to see how Google would react to one of my changes?
Establish a goal, then get clear on the KPI that will show you whether or not your experiment was a success.
Step 2 – Use your goal to build a hypothesis & define the scope.
To keep this part simple, use this formula:
Changing _X_ on my website will result in _Y_ outcome.
In most experiments, you’ll need to do a bit of SERP analysis in order to come up with a sound hypothesis and improve your chances of success. SERP analysis is a fundamental part of the hypothesis ideation process in most SEO experiments, so if your goal is traffic & rankings, do invest your time with SERP analysis for better hypothesis-ideation.
Your hypothesis should also help you define the scope of your experiment. Are you going to be running a test site-wide? On a page group? Or on a single page?
I recommend running a single-URL title test for your first experiment.
Step 3 – Check that you’ve got historical data in GA, GSC or your measurement tool of choice.
No need to get fancy with benchmarking techniques, or statistical forecasting on your first experiment.
You should, however, make sure that you’re using an analytics solution that is actively recording historical data for whichever KPI you’ve selected in Step 1 because you’re going to use this historical data when you go back to check on your results down the line.
So, if you’ve chosen organic traffic as a primary KPI, Google Analytics should do the trick. If you’ve chosen CTR, Google Search Console has your back.
Where it gets even harder is when keyword rankings are your KPI. If your goal is to improve keyword rankings. If rankings are your primary KPI, I do recommend benchmarking your rankings at the time of implementation.
Step 4 – Implement the change!
You’ve got your KPI, your hypotheses, and your historical data.
Now, all you’ve got to do is change the element (or elements) of your website as defined in your hypothesis and scope.
Important: As soon as you’ve implemented the change, make sure to annotate the date that your change went live in GA, or elsewhere. You’ll be surprised how easy it is to forget which date you’ve launched your experiments.
Step 5 – Wait, monitor, & measure your results.
If you’ve already run SEO experiments in the past, then you may already have a good sense for the time it will take to see your changes start to materialize in the form of KPI impacts. But, if this is your first experiment, then you’ll probably want to suss out your time durations through some trial and error.
This is because the time that it takes to start seeing measurable impacts in your data will vary by the website, the traffic levels, and the elements being tested.
Set a recurring calendar reminder and check back in on your KPI performance at least once per week.
Once you feel that you’re seeing a reasonable amount of data, you can do a simple “before and after” comparison.
For a simple before and after comparison, measure the amount of time that’s passed between your launch date and the current date. Then, use that time duration to gather historical data for the same number of days / weeks before your experiment, as well as gathering the historical data for the number of days after your experiment. If possible, I recommend using the same days of the week in your “before” dataset, as you would have in your “after” dataset as days of the week could influence your metrics significantly.
Let’s say that you started your experiment on a Friday, ran it for 32 days, and ended on a Monday. If you pull a “before” dataset that displays the exact 32 days prior, your “before” data would have started on a Monday and ended on a Thursday.
Assuming weekend days produce less traffic than weekdays, you would have had a “before” dataset with 8 weekend days, and an “after” dataset with 10 weekend days, which would inherently skew your results toward a bias with more weekday traffic in the after dataset.
To solve for this, simply check which days of the week you ran your start and end dates on, make sure there are no holidays, or outliers in your datasets, and then select a “before” dataset that lines up a little more closely with the same days of the week that you used for the experiment start and end dates.
Additional Resources:
As you begin to level-up your skills following your first SEO test, you can also check out the following resources to help you build up to more advanced SEO experiments.
