When Should You Prune A Variant

You should prune a variant when it’s no longer performing, when it’s causing confusion, or when you’ve gathered enough data to prove a clear winner.
 
Pruning a variant means removing it from your active testing rotation, and it’s a crucial step in any serious A/B testing workflow.
 
If you’re not sure when to prune a variant, you’re probably leaving conversions on the table or wasting traffic on losing ideas.
 
In this post, we’ll cover the key moments to prune a variant, how to identify underperformers, and why pruning is just as important as launching new tests.
 
Let’s dig into the details so you can make smarter, data-driven pruning decisions.
 

When to Prune a Variant: The Core Timing Rules

The timing of when to prune a variant depends on three main factors, and getting these right can save you weeks of wasted effort.
 
You prune a variant only after you have statistical significance, but that’s not the whole story.
 
You also prune a variant when it’s clearly losing, when it’s plateaued for too long, or when outside changes make it irrelevant.
 
Let’s break down each scenario so you know exactly when to prune a variant in your own testing routine.
 

1. Prune a Variant Once You Hit Statistical Significance

The most straightforward time to prune a variant is when your test reaches statistical significance and a clear winner emerges.
 
If you’re running a 95% confidence interval and your variant is outperforming the control by a meaningful margin, then it’s time to prune the losing variant.
 
You shouldn’t prune a variant early just because it looks like it’s losing, though, because small sample sizes can mislead you.
 
Wait for the numbers to stabilize, and only then prune a variant that’s losing with confidence.
 
Once you prune a variant, you can redirect that traffic to the winner and start planning your next experiment.
 

2. Prune a Variant When It’s Consistently Underperforming

Sometimes you don’t need to wait for full statistical significance if a variant is just terrible from day one.
 
If your variant has a conversion rate that’s 20% or more below the control after a few thousand visitors, that’s a strong sign.
 
You can prune a variant like this early because the likelihood of it recovering is very low in most cases.
 
But be careful, you still want a minimum sample size to avoid false negatives.
 
A good rule is to prune a variant after you’ve collected at least 80% of your planned sample, unless the underperformance is extreme.
 
In practice, you prune a variant when the trend is unmistakable, even if you haven’t hit that final confidence threshold yet.
 

3. Prune a Variant When the Test Has Plateaued

Another key moment to prune a variant is when your test results have flatlined for an extended period.
 
If you’ve been running for two weeks and the conversion rates aren’t moving at all, that’s a sign to prune a variant.
 
Plateaus usually mean the variant isn’t going to produce a winner, so keeping it alive just wastes traffic.
 
You can prune a variant when it’s been stuck at the same performance level for more than seven days.
 
This is especially true if you’ve already reached a decent sample size and the numbers haven’t budged.
 
Pruning a plateaued variant frees up space for fresh ideas that might actually move the needle.
 

4. Prune a Variant Due to External Changes

Sometimes the reason to prune a variant has nothing to do with your test data at all.
 
If your website undergoes a redesign, a season changes, or a major algorithm update hits, that can invalidate your test.
 
In those cases, you should prune a variant immediately because the context has shifted.
 
You prune a variant that was designed for a different homepage layout or a different user intent.
 
It’s better to prune a variant and restart your test under new conditions than to stick with outdated assumptions.
 
External disruptions are a classic signal that it’s time to prune a variant and reassess your entire testing strategy.
 

How to Decide Which Variant to Prune First

When you have multiple variants running in the same experiment, deciding which to prune can feel tricky.
 
You don’t want to prune a variant that might catch up, but you also don’t want to keep dead weight alive.
 
The best approach is to rank your variants by their current conversion rate, expected value, and trend direction.
 
You should prune a variant first if it has the lowest conversion rate and a negative trend over the last five days.
 
You also prune a variant that has a high bounce rate or a low engagement metric compared to the control.
 
The goal is to prune a variant that’s clearly dragging down your overall test efficiency.
 
Focus on pruning the worst performer first, then reassess the others after you’ve freed up more traffic.
 

1. Use Bayesian or Frequentist Metrics to Guide Pruning

You can use statistical tools to help you decide when to prune a variant with more precision.
 
Bayesian probability gives you a percentage chance that each variant beats the control, which is super intuitive for pruning.
 
If a variant has less than 5% probability of beating the control, that’s a clear trigger to prune a variant.
 
Frequentist p-values work too, but they don’t tell you the probability of being the best.
 
Whichever method you use, the math should guide you on when to prune a variant, not just gut feeling.
 
Just remember that these metrics require enough data to be reliable, so don’t prune a variant in the first few days.
 

2. Prune a Variant When It Hits a Predefined Stop Rule

You can avoid all the guesswork by setting stop rules before you even launch your test.
 
For example, you might decide to prune a variant if it falls below the control for 10 consecutive days.
 
Or you might prune a variant when the confidence interval completely excludes any chance of being better.
 
Predefined stop rules make it easy to prune a variant without emotion or bias creeping in.
 
Write down your stop rules in your test plan, and then stick to them when you need to prune a variant.
 
This approach ensures you prune a variant consistently, no matter how attached you are to that creative idea.
 

Common Mistakes When You Prune a Variant

Even experienced marketers make errors when deciding to prune a variant, so watch out for these pitfalls.
 
One big mistake is pruning a variant too early based on just a few days of data.
 
Another mistake is never pruning a variant because you keep hoping it will turn around, which wastes tons of traffic.
 
You should also avoid pruning a variant without documenting why, because that kills your institutional learning.
 
Finally, don’t prune a variant just because it’s not the current leader if the test is still early.
 
The key is to prune a variant based on evidence, not impulse or convenience.
 
Take a step back, review your metrics, and then prune a variant only when the data supports it.
 

So, When Should You Prune a Variant?

You should prune a variant when it’s lost with statistical significance, when it’s clearly underperforming, or when it’s plateaued.
 
You should also prune a variant when external factors invalidate the test environment.
 
And you should prune a variant according to your predefined stop rules to stay objective.
 
The timing of when to prune a variant isn’t about guessing, it’s about letting the data tell you the story.
 
So, prune a variant that’s wasting your traffic, and give your winning ideas the room to shine.
 
Hope this guide helps you prune a variant with confidence and improve your conversion rates going forward.