Google Search Campaign Experiments: Make the Test Earn a Decision

Before paying to test a campaign change, decide what result would make you keep it. A cheaper inquiry may look encouraging, but would the saving still matter if fewer suitable customers contacted you? Without an agreed answer, an experiment can finish with attractive numbers and no clear decision.
Google Search campaign experiments compare a proposed change with the existing approach during the same period. That is more useful than judging an edit against last week, when demand, competition and late-arriving conversions may have differed. We use that comparison to answer a practical question: has this particular change earned a place in this campaign?

Turn the proposed change into a decision
“Try a lower target cost per acquisition” names a setting to change. A hypothesis explains what you expect it to achieve: “A lower target will reduce the media cost of a qualified inquiry without losing more suitable inquiries than we can accept.” The test needs both the expected benefit and the trade-off you will tolerate.
Choose the main outcome before launch. For a lead-generation campaign, better click-through rate can help explain delivery, but it cannot replace qualified inquiries as the success measure halfway through the test. Both sides need the same qualification criteria and a consistent way of recording those outcomes.
If broader targeting is the proposed change, our guide to choosing Google Ads keyword match types helps you decide what deserves testing. Keep that choice separate from the experiment's job: establishing whether the defined change is worth adopting under the conditions you tested.
Choose the split with the customer journey in mind
Google's custom experiment setup guidance describes sharing the original campaign's traffic and budget. In Search, cookie-based allocation keeps a recognized user on one side; search-based allocation can show the same user different sides on different searches. Neither guarantees consistent identity across every device or browser.
Where your selected setup offers these choices, ask whether seeing both versions could affect a customer's response. Someone returning several times before an inquiry may need a consistent experience. Record the chosen method so the team understands what the comparison actually separates.
Check the controls available for the eligible campaign and experiment type. Google's documented flows differ: some offer a chosen percentage, while others specify an even split. Even a 50/50 allocation does not guarantee equal impressions or actual spending; auction outcomes still affect delivery.
An even split gives both sides substantial exposure. A smaller treatment share exposes less traffic to the uncertain change, but leaves fewer observations on that side. Agree the combined spending allowance and its owner alongside the split. Sharing an allowance is different from authorizing the full original budget twice.
Can the available traffic answer the question?
A result can matter commercially yet be difficult to detect reliably. At eight qualified inquiries a week, a 10% difference is less than one inquiry at that weekly volume. Ordinary variation can easily obscure a change that small during a short run.
This is where statistical power helps planning. It describes the chance of detecting a specified real effect under the test's assumptions, not the chance that your preferred version will win. Expected volume, variation, allocation, effect size and duration all belong in the discussion.
Google's Campaign Guidance explanation describes an estimated Experiment Power score for supported experiments, including Broad Match experiments in Search. It is not available in every custom test. Without that score, ask the person designing the comparison to explain the sample and duration assumptions.
Make this assessment before committing the money. If an affordable run cannot distinguish a useful improvement, consider a suitable higher-volume campaign, revise the question or defer the test. Extending it repeatedly until a favorable number appears changes the decision rule after the evidence has arrived.
A cheaper result that does not earn rollout
Northline Equipment Service is fictional. Every setting, amount, result and decision in this example is invented for teaching, not a DIGITALTHROB client result. Its team has verified that the selected Search experiment supports the proposed target CPA change.
The control, meaning the existing approach, retains a $120 target CPA. The treatment, meaning the proposed alternative, uses $105. These are bidding targets; the cost actually recorded per inquiry is calculated separately below. Both sides optimize the same qualified-inquiry action, with unchanged ads, keywords, destination and qualification rules.
The owner approves a 50/50 cookie-based allocation, a six-week run and a $16,000 combined media allowance monitored by the campaign manager. Adoption requires at least a 10% reduction in recorded cost per qualified inquiry, no more than a 10% reduction in qualified-inquiry volume, and statistical evidence of an improvement in the primary cost metric at the preselected confidence level. These are Northline's requirements, not benchmarks.
The run finishes within its allowance. After allowing the agreed outcome delay, the team reconciles the same experimental periods:
Control: $8,000 spent; 80 qualified inquiries; $8,000 ÷ 80 = $100 per qualified inquiry.
Treatment: $8,000 spent; 88 qualified inquiries; $8,000 ÷ 88 = approximately $90.91 per qualified inquiry.
Observed difference: qualified inquiries are 10% higher; cost per qualified inquiry is approximately 9.09% lower.
The volume condition passes. The cost condition does not: a 10% reduction from $100 would require $90 or less, and the treatment recorded approximately $90.91. Eight additional inquiries do not change the rule the owner approved. Equal spending also happened in this fictional result; the allocation did not guarantee it.
Read the uncertainty before calling a winner
Northline's fictional scorecard also shows a confidence interval spanning no difference for the primary metric. The interval expresses uncertainty around the estimated difference. Here, that report finding is assumed for teaching; it has not been calculated from the three summary figures above. Those totals are insufficient to reconstruct Google's experiment analysis.
Google's experiment reporting guidance explains metric differences, confidence intervals and significance indicators. Read them for the agreed metric and comparison dates. Lowering the confidence level after seeing an unclear result would move the goalposts.
The owner declines adoption. The manager ends the treatment and retains the $120 target because the cost reduction missed the commercial threshold and the report did not provide clear statistical support. This does not prove the lower target has no effect. It means this test did not justify rollout, and no further spending is authorized to chase an answer.
Finish with the campaign you actually approved
A comparison is only useful if you know what changed. Investigate broken destinations, tracking faults and spending-limit breaches promptly, keeping a dated record of necessary interventions. Use Google Ads change history alongside website changes to check whether the intended difference remained the only material change.
Before launch, review any auto-apply setting the experiment offers. Northline requires a human adoption decision, so automatic rollout is disabled in this example. At closeout, the manager confirms that the treatment has ended and the retained campaign matches the owner's decision.
The evidence applies to the tested conditions. It does not promise the same result at a different budget or establish how many sales would disappear without advertising. Bring your proposed change, current outcome volume and acceptable trade-off to our Google Ads management service. We can help shape a test whose answer is useful enough to act on, before you commit its budget.
Comments