TL;DR
- If your test does not change only one variable, you are not testing - you are hoping.
- Pre-commit the decision you will make at each possible outcome before spending a penny.
- Run tests long enough, and stop them when the decision is made - not when you feel bored.
- Log the losers. Half your learning is in the variants that failed.
The four tests worth running
Message, creative, offer, audience. Each is a separate test type. Trying to test two at once is the most common way creative "tests" generate noise rather than information.
Hypothesis > variant
Every test starts with a named hypothesis: "if we lead with the compliance angle, CPA drops 15% vs the speed angle." That is testable. "Let us try some new creative" is not.
Readable design
A readable test has: one variable, statistically adequate volume, a pre-committed decision rule, and a written record. If you cannot produce the four up-front, postpone the test until you can.
Pre-commit the decision
"If variant B wins by more than X, we promote it to always-on and retire A. If it ties, we kill it and redeploy the budget." That decision goes in the test doc before launch. It removes the usual post-test negotiation.
Stop on decision, not on boredom
Statistical power decides when tests end, not the team's patience. On the flip side: if a test has already passed its decision threshold, stop it. Money burned in a finished test is stolen from the next one.
The losers earn their keep
Log every variant, winner or loser, into a shared learning doc tagged by hypothesis. Three months in, the doc becomes the most valuable creative asset your team owns - and usually the one most teams forget to keep.
Need this installed?
Npexpos's paid search and paid social engagements both install this testing frame and train the team on it.