Receipts, not claims
Your marketing might be working. Or it might be taking credit for sales that were going to happen anyway.
Most marketing measurement answers the wrong question. It counts what happened after the campaign and calls it results. But the real question is not what happened — it is what would not have happened without you. That gap is called incrementality, and almost nobody measures it, because measuring it means being willing to be wrong.
The problem is attribution's blind spot. When someone who already loves your brand sees your ad and buys, the ad gets the credit — even though they were going to buy regardless. Scale that across a whole campaign and you get a report full of conversions that were always coming. The number looks like proof. It is mostly coincidence wearing a costume.
A holdout fixes this. You deliberately withhold the marketing from a random slice of your audience, then compare them to everyone who got it. If the treated group buys more than the held-out group, the difference is real and it is yours. If they buy the same, you just learned you were paying for sales you already had. Both answers are worth more than a dashboard that only ever says yes.
This is what separates receipts from claims. A claim says our campaign drove X in revenue. A receipt says the treated group outperformed the holdout by this much, here is the group, here is the math. One is a story; the other survives scrutiny. When you are spending real money, you want the kind of number that holds up when someone pushes on it.
Holdouts feel uncomfortable because they can prove your work did nothing. That discomfort is the value. A team that runs holdouts stops spending on the things that only looked like they worked, and doubles down on the things that actually move — which is the whole point of measuring anything at all.
