When you launch a campaign, you randomly select a percentage of the audience and don't send them anything. That group continues to shop as they normally would. In the end, you compare the results of those who received the message with those of the control group, and the difference shows what the campaign actually achieved.

Why does it matter?

Without a control group, any sale following a campaign appears to be a success, even if the customer would have made the purchase anyway. The holdout group provides a fair point of comparison.

Some best practices:

• Choose the group at random, so that it is similar to the rest of the audience.
• Use a sample size large enough to ensure that the comparison is reliable.
• Measure the two groups within the same time frame.

It's true that the control group doesn't receive the campaign, but what you gain is certainty: you know how much each action is worth, and you can invest with greater confidence in the ones that work.

Story mode

An online sports store sends a reactivation message via WhatsApp to 5,000 inactive customers but fails to reach 500 of them. For example, after two weeks, 6% of those who received the WhatsApp message made another purchase, compared to 2% of the control group. The store knows that this four-point difference is the actual effect of its campaign.

How to Use It with Data4Sales

• Data4Sales attribution is measured using a holdout group.
• This is how you calculate the actual incremental revenue for each campaign and channel—not clicks or opens.
• You can apply the same logic to campaigns on WhatsApp, email, or SMS.

Related terms

• Incremental revenue
• A/B Test
• Attribution
• Reactivated customers