How to Make an A/B Test on Wisepops?

The 2023 steps on this page skipped the success metric and the holdout. Those are the parts that matter if you care about revenue, not just clicks.

How do you A/B test a popup, bar or embed?

1. Log in and find the campaign.
2. Hover it and click the A/B test icon.
3. Duplicate the campaign (new variant) or select an existing campaign.
4. Set traffic allocation (50/50 for two variants is the default advice).
5. Optional: add a control group that sees nothing. Help docs mark this as current-plan / Essential.
6. Pick a success metric. The test concludes when one variant hits 95% confidence. Options:
   • CTR
   • Order rate (share of visitors who converted a goal with revenue)
   • Revenue per visitor
   • The goal already attached to the campaign
7. Choose how it ends: you pick the winner, or Wisepops auto-rolls the winner to 100%. You can set a max end date so the best variant ships even if significance never arrives.
8. Edit the variants (copy, offer, image, targeting) before you collect data. Editing after launch skews the test.
9. Change status from Draft to Published. Tracking starts with the first visitor.

The experiment dashboard is available 24 hours after launch. When a winner is in and you did not auto-conclude, Duplicate as new campaign on the winning variant. The original experiment stops.

How do you A/B test feed or web push?

From the campaign's arrow menu, pick either:

• two variants at 50/50 (success metric: CTR), or
• the campaign vs a control group at 50/50 (success metric: revenue per visitor)

Edit the variant, publish. Publishing one variant publishes the experiment.

Help docs say feed/web-push tests and detailed results need a 2025 plan. The marketing site says no feature gating. We can't see inside an account, so both claims stay.

What should you actually test?

One change at a time. Two or three variants, not seven. Wait for the dashboard to call significance. Even traffic splits.

Don't use CTR alone when you changed trigger delay. A popup that waits 8 seconds will often beat an immediate one on CTR and still lose on leads or bounce. Pick the metric that matches the job.

To measure "did showing this campaign make money," you need a holdout, not click attribution. That is the point of the control group.

More: goal tracking, Wisepops help.