A’s concrete time promise outperformed B at every funnel step (SAMPLE data).
The biggest drop-off for both is signup → checkout.
Payment counts are too small to support a pricing conclusion.
We observed behavior, not opinions: clicks, signups, checkout starts and payment attempts from real visitors.
Events per variant (SAMPLE counts).
Visitors respond to a concrete, time-bound promise. Interest collapses when price appears.
| Segment | Signup rate A | Signup rate B |
|---|---|---|
| Mobile | 9.1% | 4.6% |
| Desktop | 12.4% | 5.9% |
Whether to show price earlier and remove forced signup.
A vs A+visible price, 7 days, same budget.
Current methodological context. These references are not evidence for this historical run.
SKPTST: planned methodology. The engine and paid traffic acquisition are not active.
Kohavi, R.; Longbotham, R.; Sommerfield, D.; Henne, R. M. (2009). Controlled experiments on the web: survey and practical guide. Data Mining and Knowledge Discovery, 18, 140–181. Online: 30 July 2008; issue: February 2009.
Kohavi, R.; Deng, A.; Vermeer, L. (2022). A/B Testing Intuition Busters: Common Misunderstandings in Online Controlled Experiments. Proceedings of ACM SIGKDD 2022, 3168–3177. Conference: 14–18 August 2022.
Lewis, R. A.; Rao, J. M. (2015). The Unfavorable Economics of Measuring the Returns to Advertising. The Quarterly Journal of Economics, 130(4), 1941–1973. Issue: November 2015. · Blake, T.; Nosko, C.; Tadelis, S. (2015). Consumer Heterogeneity and Paid Search Effectiveness: A Large-Scale Field Experiment. Econometrica, 83(1), 155–174. Issue: January 2015.
Test behavior, not opinions.
This report
