[Experiment][Ship] Immediate Premium Upsell with Close Button
Iterating on a Previous Experiment
I previously posted about [Experiment] Immediate Premium Upsell, which I shut down after seeing an expected rise in revenue but a decrease in retention. I considered this trade-off unacceptable and viewed it as a typical trap trading short-term gains for long-term cost.
I reran the same experiment with one minor change: a close button to make it obvious that the premium version of ✍️Writeaday was optional.
Revenue Growth
Summary
I ran this experiment for almost 3 weeks, got no revenue from the control group, but got over $100 in revenue with the test group. Luckily the retention numbers here look neutral (within the error bars).
Revenue: +160% ($86 vs $209)
Day 1 Retention
This means that between the 2 groups (people who don’t see the premium upsell and people who do), there is no difference in how likely they are to come back. Showing that premium upsell once had no effect, which is great.
Day 2-3 Retention
Day 8-14 Retention
Next Steps
I’m shipping this! I could leave this running but the retention numbers all look fine. This extra revenue coming in should be helpful in reallocating into the business - either by hiring some part-time help with the coding or for hiring some illustrators to add some delightful art around the app.