Guide

Trial optimization for startups

A practical order of work for a small SaaS team: cohort tracking, free recovery settings, exit conversations and one large change a month.

By Meralume Labs. Published . 7 min read

A startup with a few hundred trials a month cannot run clean split tests, so it should count trials by cohort, switch on the free recovery settings, fix the one obstacle that leavers name, and compare one change per month against the last.

Why a small product optimizes differently

Most advice about trials comes from products with thousands of sign-ups a month. At that volume you can split traffic, wait a week and read a clean answer. At 120 trials a month you cannot, and pretending you can produces confident nonsense. The ChartMogul conversion report is a useful reminder of how wide the spread already is: across 200 products the median free-to-paid rate is 8%, yet the report finds a tenfold gap between the top fifth and the bottom fifth of self-serve products. A median is not a target. Your own cohort is the only baseline that matters.

The same report says that about a third of its respondents, 34%, were under 1 million dollars of annual recurring revenue, so small products are well represented in its data. The advice here keeps to what a one or two person team can do without a data team.

Step 1: count trials by start cohort

Group trials by the month they started. Count a trial as converted when the first successful charge lands, not when the customer enters a card, because a card that fails on day one is not a customer. Close a cohort only after its last trial has ended and the failed charges have had their retries. With the recommended Smart Retries setting that is about two weeks after the last trial ends.

A concrete month looks like this. 120 trials started, 12 paid, price $29 a month. The trial conversion rate calculator gives a rate of 10.0% and $348 of monthly revenue from the cohort. Write that down every month in one spreadsheet row. After three months you have a range, and a range is something you can judge a change against.

Step 2: price the prize before you chase it

Percentage points of conversion sound large and are worth very different amounts at different volumes. The table uses the same cohort and shows what extra points are worth in customers and revenue.

GainNew rateExtra customers a monthExtra MRRExtra ARR
+1 point11.0%1$29$348
+2 points12.0%2$58$696
+3 points13.0%4$116$1,392
+5 points15.0%6$174$2,088

A customer is worth more than their first month. With a gross margin of 80% and monthly churn of 5%, both what-if inputs, the customer lifetime value calculator gives a lifetime value of $464 for a $29 plan. 4 extra customers a month from three more points is therefore worth far more than the $116 of extra monthly revenue it shows on day one. Put your own churn in before you decide how much effort a point deserves.

Step 3: switch on the free recovery settings

Some improvements cost nothing but a settings page, and they belong first because they do not need traffic to justify. Stripe's documentation lists them, and it publishes an average recovery rate of 55% of failed payments for businesses using Stripe Billing. That is Stripe's own figure for its whole customer base, not a promise for your account, and it should be read as a reason to turn the features on rather than as a forecast.

SettingWhat the documentation saysEffort
Smart RetriesRecommended default is 8 tries within 2 weeks. Stripe does not retry a hard decline such as a lost or stolen card.One setting
Failed payment emailsStripe emails the customer after each failed payment with a link to update the card.One setting
Trial ending reminderA reminder goes out 7 days before a free trial ends. A trial shorter than 7 days sends it at the start.One setting
Expiring card emailsAn email goes out one month before the card on file expires.One setting

At startup scale the money is small, and it is worth saying so. If the 12 paying accounts above pay $29 a month, MRR is $348. With a 5% first-try failure rate and a 40% recovery rate, the failed payment recovery calculator puts the unrecovered loss at $125.28 a year. That is a real amount for a small business, it grows with every customer you add, and the settings take minutes. It is not the biggest lever you have yet.

Step 4: ask the people who left

With few trials, one conversation teaches more than a dashboard. Contact people whose trials ended without payment within a few days, while they remember, and ask what they were trying to do and what stopped them. Five to ten answers will usually show one obstacle that comes up repeatedly: a setup step nobody finished, a missing integration, a price they did not expect.

The ChartMogul report ends with three questions to answer before changing a trial, and they work as a diagnostic for a small team: do you want more people to use the product or more people to buy it, at what point does a user reach the moment that shows the value, and what does each free user cost you. The report gives no numbers for them. They are prompts for your own data.

Step 5: change one thing a month and compare cohorts

A split test needs more trials than you have. The table uses the same method as the pricing test guide, two-sided 5% significance and 80% power, for a product starting at 10% and 120 trials a month.

Change to detectTrials in each armTime to collect both arms
10% to 13%1,77430 months
10% to 20%1994 months
10% to 30%622 months

A gain of three points would take years to confirm. A gain of ten points, which is a big change, takes months. So make one large change per month, keep everything else steady, and compare the new cohort with the last two or three. Say in your notes that this is weaker evidence than a split test, and revisit a result if a later cohort disagrees.

One change is big enough to see even at this volume: asking for a card before the trial starts. The report lists 45 sign-ups and 3.6 paying customers per 1,000 visitors for free trials, against 35 sign-ups and 10.5 paying customers for trials that require a card, and a 30% conversion rate for card-required trials. Those are survey averages across different products, so your result will differ, but a gap that wide would show up in a few months of cohorts.

A 90 day order of work

  1. Weeks 1 and 2: set up the monthly cohort sheet and back-fill the last three months if you can.
  2. Weeks 3 and 4: turn on retries, failed payment emails, the trial reminder and expiring card emails.
  3. Month 2: talk to ten people whose trials ended without payment and fix the obstacle that appears most.
  4. Month 3: make one large change, such as the card requirement or the length of the trial, and compare cohorts.

Questions from small teams

How many trials do I need before any of this works?

Cohort tracking and the free settings work at any size. A split test of a small gain needs thousands of trials per arm, as the table shows, so below that use big changes and cohort comparisons.

Should I copy the benchmark rate?

No. The 8% median covers 200 different products, and the report finds a tenfold gap between the best and worst fifth. Compare against your own earlier cohorts.

Is requiring a card right for a new product?

It is a trade. The report shows fewer sign-ups and more payers per 1,000 visitors on average, and it notes that a card requirement can reduce sign-ups unless the product warrants the friction.

What should I measure first?

Trials started, paid on first charge, and failed first charges, each by start month.

When should I move to proper split tests?

When the arithmetic in the table says a test for the change you care about fits in a few weeks of your own traffic.

Sources