A dashboard will happily tell you that signups fell 12% yesterday. It will not tell you whether that matters.
Most of the time it does not. Weekday seasonality, a holiday in one market, a single enterprise account renewing a day late. These move the line without meaning anything. Treat every movement as a signal and you spend your week chasing noise. Treat none of them as signals and you find out about churn from the invoice.
The line we draw
A change is worth surfacing when three things are true at once:
- It is outside the range the metric normally moves in. Not "different from yesterday" but different from what this metric does on this weekday, at this point in the billing cycle, in this segment.
- It has a plausible cause you can point at. A drop that coincides with a deploy, a pricing change, a paid campaign ending, or a support spike is a different object than a drop with nothing attached to it.
- It has a revenue consequence you can size. "Trial starts fell" is a metric. "Trial starts fell, and at your historical conversion rate that is roughly $4,100 of MRR over the next 60 days" is a finding.
Fail any of the three and it is a chart, not a finding. The same test applies to the standard metrics: a net revenue retention figure that moved six points is a chart until you can name the accounts that moved it.
Why the third one is the hard one
The first two are pattern matching. Plenty of tools do anomaly detection, and plenty more will let you overlay a deploy marker on a chart.
The third requires connecting behaviour to billing: knowing that the visitor who arrived from a search query became a user, who belongs to an account, which maps to a Stripe customer paying a specific amount on a specific cycle. That join is where most analytics stacks give up, because it spans four systems that were never designed to agree with each other.
It is also the only reason the first two are worth doing.
What this means in practice
You should not have to open GainSignal to find out something happened. The product's job is to notice, investigate, and tell you, with the evidence attached, so the first thing you read is not a number but an argument.
Everything else in the product exists to make that argument trustworthy.