Firebase A/B Testing

Firebase A/B Testing helps us test changes in a mobile app, such as different layouts, feature settings or notification wording, before rolling them out to everyone. It compares variants against a goal such as purchases or retention. It is for app teams rather than website owners.

It can suit a small team that already has a mobile app built on Firebase, because testing sits inside the same console. It does need developers to put Remote Config or Cloud Messaging into the app. If we only have a website, it is probably not the right fit.

The Firebase documentation page we read does not state a price for this feature, and we could not confirm costs from it. Firebase has its own pricing page, so confirm on the vendor pricing page. We would also budget for developer time.

We could test app behavior and appearance through Remote Config, and test notification messages through the Notifications composer. We could target by app version, platform, language or Google Analytics user property, start with a small share of users and roll out the winner. Results can be tracked alongside crash data.

It helps rather than replaces a person. We still need someone to form the hypothesis, build the variants and decide what the results mean.

It works with Firebase Remote Config, Firebase Cloud Messaging and Google Analytics, which provides the goals and targeting properties. The documentation tags it for iOS and Android. We could not confirm links to non-Google tools.

We would check that our app already uses Remote Config or Cloud Messaging and that Analytics events are set up to measure success. We could not confirm independent reviews on G2, Capterra, GetApp or Trustpilot, so we would test with a small rollout first.