How to Build a Rollout Strategy in Statsig
Shipping a feature to 100% of users on launch day is a gamble. If the feature has a bug, every user is affected. If it degrades performance, there is no control group to compare against. If users hate it, you cannot roll it back without a code change and another deployment. Staged rollouts replace this binary ship-or-do-not-ship decision with a gradual, measurable, reversible process.
Statsig is purpose-built for staged rollouts, combining feature gates (who sees the feature) with automated metric monitoring (whether the feature is working). Unlike basic feature flag tools, Statsig statistically compares metrics between the treatment and control groups in real time, telling you whether the feature is improving or harming your KPIs before you roll it out to everyone. This guide walks you through building a complete rollout strategy in Statsig.
Step-by-step guide
Define your rollout stages and decision criteria
Before creating anything in Statsig, write down your rollout plan. A typical staged rollout has four stages: Internal (your team only), Beta (1-5% of users), Progressive (5% to 25% to 50%), and General Availability (100%). For each stage, define: the audience size, the duration (how long you will observe before proceeding), the metrics you will monitor, and the go/no-go criteria (what metric thresholds must be met to proceed to the next stage).
- Document 4-5 rollout stages with specific audience percentages
- Set minimum observation periods for each stage (e.g., internal: 3 days, beta: 7 days)
- Define 2-3 primary metrics to monitor at each stage
- Write go/no-go criteria: 'Proceed if error rate is below 0.5% and latency is under 200ms p99'
Create the feature gate in Statsig
In the Statsig console, create a new Feature Gate with a descriptive name following your naming convention (e.g., 'new_checkout_flow_v2'). Add a description that explains what the feature does and links to the relevant PRD or design doc. Set the initial state to disabled. Add targeting rules in order: first rule targets your internal team (by email domain or user ID list), second rule targets beta users, third rule targets the progressive rollout percentage. Statsig evaluates rules top-to-bottom, so order matters.
- Create the feature gate with a clear, descriptive name
- Add the first rule targeting your team by email domain for internal testing
- Add a percentage-based rule starting at 0% for the beta and progressive stages
- Write a detailed description including links to the PRD and design documentation
Configure metric monitoring
In the feature gate's Metrics tab, add the metrics Statsig should monitor. Add your primary success metric (e.g., conversion rate, engagement), guardrail metrics (e.g., error rate, page load time, crash rate), and secondary metrics (e.g., session duration, feature adoption rate). Statsig will automatically run statistical comparisons between users who see the feature (treatment) and those who do not (control), surfacing any statistically significant differences. This is the automated safety net that catches regressions.
- Add 1-2 primary success metrics that the feature is designed to improve
- Add 3-4 guardrail metrics that should not degrade (error rate, latency, crash rate)
- Configure the statistical significance threshold (95% confidence is standard)
- Enable Statsig's automated alerts for guardrail metric regressions
Implement the feature gate in code
Integrate the Statsig SDK into your application and wrap the new feature code behind the gate check. On the server side: if statsig.check_gate(user, 'new_checkout_flow_v2') returns true, serve the new experience. On the client side: use the Statsig React SDK's useGate hook to conditionally render the new UI. Ensure that exposure logging is working — Statsig needs to know which users saw which variant to calculate metrics correctly. Test the gate locally by overriding it in the Statsig console.
- Install the Statsig SDK for your platform (server and/or client)
- Wrap the new feature code in a gate check using the gate name you created
- Verify exposure events are being logged in the Statsig Diagnostics tab
- Test the on and off states locally by using Statsig's local overrides
Execute the staged rollout
Begin by enabling the internal targeting rule and deploy. Have your team use the feature for 2-3 days, focusing on functionality, edge cases, and performance. If no issues are found, enable the beta rule at 1-5% and observe for 5-7 days, checking Statsig's metric dashboard daily. If metrics are neutral or positive, increase to 25%, then 50%, observing for 3-5 days at each stage. At each stage, check the Statsig pulse results for statistically significant metric changes. Only proceed to 100% when you have confidence from the data.
- Stage 1: Enable for internal team, test for 2-3 days
- Stage 2: Roll out to 5% of users, observe metrics for 5-7 days
- Stage 3: Increase to 25%, then 50%, observing 3-5 days at each increment
- Stage 4: Roll out to 100% only after all go/no-go criteria are met
Set up automated rollback rules
Configure Statsig's automated rollback to disable the feature gate if guardrail metrics breach critical thresholds. For example: auto-disable if error rate exceeds 2% for more than 10 minutes, or if p99 latency exceeds 500ms for more than 15 minutes. Automated rollback is your safety net for issues that emerge outside business hours or when the team is not monitoring actively. It converts a potential outage into a contained incident.
- Configure auto-rollback triggers for error rate, latency, and crash rate thresholds
- Set appropriate time windows (do not rollback on momentary spikes, but do rollback on sustained degradation)
- Configure rollback notifications to alert the team via Slack and PagerDuty
- Test the rollback mechanism by temporarily lowering thresholds during internal testing
Common mistakes
Rolling out too fast without observing metrics
The pressure to ship quickly leads teams to jump from 5% to 100% after a single day of clean metrics. Some regressions only appear at scale (database contention, cache invalidation patterns) or over time (increased churn showing up after 7 days). Respect the minimum observation period at each stage — the time investment is trivial compared to the cost of a full-user-base regression.
Not monitoring guardrail metrics alongside success metrics
A feature that increases conversion by 5% but also increases error rate by 3% is not a win. Always monitor guardrail metrics (error rate, latency, crash rate) alongside your primary success metric. Statsig makes this easy by showing all configured metrics in a single pulse results view.
Leaving feature gates in code after full rollout
Feature gates left in code after 100% rollout create technical debt. Old gate checks add code complexity, slow down builds, and confuse new engineers. After a feature is at 100% for 2-4 weeks and the decision to keep it is final, remove the gate check from code and archive the gate in Statsig.
Tips
Use Statsig's targeting rules to roll out to your most engaged users first — they are more likely to give feedback and more tolerant of rough edges than casual users.
Create a rollout checklist template in your project management tool that maps to the Statsig stages, so every feature follows the same process.
Set up a Slack channel that receives Statsig's automated metric updates during active rollouts so the team has passive visibility without checking the dashboard.
After completing a rollout, document the metric impact in the feature gate's notes — this creates an archive of what worked and by how much.
How Vantage helps
Vantage helps teams plan rollout strategies during the product development phase. When generating tickets, Vantage can include rollout stage requirements in acceptance criteria, ensuring that features are built with staged deployment in mind from the start rather than bolting on feature flags after the code is written.