We were launching a new feed ranking change and needed an experiment readout within two weeks. The ideal approach would have been a full holdout with a 30-day run, but we didn't have time. So I designed a shorter seven-day test and adjusted our power calculations accordingly. I documented the statistical limitations — lower power, higher variance in engagement metrics — and flagged that the results would be directional rather than conclusive. The PM accepted that framing, we shipped based on the directional signal, and the feature launched. I noted we should do a follow-up measurement later.
We had a feed ranking change ready to ship, and the product team needed a decision in ten days. A full 30-day holdout was the methodologically correct choice, but I calculated that waiting added ten days of delayed measurement across 200 million daily active users — that inaction was itself a product bet. I made the call to run a seven-day test with reduced power, wrote a one-page limitations brief for the PM and engineering lead that named the exact confidence interval we'd accept as directional, and committed to a structured 30-day post-launch holdout to validate. Engagement lift came in at 4.2 percent — confirmed at 30 days. The framework became our standard for time-boxed launches.