Early in my role I built a model to forecast demand for a new product launch. I was confident in the inputs, but I underestimated how much a concurrent marketing campaign would inflate short-term signal. The forecast was off by about twenty percent, which caused the team to over-order inventory. I caught it quickly, recalibrated the model mid-launch, and we avoided a major stockout on the backfill. It was a good learning experience — I now always check with the marketing team before locking a forecast.
I ran an experiment measuring the impact of a new checkout feature. My design used page-level randomization, and I was confident the lift was real — I reported a two-point conversion improvement to the product team. Three weeks later, a peer flagged that our randomization unit was wrong: users were crossing between variants, inflating the signal. The feature likely had near-zero effect. I owned the error in the debrief doc, halted the rollout, and — critically — I rebuilt our experiment review checklist to require explicit randomization-unit sign-off before any experiment ships. Every experiment in my team's roadmap now goes through that gate.