Mostly geeking about about all things data
-
Getting faster to decisions in A/B tests – part 2: misinterpretations and practical challenges of classical hypothesis testing
Null hypothesis test of means is the most basic statistical procedure used in A/B testing. But the concepts built into it are not exactly intuitive. I go through 5 practical issues that anyone working with experimentation in business should be aware of.
-
Getting to decisions faster in A/B tests – part 1: literature review
I set out on a journey to learn what statistical approaches the industry uses to get to faster decisions in A/B testing. This is the first post in the series in which I set the scene and summarize outcomes of my “literature review”.
-
A/B testing, zero-inflated (truncated) distributions and power
Naive A/B testing just uses t-tests or proportion tests, with the assumption that at large sample sizes, the right statistical test does not matter that much. I explore the case of a zero-inflated upper-bounded Poisson distribution and find that using the wrong test can require 3x the sample size to achieve the same statistical power, a difference large enough to matter in a real business setting.
-
The Jungle of Metrics Layers and its Invisible Elephant
Metrics layer is the rising “missing component” of the modern data stack. I dive into functionalities provided by 10 prominent players in the space and contrast them with a player no one talks about: Microsoft Power BI.
-
Grappling with privilege
Realizing you’re privileged is uncomfortable. It’s like understanding you can have a cake and eat it, too, except you never asked for the damn cake and you now realize most people aren’t as lucky as you are.