I'm Janna Bastow, co-founder and CEO of ProdPad. AMA!

Thanks for your question Krish!

Pricing experiments should be fundamentally structured just like any other experiments. Start with a hypothesis, outline what problem it’s expected to solve or impact it’s expected to have, and devise a way to test (as much as possible) discrete elements so you can observe whether changes made a difference or not. That said, the nature of pricing makes it unique to test, as a lot of companies worry about the optics of ‘playing’ with pricing in live environments and the potential impact on their brand, while not seeing anything wrong with running A/B tests that change other elements like the interface. Fortunately, not all experiments need to be run live! Pricing surveys, customer interviews, and other analysis jobs can get you a long way in understanding the impact of adjusting pricing.

The second part of your question might be a bit of a trick question, as better experimentation is a key part in enabling better decision making. Perhaps it’s worth qualifying what we mean by a good experiment. Any experiment should be started with a hypothesis and be given enough context that you and your team know what decisions are hanging off the experiment before you begin. There’s no point running experiments if it’s not clear what you’re expecting to see change or why.

4 Likes