Opportunity Cost and Testing — In 1 Minute

Welcome to One Minute Mondays, because on Monday, you only have one minute. Each Monday, we’ll post an education-packed tip presented by someone from the Optimizely team. Learn something new and connect with the Optimizely team — all in 60 seconds!

This week, Opportunity Cost and Testing from Khattaab Khan, a Strategic Consultant at Optimizely.

Opportunity costs are an essential consideration when managing your testing program. Don’t fall into the trap of letting tests run on and on in hopes of achieving statistical significance on minimal conversion lift. When tests are trending inconclusive, consider  other high value, easy-to-implement tests that you could be running instead.

“How long should I run my test?” is a question I hear all the time. The sample size calculator available on then the Optimizely homepage is a great estimate of test duration, but you should be relying on the visitor estimates displayed on the results dashboard. These figures are computed in real time leveraging our Stats Engine, to consider traffic volume and conversion distribution.

Iterative testing is an essential tenet of a healthy testing culture. You should always be learning from your past test results to consider future test ideas and incremental UX changes. However, testing is all about challenging the status quo. Challenge the assumption that the current layout meets users’ expectations because a more impactful change to the UX experience may achieve more ROI than incrementally improving a broken product.

Watch more One Minute Mondays and join the conversation in Optiverse, the optimization community.