| | AUGUST 20249us follow one example all the way through. We believe that the "Add to Bag" button is not visible enough so we wish to move it up in the product pages within our website. Prioritizing IdeasCreate a system that allows the best ideas to surface, one that does not consider who submitted the idea. The goal is to remove the emotion from the prioritization process. I could write a whole article on this, but in summary, come up with a process that gives ideas points, either zero or one, no gray areas, no in-between. For example, will the test be seen by over 50 percent of traffic to the website? If the answer is yes, the idea gets one point. If the answer is no, the idea gets zero points. In our example, our idea would score one point since more than 50 percent of people view product pages.After you have come up with many criteria (around 10 to 12), test the prioritization. Rate 10 ideas through the system and review the results. If an idea did not get proper points, why? What is missing? This will highlight criteria that need to be added, such as "Does it lessen manual workload?" Writing a HypothesisFormulate a hypothesis by stating "If ... then ... because ..." Be bold and specific in the hypothesis. A good question to help you determine your hypothesis is, what will you do if there is no change? Which version would you implement if the results are flat? In our example, the hypothesis could be: If we move the "Add to Bag" button higher up in the product pages then the add to cart rate will increase and thus conversion rate would increase because customers would clearly know the next step to follow and it would eliminate friction.Executing a TestYou will have many details to iron out, such as who you will target, how long the test will run (many online calculators can help you determine the length based on your sample size), and when is someone a part of the experiment (when they view a specific page).Tip: Don't test something you would not implement. In our example, we could target all visitors for ~4 weeks. A person would be a part of the test if/when they viewed a product page.Reading ResultsPull the results and determine statistical significance. Track beyond the simple metrics. Look at interdependencies and search beyond one metric. You should always analyze with the story in mind, and not just read numbers. Tip: Don't base your results on raw numbers (such as revenue), but focus on ratios to determine significant results, especially if your groups are unevenly distributed. In our example: let's pretend that both the add-to-cart rate and conversion rate decreased. We also noticed that interactions with other components on the page decreased. This teaches us that the other components contribute to the evaluation phase and need to be more prominent on the page in order for customers to feel comfortable purchasing the product.Implement ChangesIf the test variation wins (the contender), implement the change. If the control wins, either formulate a different idea with your newly acquired knowledge or move on to the next test idea.In our example, we would leave the control and test rearranging the other components or instead of moving the "Add to Bag" button, we could change the aesthetics of the button to make it stand out more.Now, what does A/B testing have to do with being wrong? Just about everything. We are wrong so many times. Testing allows us to understand the reason why we are wrong thus avoiding mistakes. It is in these scenarios that we learn the most. Every single time I am proven wrong, I learn something that I didn't know before. So, embrace all the times you are wrong and happy learning. A/B testing enables us to make changes in a controlled environment so that we can confidently understand the impact a change has on customer behavior, conversion, and engagement
< Page 8 | Page 10 >