|
|
Power: If the power is set to 80%, the probability of making a type 2 error is about 20%. In order to reduce the risk of making a false judgment when it is actually true, the detection power is set high. Significance (significance≒confidence): Setting significance to 95% means that the probability of causing a type 1 error is 5% or less. Set significance high to reduce the risk of deciding that something is correct when it is actually incorrect. Estimation of test period You can make a trial calculation using the AB test size calculator . Enter the conversion rate from the test page Enter the predicted improvement rate of the key metric Enter the number of unique users for the page Select the power and significance (significance≒confidence) You can estimate the period. Propagate Once we get the test results, we will develop our findings from them.
Propagate What needs to be revealed What were the results? What did you learn from that? What's the next step? Segmentation is important Always segment by device Consider the amount of traffic before segmenting too much Was the traffic sufficient? next step repeat Pivot (if an invalid hypothesis is obtained) Investigate Further India Phone Number Perform new tests for other research opportunities repetition There are various ways to approach iteration. Re-select segments to test variations on Try other variations that didn't apply in that segment Redo the test with the winning pattern vs. the original pattern.
Cycle of iteration * (from top to bottom) Decide on conversions to improve ⇒ Change hypothesis ⇒ Identify and generate variables ⇒ Conduct tests ⇒ Measure results repeat cycle pivot If an invalid hypothesis is obtained, investigate what went wrong. Was the hypothesis wrong? Was the data set incorrect? Was the solution wrong? Investigate Further If we analyze the test results and have collected enough data to show that it is having an impact on users, we treat it as a research opportunity for AB testing, usability testing, etc.
|
|
發表於 2024-3-10 18:33:02