Introduction to Fake Door Testing

Strictly speaking, fake door testing is not one of AB testing method. However, it can find out the information which do not have history data. For example, if we want to how many people are interested in the upcoming feature before releasing, and there is no other features like the new one, then it will not use AB testing but fake door testing.

How to Do the Fake Door Testing

As the name of testing, "Fake door" means we need to find a proxy to find out the interest of users. For example, we can make a button for the new feature, but it will show up "under construction" when users click it. Although it cannot directly say that those who click button are really interested in that new feature, it still can tell us some information.

The hardest part of fake door testing is to find proxy. The prefect proxy is low cost, is directly related to the new feature, and cannot reduce the user experience during the testing period.

Example

When I finished my project "handpalm AI", I put a button into chatbot to show "Sorry! This new feature is under construction now. Please wait for our news".
After a month, I found out that there are 35 in 106 people clicking the new button. At this moment, I can expect almost 1/3 users are interested in the new features. I have more confidence about the estimation of sales and revenue when I release the new feature.

Addition

Sometimes, we can collect users information rather than just say "under construction". we can ask those who are interested in the new features to provide their information in exchange for discount and early notification. In this way, we can increase those royalty and get more directly evidence to know how many people are interested.