204. AB testing sanity check

easy apple

************

  • **************
    • *****************************************************************************5%,*****************************************
  • ************************************************************************************************************
    • ***********************************************************************************

Table: apple_search

Search id by two different ab testing groups: --0: control, 1, testing

   col_name              | col_type
-------------------------+--------------------------
  search_id              | uuid
  ab_testing_bucket_id   | smallint --0: control, 1, testing

Sample results

 sample_difference_percentage
------------------------------
           7.6923076923076923

Solution postgres

WITH control AS (
    SELECT COUNT(*) AS control_count
    FROM apple_search
    WHERE ab_testing_bucket_id = 0
),
testing AS (
    SELECT COUNT(*) AS testing_count
    FROM apple_search
    WHERE ab_testing_bucket_id = 1
)
SELECT
    (control_count - testing_count) * 100.0 / control_count AS sample_difference_percentage
FROM control, testing;
    

Explanation

This query is related to A/B testing, a method of comparing two versions of a webpage or other product to see which one performs better. A/B testing is essentially an experiment where two or more variants of a page are shown to users at random, and statistical analysis is used to determine which variation performs better for a given conversion goal.

In this case, the apple_search table seems to contain data about searches done on a website, and the column ab_testing_bucket_id is used to track which variant of the webpage the user saw when they did the search. If ab_testing_bucket_id is 0, that means the user saw the control version of the page (i.e., the original version). If ab_testing_bucket_id is 1, that means the user saw the testing version of the page (i.e., the new or different version).

Now, the query is doing the following:

  1. In the control subquery, it is counting the total number of searches done by users who saw the control version of the page. This count is being saved as control_count.

  2. In the testing subquery, it is counting the total number of searches done by users who saw the testing version of the page. This count is being saved as testing_count.

  3. In the main part of the query, it is calculating the difference between the number of searches done on the control version and the testing version, dividing by the number of searches done on the control version, and then multiplying by 100 to get a percentage. This is being saved as sample_difference_percentage.

In other words, the query is calculating what percentage of the total number of searches done on the control version of the page was not done on the testing version. This could be used to measure if there was a significant drop or increase in the number of searches when users saw the testing version of the page compared to the control version.

Expected results


More Apple questions

ID Title Level FTPR
202 Monthly active paid subscriptions apple hard
50%
203 Top product by country by month apple hard
100%
205 Search results recall apple medium
30%
206 Search results recall by testing group apple medium
30%
207 Results shown rate by group apple medium
50%
208 Top 3 urls by testing groups apple hard
10%