Derive Insights from BigQuery Data: Challenge Lab - GSP787

Overview

You must complete a series of tasks within the allocated time period. Instead of following step-by-step instructions, you'll be given a scenario and a set of tasks - you figure out how to complete it on your own! An automated scoring system (shown on this page) will provide feedback on whether you have completed your tasks correctly.

To score 100% you must complete all tasks within the time period!

When you take a Challenge Lab, you will not be taught Google Cloud concepts. To build the solution to the challenge presented, use skills learned from the labs in the course this challenge lab is part of. You will be expected to extend your learned skills; you will be expected to change broken queries.

This lab is recommended for students who have enrolled in the Derive Insights from BigQuery Data skill badge. Are you ready for the challenge?

Setup and requirements

Before you click the Start Lab button

Read these instructions. Labs are timed and you cannot pause them. The timer, which starts when you click Start Lab, shows how long Google Cloud resources are made available to you.

This hands-on lab lets you do the lab activities in a real cloud environment, not in a simulation or demo environment. It does so by giving you new, temporary credentials you use to sign in and access Google Cloud for the duration of the lab.

To complete this lab, you need:

• Access to a standard internet browser (Chrome browser recommended).

Note: Use an Incognito (recommended) or private browser window to run this lab. This prevents conflicts between your personal account and the student account, which may cause extra charges incurred to your personal account.

• Time to complete the lab—remember, once you start, you cannot pause a lab.

Note: Use only the student account for this lab. If you use a different Google Cloud account, you may incur charges to that account.

Scenario

You're part of a public health organization which is tasked with identifying answers to queries related to the Covid-19 pandemic. Obtaining the right answers will help the organization in planning and focusing healthcare efforts and awareness programs appropriately.

The dataset and table that will be used for this analysis will be : bigquery-public-data.covid19_open_data.covid19_open_data. This repository contains country-level datasets of daily time-series data related to COVID-19 globally. It includes data relating to demographics, economy, epidemiology, geography, health, hospitalizations, mobility, government response, and weather.

Task 1. Total confirmed cases

• Build a query that will answer "What was the total count of confirmed cases on May 15, 2020?" The query needs to return a single row containing the sum of confirmed cases across all countries. The name of the column should be total_cases_worldwide.

Columns to reference:

• cumulative_confirmed

• date

Click Check my progress to verify the objective.

Total Confirmed Cases

Task 2. Worst affected areas

• Build a query for answering "How many states in the US had more than 100 deaths on May 15, 2020?" The query needs to list the output in the field count_of_states.

Note: Don't include NULL values.

Columns to reference:

• country_name

• subregion1_name (for state information)

• cumulative_deceased

Click Check my progress to verify the objective.

Worst Affected Areas

Task 3. Identify hotspots

• Build a query that will answer "List all the states in the United States of America that had more than 1500 confirmed cases on May 15, 2020?" The query needs to return the State Name and the corresponding confirmed cases arranged in descending order. Name of the fields to return state and total_confirmed_cases.

Columns to reference:

• country_code

• subregion1_name (for state information)

• cumulative_confirmed

Click Check my progress to verify the objective.

Identifying Hotspots

Task 4. Fatality ratio

1. Build a query that will answer "What was the case-fatality ratio in Italy for the month of April 2020?" Case-fatality ratio here is defined as (total deaths / total confirmed cases) * 100.

2. Write a query to return the ratio for the month of April 2020 and contain the following fields in the output: total_confirmed_cases, total_deaths, case_fatality_ratio.

Columns to reference:

• country_name

• cumulative_confirmed

• cumulative_deceased

Click Check my progress to verify the objective.

Fatality Ratio

Task 5. Identifying specific day

• Build a query that will answer: "On what day did the total number of deaths cross 8000 in Italy?" The query should return the date in the format yyyy-mm-dd.

Columns to reference:

• country_name

• cumulative_deceased

Click Check my progress to verify the objective.

Identifying Hotspots

Task 6. Finding days with zero net new cases

The following query is written to identify the number of days in India between 22, Feb 2020 and 13, March 2020 when there were zero increases in the number of confirmed cases. However it is not executing properly.

• You need to update the query to complete it and obtain the result:

WITH india_cases_by_date AS (
  SELECT
    date,
    SUM(cumulative_confirmed) AS cases
  FROM
    `bigquery-public-data.covid19_open_data.covid19_open_data`
  WHERE
    country_name="India"
    AND date between '2020-02-22' and '2020-03-13'
  GROUP BY
    date
  ORDER BY
    date ASC
 )

, india_previous_day_comparison AS
(SELECT
  date,
  cases,
  LAG(cases) OVER(ORDER BY date) AS previous_day,
  cases - LAG(cases) OVER(ORDER BY date) AS net_new_cases
FROM india_cases_by_date
)

Click Check my progress to verify the objective.

Finding days with zero net new cases

Task 7. Doubling rate

• Using the previous query as a template, write a query to find out the dates on which the confirmed cases increased by more than 5% compared to the previous day (indicating doubling rate of ~ 7 days) in the US between the dates March 22, 2020 and April 20, 2020. The query needs to return the list of dates, the confirmed cases on that day, the confirmed cases the previous day, and the percentage increase in cases between the days.

  • Use the following names for the returned fields: Date, Confirmed_Cases_On_Day, Confirmed_Cases_Previous_Day and Percentage_Increase_In_Cases.

Click Check my progress to verify the objective.

Doubling rate

Task 8. Recovery rate

1. Build a query to list the recovery rates of countries arranged in descending order (limit to 5) on the date May 10, 2020.

2. Restrict the query to only those countries having more than 50K confirmed cases.

   • The query needs to return the following fields: country, recovered_cases, confirmed_cases, recovery_rate.

Columns to reference:

• country_name

• cumulative_confirmed

• cumulative_recovered

Click Check my progress to verify the objective.

Recovery rate

Task 9. CDGR - Cumulative daily growth rate

• The following query is trying to calculate the CDGR on May 15, 2020(Cumulative Daily Growth Rate) for France since the day the first case was reported.The first case was reported on Jan 24, 2020.

• The CDGR is calculated as:

((last_day_cases/first_day_cases)^1/days_diff)-1)

Where :

• last_day_cases is the number of confirmed cases on May 10, 2020

• first_day_cases is the number of confirmed cases on Jan 24, 2020

• days_diff is the number of days between Jan 24 - May 10, 2020

• The query isn’t executing properly. Can you fix the error to make the query execute successfully?

WITH
  france_cases AS (
  SELECT
    date,
    SUM(cumulative_confirmed) AS total_cases
  FROM
    `bigquery-public-data.covid19_open_data.covid19_open_data`
  WHERE
    country_name="France"
    AND date IN ('2020-01-24',
      '2020-05-15')
  GROUP BY
    date
  ORDER BY
    date)
, summary as (
SELECT
  total_cases AS first_day_cases,
  LEAD(total_cases) AS last_day_cases,
  DATE_DIFF(LEAD(date) OVER(ORDER BY date),date, day) AS days_diff
FROM
  france_cases
LIMIT 1
)

select first_day_cases, last_day_cases, days_diff, SQRT((last_day_cases/first_day_cases),(1/days_diff))-1 as cdgr
from summary

Note: Refer to the following Functions, operators, and conditionals documentation to learn more about the SQL function referenced LEAD().

Click Check my progress to verify the objective.

CDGR - Cumulative Daily Growth Rate

Task 10. Create a Looker Studio report

• Create a Looker Studio report that plots the following for the United States:

• Use the BigQuery connector, authorize access, select Custom Query under your project qwiklabs-gcp-03-e96502303e35, enter the query, then click Add and Add to report.

  • Number of Confirmed Cases

  • Number of Deaths

  • Date range : 2020-03-25 to 2020-04-24

Click Check my progress to verify the objective.

Create a Looker Studio report

Note: Use the below image as a reference while building the report and make sure your report is similar to it.

Note: Do not use the Explore with Looker Studio option from BigQuery.

---

Solution of Lab

Quick

curl -LO raw.githubusercontent.com/ePlus-DEV/storage/refs/heads/main/labs/GSP787/lab.sh
source lab.sh

Script Alternative

curl -LO https://raw.githubusercontent.com/Itsabhishek7py/GoogleCloudSkillsboost/refs/heads/main/Derive%20Insights%20from%20BigQuery%20Data%3A%20Challenge%20Lab/abhishek.sh
sudo chmod +x abhishek.sh
./abhishek.sh

SELECT
  DATE(date) AS date
FROM (
  SELECT
    date,
    SUM(cumulative_deceased) AS total_deaths
  FROM
    `bigquery-public-data.covid19_open_data.covid19_open_data`
  WHERE
    country_name = 'Italy'
    AND date >= '2020-01-01'
  GROUP BY
    date
)
WHERE
  total_deaths > 8000
ORDER BY
  date ASC
LIMIT 1;

---

Manual

https://www.youtube.com/watch?v=rO1I0cB6SEY

https://www.youtube.com/watch?v=_iRWtfBM7UQ

https://www.youtube.com/watch?v=2meJLDyQ3MM

Task 1. Total confirmed cases

SELECT
  SUM(cumulative_confirmed) AS total_cases_worldwide
FROM
  `bigquery-public-data.covid19_open_data.covid19_open_data`
WHERE
  date = '2020-05-15';

Task 2. Worst affected areas

SELECT
  COUNT(*) AS count_of_states
FROM (
  SELECT
    subregion1_name,
    SUM(cumulative_deceased) AS total_deaths
  FROM
    `bigquery-public-data.covid19_open_data.covid19_open_data`
  WHERE
    country_name = 'United States of America'
    AND date = '2020-05-15'
    AND subregion1_name IS NOT NULL
  GROUP BY
    subregion1_name
  HAVING
    SUM(cumulative_deceased) > 250
);

Task 3. Identify hotspots

SELECT
  subregion1_name AS state,
  SUM(cumulative_confirmed) AS total_confirmed_cases
FROM
  `bigquery-public-data.covid19_open_data.covid19_open_data`
WHERE
  country_name = 'United States of America'
  AND date = '2020-05-15'
  AND subregion1_name IS NOT NULL
GROUP BY
  subregion1_name
HAVING
  total_confirmed_cases > 1500
ORDER BY
  total_confirmed_cases DESC;

Task 4. Fatality ratio

SELECT
  SUM(cumulative_confirmed) AS total_confirmed_cases,
  SUM(cumulative_deceased) AS total_deaths,
  (SUM(cumulative_deceased) / SUM(cumulative_confirmed)) * 100 AS case_fatality_ratio
FROM
  `bigquery-public-data.covid19_open_data.covid19_open_data`
WHERE
  country_name = 'Italy'
  AND date BETWEEN '2020-05-01' AND '2020-05-31';

Task 5. Identify a specific day

SELECT
  date
FROM (
  SELECT
    date,
    SUM(cumulative_deceased) AS total_deaths
  FROM
    `bigquery-public-data.covid19_open_data.covid19_open_data`
  WHERE
    country_name = 'Italy'
  GROUP BY
    date
)
WHERE
  total_deaths > 10000
ORDER BY
  date ASC
LIMIT 1;

Task 6. Find days with zero net new cases

WITH india_cases_by_date AS (
  SELECT
    date,
    SUM(cumulative_confirmed) AS cases
  FROM
    `bigquery-public-data.covid19_open_data.covid19_open_data`
  WHERE
    country_name = 'India'
    AND date BETWEEN '2020-02-21' AND '2020-03-12'
  GROUP BY
    date
  ORDER BY
    date ASC
),

india_previous_day_comparison AS (
  SELECT
    date,
    cases,
    LAG(cases) OVER (ORDER BY date) AS previous_day,
    cases - LAG(cases) OVER (ORDER BY date) AS net_new_cases
  FROM
    india_cases_by_date
)

SELECT
  COUNT(*) AS days_with_zero_net_new_cases
FROM
  india_previous_day_comparison
WHERE
  net_new_cases = 0;

Task 7. Doubling rate

WITH us_cases_by_date AS (
  SELECT
    date,
    SUM(cumulative_confirmed) AS cases
  FROM
    `bigquery-public-data.covid19_open_data.covid19_open_data`
  WHERE
    country_name = 'United States of America'
    AND date BETWEEN '2020-03-22' AND '2020-04-20'
  GROUP BY
    date
),

us_previous_day_comparison AS (
  SELECT
    date,
    cases,
    LAG(cases) OVER (ORDER BY date) AS previous_day
  FROM
    us_cases_by_date
)

SELECT
  date AS Date,
  cases AS Confirmed_Cases_On_Day,
  previous_day AS Confirmed_Cases_Previous_Day,
  ((cases - previous_day) / previous_day) * 100
    AS Percentage_Increase_In_Cases
FROM
  us_previous_day_comparison
WHERE
  ((cases - previous_day) / previous_day) * 100 > 20
ORDER BY
  date;

Task 8. Recovery rate

WITH country_data AS (
  SELECT
    country_name AS country,
    SUM(cumulative_recovered) AS recovered_cases,
    SUM(cumulative_confirmed) AS confirmed_cases
  FROM
    `bigquery-public-data.covid19_open_data.covid19_open_data`
  WHERE
    date = '2020-05-10'
  GROUP BY
    country_name
)

SELECT
  country,
  recovered_cases,
  confirmed_cases,
  (recovered_cases / confirmed_cases) * 100 AS recovery_rate
FROM
  country_data
WHERE
  confirmed_cases > 50000
ORDER BY
  recovery_rate DESC
LIMIT 20;

Task 9. CDGR - Cumulative daily growth rate

WITH france_cases AS (
  SELECT
    date,
    SUM(cumulative_confirmed) AS total_cases
  FROM
    `bigquery-public-data.covid19_open_data.covid19_open_data`
  WHERE
    country_name = 'France'
    AND date IN ('2020-01-24', '2020-05-15')
  GROUP BY
    date
  ORDER BY
    date
),

summary AS (
  SELECT
    total_cases AS first_day_cases,
    LEAD(total_cases) OVER (ORDER BY date) AS last_day_cases,
    DATE_DIFF(
      LEAD(date) OVER (ORDER BY date),
      date,
      DAY
    ) AS days_diff
  FROM
    france_cases
  LIMIT 1
)

SELECT
  first_day_cases,
  last_day_cases,
  days_diff,
  POW(
    (last_day_cases / first_day_cases),
    (1 / days_diff)
  ) - 1 AS cdgr
FROM
  summary;

Task 10. Create a Data Studio report

SELECT
  date,
  SUM(cumulative_confirmed) AS country_cases,
  SUM(cumulative_deceased) AS country_deaths
FROM
  `bigquery-public-data.covid19_open_data.covid19_open_data`
WHERE
  country_name = 'United States of America'
  AND date BETWEEN '2020-03-19' AND '2020-04-29'
GROUP BY
  date
ORDER BY
  date;