Overview

Cloud Functions can extend your applications and services by integrating with Google Cloud databases namely Firestore, Cloud Spanner, Cloud SQL, Cloud Bigtable and with Memorystore, Google Cloud's in-memory datastore cache service.

In this lab, you create Cloud Functions that integrate with Firestore, Google Cloud's serverless NoSQL document database. You'll use the Cloud Functions Framework and Firestore client library for Node.js to create functions, and set up triggers to execute them when events occur in the database.

A Firestore function's lifecycle typically involves these steps:

• Wait for changes to a particular document in the Firestore database.

• Trigger when an event occurs.

• Perform tasks using a data object that is received with a snapshot of the affected document.

Objectives

In this lab, you will:

• Set up a Firestore database.

• Develop and deploy an event-driven function to log information when a document is created in Firestore.

• Develop and deploy an event-driven function to update the document contents.

• Access and use Secrets with Cloud Functions.

• Use the Google Cloud console to view logs generated by your function.

Setup

Before you click the Start Lab button

Note: Read these instructions.

Labs are timed and you cannot pause them. The timer, which starts when you click Start Lab, shows how long Google Cloud resources will be made available to you.

This Qwiklabs hands-on lab lets you do the lab activities yourself in a real cloud environment, not in a simulation or demo environment. It does so by giving you new, temporary credentials that you use to sign in and access Google Cloud for the duration of the lab.

What you need

To complete this lab, you need:

• Access to a standard internet browser (Chrome browser recommended).

• Time to complete the lab.

Note: If you already have your own personal Google Cloud account or project, do not use it for this lab.

Note: If you are using a Pixelbook, open an Incognito window to run this lab.

Activate Google Cloud Shell

Google Cloud Shell is a virtual machine that is loaded with development tools. It offers a persistent 5GB home directory and runs on the Google Cloud.

Google Cloud Shell provides command-line access to your Google Cloud resources.

1. In Cloud console, on the top right toolbar, click the Open Cloud Shell button.

   

2. Click Continue.

It takes a few moments to provision and connect to the environment. When you are connected, you are already authenticated, and the project is set to your PROJECT_ID. For example:

gcloud is the command-line tool for Google Cloud. It comes pre-installed on Cloud Shell and supports tab-completion.

• You can list the active account name with this command:

gcloud auth list

Output:

Credentialed accounts:
 - @.com (active)

Example output:

Credentialed accounts:
 - google1623327_student@qwiklabs.net

• You can list the project ID with this command:

gcloud config list project

Output:

[core]
project =

Example output:

[core]
project = qwiklabs-gcp-44776a13dea667a6

Note: Full documentation of gcloud is available in the gcloud CLI overview guide .

Task 1. Set up the environment

In this task, you set up environment variables and enable relevant service APIs that are needed to perform this lab.

Set environment variables

Before you create Cloud Functions, you set some environment variables.

1. Sign in to the Google Cloud console with your lab credentials, and open the Cloud Shell terminal window.

2. Run the following command in Cloud Shell to set your Project ID and REGION environment variables.

     PROJECT_ID=$(gcloud config get-value project)
     REGION=us-east1
   

3. Set an environment variable for the Project Number:

    PROJECT_NUMBER=$(gcloud projects list \
     --filter="project_id:$PROJECT_ID" \
     --format='value(project_number)')
   

4. Set the default region for Cloud Functions:

     gcloud config set functions/region us-east1
   

Enable APIs

1. To enable service APIs that are needed for this lab, run the following command:

    gcloud services enable \
      artifactregistry.googleapis.com \
      cloudfunctions.googleapis.com \
      cloudbuild.googleapis.com \
      eventarc.googleapis.com \
      run.googleapis.com \
      logging.googleapis.com \
      storage.googleapis.com \
      pubsub.googleapis.com
   

Task 2. Set up Firestore

To perform the tasks in this lab, you need to set up a Firestore database. Firestore stores data in the form of documents and collections. To use Cloud Functions with Firestore, you must first set up Firestore before deploying the functions.

1. In the Cloud Console, on the Navigation menu (), click Firestore.

2. Click Create Database.

3. Click Native mode (recommended), and then click Continue.

4. In Location type, click Region, and then select the lab region us-east1 from the list.

   Note: If the list of regions is not populated, refresh the browser or try the wizard again from the Cloud console menu.

5. For Secure rules, select Test rules.

6. Click Create Database.

Click Check my progress to verify the objective.

Set up Firestore.

Check my progress

Task 3. Develop an event-driven function for new Firestore documents

After your Firestore database is created, you can develop your function code. In this task, you write your function's source code that responds to the creation of new documents in the database. The function logs information about the data received in the function invocation.

Set up your working directory

Firestore cloud functions are invoked with a cloudevents data structure that can be decoded using Protocol Buffers with the protobuf.js NPM module. For more information see the links that are provided at the end of the lab.

1. Copy the required .proto and dependency files into a directory named firestore_functions:

    gcloud storage cp -R gs://cloud-training/CBL493/firestore_functions .
   

   Change to the firestore_functions directory:

    cd firestore_functions
   

   The firestore_functions directory also contains empty node.js and package.json files which you will update in the next subtask.

Write the function code

1. In the Cloud Shell toolbar, click Open Editor.

   You can switch between Cloud Shell and the code editor using Open Editor and Open Terminal, or click Open in new window to leave the editor open in a separate tab.

2. In the editor navigation menu, click Explorer (), and then click Open Folder.

3. Select firestore_functions under the student home folder, and then click Ok.

4. In the editor navigation menu, select the index.js file.

5. In the editor, add the following code to the firestore-functions/index.js file:

     /**
      * Cloud Event Function triggered by a change to a Firestore document.
      */
     const functions = require('@google-cloud/functions-framework');
     const protobuf = require('protobufjs');
   
     functions.cloudEvent('newCustomer', async cloudEvent => {
       console.log(`Function triggered by event on: ${cloudEvent.source}`);
       console.log(`Event type: ${cloudEvent.type}`);
   
       console.log('Loading protos...');
       const root = await protobuf.load('data.proto');
       const DocumentEventData = root.lookupType('google.events.cloud.firestore.v1.DocumentEventData');
   
       console.log('Decoding data...');
       const firestoreReceived = DocumentEventData.decode(cloudEvent.data);
   
       console.log('\nNew document:');
       console.log(JSON.stringify(firestoreReceived.value, null, 2));
     });
   

   The code uses the functions framework Node.js library to create a function that processes data delivered using the cloudEvent specification.

6. In the editor, add the following to the firestore-functions/package.json file:

     {
      "name": "firestore_functions",
      "version": "0.0.1",
      "main": "index.js",
      "dependencies": {
        "@google-cloud/functions-framework": "^3.1.3",
        "protobufjs": "^7.2.2",
        "@google-cloud/firestore": "^6.0.0"
      }
     }
   

Update service account permissions

Grant the Cloud Functions service agent certain permissions before deploying the function. Run the following commands in Cloud Shell.

1. Click Open Terminal.

2. Set an environment variable for the Cloud Functions service agent's service account:

    SERVICE_ACCOUNT=service-$PROJECT_NUMBER@gcf-admin-robot.iam.gserviceaccount.com
   

3. To view and get artifacts from Artifact Registry, grant the artifactregistry.reader role to the Cloud Functions service account:

    gcloud projects add-iam-policy-binding $PROJECT_ID --member serviceAccount:$SERVICE_ACCOUNT --role roles/artifactregistry.reader
   

   Note: If you receive an error from the previous step indicating that the service account does not exist, or that the request has invalid authentication credentials, perform the following steps:

4. Disable the Cloud Functions API:

    gcloud services disable cloudfunctions.googleapis.com
   

5. Re-enable the Cloud Functions API:

    gcloud services enable cloudfunctions.googleapis.com
   

6. Wait a few seconds, and then rerun the command to grant the artifactregistry.reader role to the Cloud Functions service account:

    gcloud projects add-iam-policy-binding $PROJECT_ID --member serviceAccount:$SERVICE_ACCOUNT --role roles/artifactregistry.reader
   

Deploy the function

• To deploy the function, run the following command from Cloud Shell:

    gcloud functions deploy newCustomer \
    --gen2 \
    --runtime=nodejs20 \
    --region=us-east1 \
    --trigger-location=us-east1 \
    --source=. \
    --entry-point=newCustomer \
    --trigger-event-filters=type=google.cloud.firestore.document.v1.created \
    --trigger-event-filters=database='(default)' \
    --trigger-event-filters-path-pattern=document='customers/{name}'
  

  Firestore supports the created, updated, deleted, and written events which you specify as the trigger-event-filter option to the deploy command. The path pattern states that all documents in the customers collection should be monitored.

  After the command executes successfully, the command generates the URL for the function endpoint, as shown in this sample partial command output:

    ...
    state: ACTIVE
    updateTime: '2024-03-19T21:38:04.917134057Z'
    url: https://us-east1-us-east1.cloudfunctions.net/newCustomer
  

  If you receive an error when deploying the function, wait a few minutes before retrying the deploy command. The service account permissions for Eventarc might take some time to propagate.

Test the function

1. Navigate to Firestore Studio in the Cloud console.

2. To create a new document collection, click Start collection.

3. For Collection ID, type customers

4. To generate an ID for a document in this collection, click into Document ID.

5. For this document, add a field with the following values:

   | Field name | Field type | Field value | | --- | --- | --- | | firstname | string | Lucas |

6. Click Save.

7. To verify that your cloud function was invoked, on the Navigation menu (), under Serverless, click Cloud Functions.

8. Click the function name newCustomer.

9. Click Logs.

10. Verify that the log entries generated from the function code are present and display the data from the database document that you created.

    You might need to click Refresh to view the latest log entries.

Click Check my progress to verify the objective.

Develop an event-driven function for new Firestore documents.

Check my progress

Task 4. Develop an event-driven function for Firestore to update a document

In this task, you develop a function that is triggered when a document is updated in the Firestore database. Your function adds a new field to the document with a value that is derived from the values of some of the other document's fields.

Write the function code

1. In the editor, add the following code below in the firestore-functions/index.js file:

    const Firestore = require('@google-cloud/firestore');
    const firestore = new Firestore({
      projectId: process.env.GOOGLE_CLOUD_PROJECT,
    });
   
    functions.cloudEvent('updateCustomer', async cloudEvent => {
      console.log('Loading protos...');
      const root = await protobuf.load('data.proto');
      const DocumentEventData = root.lookupType(
       'google.events.cloud.firestore.v1.DocumentEventData'
      );
   
      console.log('Decoding data...');
      const firestoreReceived = DocumentEventData.decode(cloudEvent.data);
   
      const resource = firestoreReceived.value.name;
      const affectedDoc = firestore.doc(resource.split('/documents/')[1]);
   
      // Fullname already exists, so don't update again to avoid infinite loop.
      if (firestoreReceived.value.fields.hasOwnProperty('fullname')) {
        console.log('Fullname is already present in document.');
        return;
      }
   
      if (firestoreReceived.value.fields.hasOwnProperty('lastname')) {
        const lname = firestoreReceived.value.fields.lastname.stringValue;
        const fname = firestoreReceived.value.fields.firstname.stringValue;
        const fullname = `${fname} ${lname}`
        console.log(`Adding fullname --> ${fullname}`);
        await affectedDoc.update({
         fullname: fullname
        });
      }
    });
   

   You can define multiple functions with their own entry points in the same project main file and deploy them separately to Cloud Functions.

   With this approach, every function may share the same set of dependencies even if some of those functions do not need those dependencies.

   To minimize the number of dependencies needed for a particular function and reduce it's memory requirements, it is recommended to keep each function's source code in it's own top-level directory with it's own project configuration files.

Deploy the function

1. To deploy the new function, run the following command from Cloud Shell:

    gcloud functions deploy updateCustomer \
    --gen2 \
    --runtime=nodejs20 \
    --region=us-east1 \
    --trigger-location=us-east1 \
    --source=. \
    --entry-point=updateCustomer \
    --trigger-event-filters=type=google.cloud.firestore.document.v1.updated \
    --trigger-event-filters=database='(default)' \
    --trigger-event-filters-path-pattern=document='customers/{name}'
   

2. Verify the command output indicating that the function has been deployed and the state is Active.

Test the function

1. In the Cloud Console, in Firestore Studio, select the existing documents in the customers collection with a firstname field value of Lucas.

2. For this document, click Add Field.

3. Add a field with the following values:

   | Field name | Field type | Field value | | --- | --- | --- | | lastname | string | Sherman |

4. Click Save Field.

5. Wait for a few seconds, and then verify that you see a new field fullname is added to the document.

   This indicates that your cloud function updateCustomer was invoked when the document was updated.

6. To verify that your cloud function was invoked, on the Navigation menu (), under Serverless, click Cloud Functions.

7. Click the function name updateCustomer.

8. Click Logs.

9. Verify that the log entries generated from the function code are present that indicate that the fullname field was added to the document.

   You might need to click Refresh to view the latest log entries.

Click Check my progress to verify the objective.

Develop an event-driven function for Firestore to update a document.

Check my progress

Task 5. Using Secrets with Cloud Functions

Secret Manager is a Google Cloud service that securely stores data like API keys, passwords, certificates, credentials, and other sensitive information. You can then access these secrets from Cloud Functions or other services for use in your function logic or service implementation.

In this task, you create and store a credential as a secret in Secret Manager. You develop a function to access the key in your function logic.

Create a secret

1. To create and use secrets, run the following command in Cloud Shell and enable the Secret Manager API:

    gcloud services enable secretmanager.googleapis.com
   

2. Create and store a secret named api-cred with value secret_api_key in Secret Manager:

    echo -n "secret_api_key" | gcloud secrets create api-cred --replication-policy="automatic" --data-file=-
   

   The replication-policy is used to determine which regions are used to store secrets and their versions. To specify one or more regions where a secret can be replicated, you can set the replication-policy to user-managed.

Grant access

To access a secret, your function's runtime service account must be granted access to the secret.

• By default, 2nd generation Cloud Functions uses the Compute Engine default service account as a function's runtime service account.

  To authenticate with Secret Manager, grant the Secret Manager Secret Accessor role to the Compute Engine default service account:

    gcloud secrets add-iam-policy-binding api-cred --member=serviceAccount:$PROJECT_NUMBER-compute@developer.gserviceaccount.com --project=$PROJECT_ID --role='roles/secretmanager.secretAccessor'
  

  Note: For production use, you should configure your function to authenticate with its own user-managed service account, that is assigned the least-permissive set of roles required to accomplish that function's tasks.

Access the secret from your function

In this subtask, you modify the previously developed newCustomer function to access the secret.

• In the editor, add the following code to the newCustomer function in the index.js file. Add the code at the end of the function after the last console.log statement in the function body:

     // BEGIN access a secret
     const fs = require('fs/promises');
     try {
       const secret = await fs.readFile('/etc/secrets/api_cred/latest', { encoding: 'utf8' });
       // use the secret. For lab testing purposes, we log the secret.
       console.log('secret: ', secret);
     } catch (err) {
       console.log(err);
     }
     // End access a secret
  

  The function code reads the secret value from a file that is mounted to a volume that is accessible to the function. You configure this method of accessing the secret when you deploy the function in the next step.

Redeploy the function

1. In Cloud Shell, redeploy the newCustomer function with the secret:

    gcloud functions deploy newCustomer \
     --gen2 \
     --runtime=nodejs20 \
     --region=us-east1 \
     --trigger-location=us-east1 \
     --source=. \
     --entry-point=newCustomer \
     --trigger-event-filters=type=google.cloud.firestore.document.v1.created \
     --trigger-event-filters=database='(default)' \
     --trigger-event-filters-path-pattern=document='customers/{name}' \
     --set-secrets '/etc/secrets/api_cred/latest=api-cred:latest'
   

   The set-secrets deploy command option specifies the mount path: /etc/secrets/api_cred, and the secret path: /latest.

   By referencing a secret as a volume, your function accesses the latest secret value from Secret Manager each time the file is read from disk.

   Your function can also access a specific version of the secret value as an environment variable. View the documentation on using secrets with Cloud Functions for more information.

2. After the function is deployed, verify that it has access to the secret:

    gcloud functions describe newCustomer
   

   The output from the describe command includes information about the secret. Here's a partial output from the command:

    ...
    secretVolumes:
    - mountPath: /etc/secrets/api_cred
      projectId: 'qwiklabs-gcp-04-1024504e6418'
      secret: api-cred
      versions:
      - path: /latest
        version: latest
    ...
   

Test the function

1. To test the function, repeat the test from the previous task to add a new customer document from Firestore Studio in Cloud console.

2. To view the function's logs in the Cloud console, in the navigation menu, click Cloud Functions, and then click the newCustomer function name.

3. To view the function's logs, click Logs.

   Verify that the entry to log the value of the secret key is present:

    secret: secret_api_key
   

Click Check my progress to verify the objective.

---

Solution of Lab

Set up Firestore

• Navigation Firestore

• Click Create Database

• Click Native mode (recommended), and then click Continue

• Choose Region from lab

• For Secure rules, select Test rules

Create and Save a Collection

1. Click Start collection.

2. Fill in the details as follows, then click Save:

   • Collection ID: customers

   • Document ID: Randomly Genarate

3. Add Fields to the collection:

   • Field name: firstname

     • Field type: string

     • Field value: Lucas

   • Click the Add a Field (+) button to add another field:

     • Field name: lastname

       • Field type: string

       • Field value: Sherman

4. Click Save to finalize the collection.

---

Open the Cloud Shell terminal window.

export REGION=

curl -LO raw.githubusercontent.com/quiccklabs/Labs_solutions/master/Integrating%20Cloud%20Functions%20with%20Firestore/quicklab.sh
sudo chmod +x quicklab.sh
./quicklab.sh

---

Wait for command to execute.

Now Delete the old field and again create the same.

Field name: lastname

• Field type: string

• Field value: Sherman