# Creating PDFs with Go and Cloud Run - GSP762 ## **Overview** In this lab you will build a PDF converter web app on Cloud Run, which is a serverless service, that automatically converts files stored in Google Drive into PDFs stored in segregated Google Drive folders. ## **Objectives** In this lab, you will: * Convert a Go application to a container * Learn how to build containers with Google Cloud Build * Create a Cloud Run service that converts files to PDF files in the cloud. * Understand how to create Service Accounts and add permissions * Use event processing with Cloud Storage ## **Setup and requirements** ### Before you click the Start Lab button Read these instructions. Labs are timed and you cannot pause them. The timer, which starts when you click **Start Lab**, shows how long Google Cloud resources will be made available to you. This hands-on lab lets you do the lab activities yourself in a real cloud environment, not in a simulation or demo environment. It does so by giving you new, temporary credentials that you use to sign in and access Google Cloud for the duration of the lab. To complete this lab, you need: * Access to a standard internet browser (Chrome browser recommended). **Note:** Use an Incognito or private browser window to run this lab. This prevents any conflicts between your personal account and the Student account, which may cause extra charges incurred to your personal account. * Time to complete the lab---remember, once you start, you cannot pause a lab. **Note:** If you already have your own personal Google Cloud account or project, do not use it for this lab to avoid extra charges to your account. ### How to start your lab and sign in to the Google Cloud console 1. Click the **Start Lab** button. If you need to pay for the lab, a pop-up opens for you to select your payment method. On the left is the **Lab Details** panel with the following: * The **Open Google Cloud console** button * Time remaining * The temporary credentials that you must use for this lab * Other information, if needed, to step through this lab 2. Click **Open Google Cloud console** (or right-click and select **Open Link in Incognito Window** if you are running the Chrome browser). The lab spins up resources, and then opens another tab that shows the **Sign in** page. ***Tip:*** Arrange the tabs in separate windows, side-by-side. **Note:** If you see the **Choose an account** dialog, click **Use Another Account**. 3. If necessary, copy the **Username** below and paste it into the **Sign in** dialog. ```apache student-04-cfc4b677fd59@qwiklabs.net ``` You can also find the **Username** in the **Lab Details** panel. 4. Click **Next**. 5. Copy the **Password** below and paste it into the **Welcome** dialog. ```apache gF3jvDXMAOHB ``` You can also find the **Password** in the **Lab Details** panel. 6. Click **Next**. **Important:** You must use the credentials the lab provides you. Do not use your Google Cloud account credentials. **Note:** Using your own Google Cloud account for this lab may incur extra charges. 7. Click through the subsequent pages: * Accept the terms and conditions. * Do not add recovery options or two-factor authentication (because this is a temporary account). * Do not sign up for free trials. After a few moments, the Google Cloud console opens in this tab. **Note:** To view a menu with a list of Google Cloud products and services, click the **Navigation menu** at the top-left. ![Navigation menu icon](https://cdn.qwiklabs.com/nUxFb6oRFr435O3t6V7WYJAjeDFcrFb16G9wHWp5BzU%3D align="left") ### Activate Cloud Shell Cloud Shell is a virtual machine that is loaded with development tools. It offers a persistent 5GB home directory and runs on the Google Cloud. Cloud Shell provides command-line access to your Google Cloud resources. 1. Click **Activate Cloud Shell** ![Activate Cloud Shell icon](https://cdn.qwiklabs.com/ep8HmqYGdD%2FkUncAAYpV47OYoHwC8%2Bg0WK%2F8sidHquE%3D align="left") at the top of the Google Cloud console. When you are connected, you are already authenticated, and the project is set to your **Project\_ID**, `qwiklabs-gcp-02-1b9e6c8e61fc`. The output contains a line that declares the **Project\_ID** for this session: ```json Your Cloud Platform project in this session is set to qwiklabs-gcp-02-1b9e6c8e61fc ``` `gcloud` is the command-line tool for Google Cloud. It comes pre-installed on Cloud Shell and supports tab-completion. 2. (Optional) You can list the active account name with this command: ```apache gcloud auth list ``` 3. Click **Authorize**. **Output:** ```apache ACTIVE: * ACCOUNT: student-04-cfc4b677fd59@qwiklabs.net To set the active account, run: $ gcloud config set account `ACCOUNT` ``` 4. (Optional) You can list the project ID with this command: ```apache gcloud config list project ``` **Output:** ```apache [core] project = qwiklabs-gcp-02-1b9e6c8e61fc ``` **Note:** For full documentation of `gcloud`, in Google Cloud, refer to [the gcloud CLI overview guide](https://cloud.google.com/sdk/gcloud). ## **Architecture** In this lab you will assist the Pet Theory Veterinary practice to automatically convert their invoices into PDFs so that customers can open them reliably. ![Architecture diagram](https://cdn.qwiklabs.com/YPjO7fgN961vvryqb0siJegP8R2AAfiYsnvaEFRKO2M%3D align="left") ## **Using Google APIs** During this lab you will use Google APIs. The following APIs have been enabled for you: | **Name** | **API** | | --- | --- | | Cloud Build | cloudbuild.googleapis.com | | Cloud Storage | storage-component.googleapis.com | | Cloud Run | run.googleapis.com | ## **Task 1. Get the source code** Get started by downloading the code necessary for this lab. 1. Activate your lab account: ```apache gcloud auth list --filter=status:ACTIVE --format="value(account)" ``` 2. Run the following to clone the Pet Theory repository: ```apache git clone https://github.com/Deleplace/pet-theory.git ``` 3. Move to the correct directory: ```apache cd pet-theory/lab03 ``` ## **Task 2. Creating an invoice microservice** In this section you will create a Go application to process requests. As outlined in the architecture diagram, you will integrate Cloud Storage as part of the solution. 1. Click the **Open Editor** icon and then click **Open in a new window**. 2. Navigate to **pet-theory** > **lab03** > **server.go** 3. Open the `server.go` source code and edit it to match the text below: ```python package main import ( "fmt" "log" "net/http" "os" "os/exec" "regexp" "strings" ) func main() { http.HandleFunc("/", process) port := os.Getenv("PORT") if port == "" { port = "8080" log.Printf("Defaulting to port %s", port) } log.Printf("Listening on port %s", port) err := http.ListenAndServe(fmt.Sprintf(":%s", port), nil) log.Fatal(err) } func process(w http.ResponseWriter, r *http.Request) { log.Println("Serving request") if r.Method == "GET" { fmt.Fprintln(w, "Ready to process POST requests from Cloud Storage trigger") return } // // Read request body containing Cloud Storage object metadata // gcsInputFile, err1 := readBody(r) if err1 != nil { log.Printf("Error reading POST data: %v", err1) w.WriteHeader(http.StatusBadRequest) fmt.Fprintf(w, "Problem with POST data: %v \n", err1) return } // // Working directory (concurrency-safe) localDir, err := os.MkdirTemp("", "") if err != nil { log.Printf("Error creating local temp dir: %v", err) w.WriteHeader(http.StatusInternalServerError) fmt.Fprintf(w, "Could not create a temp directory on server. \n") return } defer os.RemoveAll(localDir) // // Download input file from Cloud Storage // localInputFile, err2 := download(gcsInputFile, localDir) if err2 != nil { log.Printf("Error downloading Cloud Storage file [%s] from bucket [%s]: %v", gcsInputFile.Name, gcsInputFile.Bucket, err2) w.WriteHeader(http.StatusInternalServerError) fmt.Fprintf(w, "Error downloading Cloud Storage file [%s] from bucket [%s]", gcsInputFile.Name, gcsInputFile.Bucket) return } // // Use LibreOffice to convert local input file to local PDF file. // localPDFFilePath, err3 := convertToPDF(localInputFile.Name(), localDir) if err3 != nil { log.Printf("Error converting to PDF: %v", err3) w.WriteHeader(http.StatusInternalServerError) fmt.Fprintf(w, "Error converting to PDF.") return } // // Upload the freshly generated PDF to Cloud Storage // targetBucket := os.Getenv("PDF_BUCKET") err4 := upload(localPDFFilePath, targetBucket) if err4 != nil { log.Printf("Error uploading PDF file to bucket [%s]: %v", targetBucket, err4) w.WriteHeader(http.StatusInternalServerError) fmt.Fprintf(w, "Error downloading Cloud Storage file [%s] from bucket [%s]", gcsInputFile.Name, gcsInputFile.Bucket) return } // // Delete the original input file from Cloud Storage. // err5 := deleteGCSFile(gcsInputFile.Bucket, gcsInputFile.Name) if err5 != nil { log.Printf("Error deleting file [%s] from bucket [%s]: %v", gcsInputFile.Name, gcsInputFile.Bucket, err5) // This is not a blocking error. // The PDF was successfully generated and uploaded. } log.Println("Successfully produced PDF") fmt.Fprintln(w, "Successfully produced PDF") } func convertToPDF(localFilePath string, localDir string) (resultFilePath string, err error) { log.Printf("Converting [%s] to PDF", localFilePath) cmd := exec.Command("libreoffice", "--headless", "--convert-to", "pdf", "--outdir", localDir, localFilePath) cmd.Stdout, cmd.Stderr = os.Stdout, os.Stderr log.Println(cmd) err = cmd.Run() if err != nil { return "", err } pdfFilePath := regexp.MustCompile(`\.\w+$`).ReplaceAllString(localFilePath, ".pdf") if !strings.HasSuffix(pdfFilePath, ".pdf") { pdfFilePath += ".pdf" } log.Printf("Converted %s to %s", localFilePath, pdfFilePath) return pdfFilePath, nil } ``` 4. Now run the following to build the application: ```apache go build -o server ``` The functions called by this top-level code are in source files: * server.go * notification.go * gcs.go With the application has been successfully built, you can create the pdf-conversion service. ## **Task 3. Create a pdf-conversion service** The PDF service will use Cloud Run and Cloud Storage to initiate a process each time a file is uploaded to the designated storage. To achieve this you will use a common pattern of event notifications together with Cloud Pub/Sub. Doing this enables the application to concentrate only on processing information. Transporting and passing information is performed by other services, which allows you to keep the application simple. Building the invoice module requires the integration of two components: ![Container including two components: server and LibreOffice](https://cdn.qwiklabs.com/5QtYIdxfEJ0t%2BtmbXG%2BWNvKFHMlLEaHgo4a4Drz2k54%3D align="left") Adding the LibreOffice package means it can be used in your application. 1. In the **Open editor**, Open the existing `Dockerfile` manifest and update the file as shown below: ```dockerfile FROM amd64/debian RUN apt-get update -y \ && apt-get install -y libreoffice \ && apt-get clean WORKDIR /usr/src/app COPY server . CMD [ "./server" ] ``` 2. **Save** the updated `Dockerfile`. 3. Initiate a rebuild of the `pdf-converter` image using Cloud Build: ```apache gcloud builds submit \ --tag gcr.io/$GOOGLE_CLOUD_PROJECT/pdf-converter ``` Click **Check my progress** to verify that you've performed the above task. Build an image with Cloud Build **Check my progress** 4. Deploy the updated pdf-converter service. **Note:** It's a good idea to give LibreOffice 2GB of RAM to work with, see the line with the `--memory` option. 5. Run these commands to build the container and to deploy it: ```apache gcloud run deploy pdf-converter \ --image gcr.io/$GOOGLE_CLOUD_PROJECT/pdf-converter \ --platform managed \ --region us-central1 \ --memory=2Gi \ --no-allow-unauthenticated \ --set-env-vars PDF_BUCKET=$GOOGLE_CLOUD_PROJECT-processed \ --max-instances=3 ``` Click **Check my progress** to verify that you've performed the above task. PDF Converter service deployed **Check my progress** The Cloud Run service has now been successfully deployed. However we deployed an application that requires the correct permissions to access it. ## **Task 4. Create a Service Account** A [Service Account](https://cloud.google.com/iam/docs/understanding-service-accounts) is a special type of account with access to Google APIs. In this lab uses a Service Account to access Cloud Run when a Cloud Storage event is processed. Cloud Storage supports a rich set of notifications that can be used to trigger events. Next, update the code to notify the application when a file has been uploaded. 1. Click the **Navigation menu** > **Cloud Storage**, and verify that two buckets have been created. You should see: * `qwiklabs-gcp-02-1b9e6c8e61fc`\-processed * `qwiklabs-gcp-02-1b9e6c8e61fc`\-upload 2. Create a Pub/Sub notification to indicate a new file has been uploaded to the docs bucket ("uploaded"). The notifications will be labeled with the topic "new-doc". ```apache gsutil notification create -t new-doc -f json -e OBJECT_FINALIZE gs://$GOOGLE_CLOUD_PROJECT-upload ``` **Expected Output:** ```bash Created Cloud Pub/Sub topic projects/qwiklabs-gcp-02-1b9e6c8e61fc/topics/new-doc Created notification config projects/_/buckets/qwiklabs-gcp-02-1b9e6c8e61fc-upload/notificationConfigs/1 ``` 3. Create a new service account to trigger the Cloud Run services: ```apache gcloud iam service-accounts create pubsub-cloud-run-invoker --display-name "PubSub Cloud Run Invoker" ``` **Expected Output:** ```apache Created service account [pubsub-cloud-run-invoker]. ``` 4. Give the service account permission to invoke the PDF converter service: ```apache gcloud run services add-iam-policy-binding pdf-converter \ --member=serviceAccount:pubsub-cloud-run-invoker@$GOOGLE_CLOUD_PROJECT.iam.gserviceaccount.com \ --role=roles/run.invoker \ --region us-central1 \ --platform managed ``` **Expected Output:** ```apache Updated IAM policy for service [pdf-converter]. bindings: - members: - serviceAccount:pubsub-cloud-run-invoker@qwiklabs-gcp-02-1b9e6c8e61fc.iam.gserviceaccount.com role: roles/run.invoker etag: BwYYfbXS240= version: 1 ``` 5. Find your project number by running this command: ```apache PROJECT_NUMBER=$(gcloud projects list \ --format="value(PROJECT_NUMBER)" \ --filter="$GOOGLE_CLOUD_PROJECT") ``` 6. Enable your project to create Cloud Pub/Sub authentication tokens: ```apache gcloud projects add-iam-policy-binding $GOOGLE_CLOUD_PROJECT \ --member=serviceAccount:qwiklabs-gcp-02-1b9e6c8e61fc@qwiklabs-gcp-02-1b9e6c8e61fc.iam.gserviceaccount.com \ --role=roles/iam.serviceAccountTokenCreator ``` Click **Check my progress** to verify that you've performed the above task. Service Account created **Check my progress** With the Service Account created it can be used to invoke the Cloud Run Service. ## **Task 5. Testing the Cloud Run service** Before progressing further, test the deployed service. Remember the service requires authentication, so test that to ensure it is actually private. 1. Save the URL of your service in the environment variable **$SERVICE\_URL**: ```apache SERVICE_URL=$(gcloud run services describe pdf-converter \ --platform managed \ --region us-central1 \ --format "value(status.url)") ``` 2. Display the SERVICE URL: ```apache echo $SERVICE_URL ``` 3. Make an anonymous GET request to your new service: ```apache curl -X GET $SERVICE_URL ``` **Expected Output:** ```xml 403 Forbidden

Error: Forbidden

Your client does not have permission to get URL / from this server.

``` **NOTE:** The anonymous GET request will result in an error message: `"Your client does not have permission to get URL"`. This is good; you don't want the service to be callable by anonymous users. 4. Now try invoking the service as an authorized user: ```apache curl -X GET -H "Authorization: Bearer $(gcloud auth print-identity-token)" $SERVICE_URL ``` **Expected Output:** ```apache Ready to process POST requests from Cloud Storage trigger ``` Great work, you have successfully deployed an authenticated Cloud Run service. ## **Task 6. Cloud Storage trigger** To initiate a notification when new content is uploaded to Cloud Storage, add a subscription to your existing Pub/Sub Topic. **Note:** Cloud Storage notifications will automatically push a message to your Topic queue when new content is uploaded. Using notifications allows you to create powerful applications that respond to events without needing to write additional code. * Create a Pub/Sub subscription so that the PDF converter will be run whenever a message is published to the topic `new-doc`: ```apache gcloud pubsub subscriptions create pdf-conv-sub \ --topic new-doc \ --push-endpoint=$SERVICE_URL \ --push-auth-service-account=pubsub-cloud-run-invoker@$GOOGLE_CLOUD_PROJECT.iam.gserviceaccount.com ``` **Expected Output:** ```apache Created subscription [projects/qwiklabs-gcp-02-1b9e6c8e61fc/subscriptions/pdf-conv-sub]. ``` Click **Check my progress** to verify that you've performed the above task. Confirm Pub/Sub Subscription **Check my progress** Now whenever a file is uploaded the Pub/Sub subscription will interact with your Service Account. The Service Account will then initiate your PDF Converter Cloud Run service. ## **Task 7. Testing Cloud Storage notification** To test the Cloud Run service, use the example files available. 1. Copy the test files into your upload bucket: ```apache gsutil -m cp -r gs://spls/gsp762/* gs://$GOOGLE_CLOUD_PROJECT-upload ``` **Expected Output:** ```bash Copying gs://spls/gsp762/cat-and-mouse.jpg [Content-Type=image/jpeg]... Copying gs://spls/gsp762/file-sample_100kB.doc [Content-Type=application/msword]... Copying gs://spls/gsp762/file-sample_500kB.docx [Content-Type=application/vnd.openxmlformats-officedocument.wordprocessingml.document]... Copying gs://spls/gsp762/file_example_XLS_10.xls [Content-Type=application/vnd.ms-excel]... Copying gs://spls/gsp762/file-sample_1MB.docx [Content-Type=application/vnd.openxmlformats-officedocument.wordprocessingml.document]... Copying gs://spls/gsp762/file_example_XLSX_50.xlsx [Content-Type=application/vnd.openxmlformats-officedocument.spreadsheetml.sheet]... Copying gs://spls/gsp762/file_example_XLS_100.xls [Content-Type=application/vnd.ms-excel]... Copying gs://spls/gsp762/file_example_XLS_50.xls [Content-Type=application/vnd.ms-excel]... Copying gs://spls/gsp762//Copy of cat-and-mouse.jpg [Content-Type=image/jpeg]... ``` 2. In the Cloud Console, click **Cloud Storage > Buckets** followed by the bucket name whose name ends in "`qwiklabs-gcp-02-1b9e6c8e61fc`\-upload" 3. Click the **Refresh** button a few times and see how the files are deleted, one by one, as they are converted to PDFs. 4. Then click **Buckets**, followed by the bucket whose name ends in "`qwiklabs-gcp-02-1b9e6c8e61fc`\-processed". It should contain PDF versions of all files. **NOTE:** It can take a few minutes for the processing of the files. Use the Bucket refresh option to check the processing completion state. 5. Feel free to open the PDF files to make sure they were properly converted. 6. Once the upload is done, click **Navigation menu > Cloud Run** and click on the **pdf-converter** service. 7. Select the **LOGS** tab and add a filter of "Converting" to see the converted files. 8. Navigate to **Navigation menu > Cloud Storage** and open the bucket name ending in "`qwiklabs-gcp-02-1b9e6c8e61fc`\-upload" to confirm all files uploaded have been processed. Excellent work, you have successfully built a new service to create a PDF using files uploaded to Cloud Storage. --- ## Solution of Lab %[https://www.youtube.com/watch?v=HTeRNOolrcM] ```apache export REGION= ``` ![](https://cdn.hashnode.com/res/hashnode/image/upload/v1724074420454/3de33aa3-3b1d-43ae-9098-92a54550d698.png align="center") ```apache curl -LO raw.githubusercontent.com/quiccklabs/Labs_solutions/master/Creating%20PDFs%20with%20Go%20and%20Cloud%20Run/quicklabgsp762.sh sudo chmod +x quicklabgsp762.sh ./quicklabgsp762.sh ```