Google GCP-DE Exam : Data Engineer

Which of these rules apply when you add preemptible workers to a Dataproc cluster (select 2 answers)?

• A. Preemptible workers cannot use persistent disk.
• B. Preemptible workers cannot store data.
• C. A Dataproc cluster cannot have only preemptible workers.
• D. If a preemptible worker is reclaimed, then a replacement worker must be added manually.

You receive data files in CSV format monthly from a third party. You need to cleanse this data, but every third month the schema of the files changes. Your requirements for implementing these transformations include:
Executing the transformations on a schedule
Enabling non-developer analysts to modify transformations
Providing a graphical tool for designing transformations
What should you do?

• A. Use Cloud Dataprep to build and maintain the transformation recipes, and execute them on a scheduled basis
• B. Load each month's CSV data into BigQuery, and write a SQL query to transform the data to a standard scheme
• C. Merge the transformed tables together with a SQL query
• D. Help the analysts write a Cloud Dataflow pipeline in Python to perform the transformatio
• E. Use Apache Spark on Cloud Dataproc to infer the schema of the CSV file before creating a Dataframe.Then implement the transformations in Spark SQL before writing the data out to Cloud Storage and loading into BigQuery
• F. The Python code should be stored in a revision control system and modified as the incoming data's schema changes

Which of these sources can you not load data into BigQuery from?

• A. Google Cloud Storage
• B. Google Drive
• C. Google Cloud SQL
• D. File upload

You have developed three data processing jobs. One executes a Cloud Dataflow pipeline that transforms data uploaded to Cloud Storage and writes results to BigQuery. The second ingests data from on-premises servers and uploads it to Cloud Storage. The third is a Cloud Dataflow pipeline that gets information from third-party data providers and uploads the information to Cloud Storage. You need to be able to schedule and monitor the execution of these three workflows and manually execute them when needed. What should you do?

• A. Set up cron jobs in a Compute Engine instance to schedule and monitor the pipelines using GCP API calls.
• B. Use Stackdriver Monitoring and set up an alert with a Webhook notification to trigger the jobs.
• C. Develop an App Engine application to schedule and request the status of the jobs using GCP API calls.
• D. Create a Direct Acyclic Graph in Cloud Composer to schedule and monitor the jobs.

Your company produces 20,000 files every hour. Each data file is formatted as a comma separated values (CSV) file that is less than 4 KB. All files must be ingested on Google Cloud Platform before they can be processed. Your company site has a 200 ms latency to Google Cloud, and your Internet connection bandwidth is limited as 50 Mbps. You currently deploy a secure FTP (SFTP) server on a virtual machine in Google Compute Engine as the data ingestion point. A local SFTP client runs on a dedicated machine to transmit the CSV files as is. The goal is to make reports with data from the previous day available to the executives by 10:00 a.m. each day. This design is barely able to keep up with the current volume, even though the bandwidth utilization is rather low.
You are told that due to seasonality, your company expects the number of files to double for the next three months. Which two actions should you take? (choose two.)

• A. Redesign the data ingestion process to use gsutil tool to send the CSV files to a storage bucket in parallel.
• B. Contact your internet service provider (ISP) to increase your maximum bandwidth to at least 100 Mbps.
• C. Transmit the TAR files instead, and disassemble the CSV files in the cloud upon receiving them.
• D. Assemble 1,000 files into a tape archive (TAR) fil
• E. Create an S3-compatible storage endpoint in your network, and use Google Cloud Storage Transfer Service to transfer on-premices data to the designated storage bucket.
• F. Introduce data compression for each file to increase the rate file of file transfer.