Use Cloud Composer to automate the pipeline using Apache Airflow workflows, and secure the data by using Google Cloud Key Management Service (KMS). -> Correct. Cloud Composer, based on Apache Airflow, is designed to orchestrate complex data workflows, making it a good fit for automating the pipeline. It can run tasks multiple times per day and is scalable enough to handle large data volumes. By integrating with Google Cloud Key Management Service (KMS), it ensures that the financial data remains secure and private, which is a key requirement in this use case.
Use Cloud Functions to extract financial data from various sources, transform it, and load it into the data warehouse. -> Incorrect. Cloud Functions are designed for lightweight, event-driven applications and aren't the best fit for complex, large-scale data processing pipelines. Moreover, handling the orchestration of multiple steps (extraction, transformation, and loading) might become complex using only Cloud Functions.
Use Cloud Dataproc to extract financial data from various sources, transform it, and load it into the data warehouse. -> Incorrect. While Cloud Dataproc can handle large-scale data processing, it doesn't provide built-in orchestration features for running complex pipelines multiple times per day. Also, it alone doesn't offer the security features required for financial data.
Use Cloud Functions and Cloud Dataproc to extract financial data from various sources, transform it, and load it into the data warehouse. -> Incorrect. While combining Cloud Functions for lightweight tasks and Cloud Dataproc for heavy data processing might seem like a good idea, this option does not mention the security aspect, which is crucial for a financial services company.
https://cloud.google.com/composer/docs