LTM
GCP Data Engineer
Primary Skills: GCP, BigQuery, Python, Qlik Location: PAN India Experience: 7 to 10 years Working Mode: Hybrid Detailed JD We are seeking an experienced Google Cloud Data Engineer to design, develop, and maintain scalable cloud-based data platforms and ETL/ELT pipelines using Google Cloud Platform technologies. Key Responsibilities Design and implement scalable data pipelines using Python, BigQuery, Cloud Storage, Dataflow, Cloud Functions, Cloud Composer, Airflow, and Dataform. Develop batch and streaming data ingestion frameworks. Build dimensional data models and optimize BigQuery performance and cost. Implement CI/CD, source control using Git/Bitbucket, testing, monitoring, and data quality processes. Collaborate with architects, analysts, and business stakeholders. Troubleshoot production issues and perform performance tuning. Implement security, IAM, governance, and metadata management. Understand existing Airflow/Composer DAGs and develop new DAGs as required. Communicate effectively with customers and stakeholders and work flexible hours when required. Required Qualifications Bachelor's degree in Computer Science, Information Systems, Engineering, or a related field. 5+ years of data engineering experience, with 3+ years of experience on Google Cloud Platform. Strong Python and SQL skills. Hands-on experience with BigQuery, Cloud Storage, Dataflow, Cloud Composer, Dataform, Cloud Functions, Cloud Pub/Sub, and Cloud Scheduler. Experience with Git, Jira, and Agile/Scrum methodologies. Knowledge of data warehousing, dimensional modeling, CI/CD, and cloud security. Strong troubleshooting and problem-solving skills. Excellent customer communication and collaboration skills.
The role
What you'll do
- Design, develop, and maintain scalable GCP-based data platforms and ETL/ELT pipelines.
- Develop data pipelines using Python, BigQuery, Cloud Storage, Dataflow, Cloud Functions, Cloud Composer/Airflow, and Dataform.
- Build and maintain batch and streaming data ingestion frameworks.
- Develop dimensional data models and optimize BigQuery queries, performance, and cost.
- Design, develop, and maintain Airflow/Cloud Composer DAGs and understand existing workflows.
- Work with Git/Bitbucket and follow Agile/Scrum development practices
- Implement and maintain cloud security, IAM, governance, and metadata management
You bring
Requirements
- 7–10 years of overall experience in Data Engineering
- Minimum 3+ years of hands-on experience with Google Cloud Platform (GCP)
- Strong hands-on experience with Python and SQL
- Mandatory hands-on experience with Google BigQuery
- Experience with GCP Dataflow and Google Cloud Storage (GCS)
- Hands-on experience with Cloud Functions, Pub/Sub, Cloud Scheduler, and Dataform
- Strong experience with Cloud Composer / Apache Airflow, including development and maintenance of DAGs
Skills