Job Summary
We are seeking a skilled Data Engineer with strong expertise in Google Cloud Platform (GCP) and BigQuery to design, develop, and maintain scalable data pipelines, data warehouses, and cloud-based analytics solutions. The ideal candidate should have hands-on experience in building enterprise data platforms, optimizing data processing workloads, and supporting business intelligence and advanced analytics initiatives.
Key Responsibilities
- Design, develop, and maintain robust data pipelines on GCP.
- Build and manage scalable data warehouse solutions using BigQuery.
- Develop ETL/ELT processes for ingesting, transforming, and loading data from multiple sources.
- Implement batch and real-time data processing solutions.
- Optimize BigQuery performance, partitioning, clustering, and cost management.
- Work closely with data analysts, data scientists, and business stakeholders to understand data requirements.
- Ensure data quality, integrity, security, and governance across data platforms.
- Develop reusable frameworks and automation for data ingestion and transformation.
- Monitor and troubleshoot production data pipelines and resolve performance issues.
- Participate in Agile development activities, code reviews, and release processes.
Required Skills
- Strong experience with Google Cloud Platform (GCP) services.
- Hands-on expertise in BigQuery.
- Experience with Cloud Composer (Airflow), Dataflow, Dataproc, Pub/Sub, and Cloud Storage.
- Strong SQL development and query optimization skills.
- Proficiency in Python and data engineering frameworks.
- Experience building ETL/ELT pipelines and data integration solutions.
- Knowledge of data modeling, dimensional modeling, and data warehousing concepts.
- Experience with Git, CI/CD pipelines, and DevOps practices.
- Understanding of data governance, metadata management, and security best practices.