Job Summary
We are seeking a skilled Data Engineer with strong expertise in Google Cloud Platform (GCP) and BigQuery to design, develop, and maintain scalable data pipelines, data warehouses, and cloud-based analytics solutions. The ideal candidate should have hands-on experience in building enterprise data platforms, optimizing data processing workloads, and supporting business intelligence and advanced analytics initiatives.
Key Responsibilities
- Design, develop, and maintain robust data pipelines on GCP.
- Build and manage scalable data warehouse solutions using BigQuery.
- Develop ETL/ELT processes for ingesting, transforming, and loading data from multiple sources.
- Implement batch and real-time data processing solutions.
- Optimize BigQuery performance, partitioning, clustering, and cost management.
- Work closely with data analysts, data scientists, and business stakeholders to understand data requirements.
- Ensure data quality, integrity, security, and governance across data platforms.
- Develop reusable frameworks and automation for data ingestion and transformation.
- Monitor and troubleshoot production data pipelines and resolve performance issues.
- Participate in Agile development activities, code reviews, and release processes.
Required Skills
- Strong experience with Google Cloud Platform (GCP) services.
- Hands-on expertise in BigQuery.
- Experience with Cloud Composer (Airflow), Dataflow, Dataproc, Pub/Sub, and Cloud Storage.
- Strong SQL development and query optimization skills.
- Proficiency in Python and data engineering frameworks.
- Experience building ETL/ELT pipelines and data integration solutions.
- Knowledge of data modeling, dimensional modeling, and data warehousing concepts.
- Experience with Git, CI/CD pipelines, and DevOps practices.
- Understanding of data governance, metadata management, and security best practices.
Preferred Skills
- Experience with Apache Spark, Kafka, and Apache Beam.
- Knowledge of Looker, Tableau, or Power BI.
- Experience with data lake and lakehouse architectures.
- Understanding of machine learning data pipelines.
- Retail, eCommerce, Supply Chain, or Customer Analytics domain experience.
Qualifications
- Bachelor's or Master's degree in Computer Science, Information Technology, Data Engineering, or a related field.
- Preferred GCP Certifications:
- Google Professional Data Engineer
- Associate Cloud Engineer
Key Competencies
- Strong analytical and problem-solving skills.
- Excellent communication and stakeholder management abilities.
- Ability to work in a fast-paced Agile environment.
- Strong troubleshooting and performance tuning expertise.
- Attention to detail and commitment to data quality.
Nice to Have
- Experience with Vertex AI and AI/ML data pipelines.
- Exposure to Generative AI data engineering workloads.
- Experience handling large-scale enterprise data platforms and cloud migrations.