Senior Data Engineer I

About the Team – Global Campaign Tracking (GCT)

Global Campaign Tracking (GCT) is a mission-critical enterprise data platform that powers digital acquisition marketing across American Express Consumer and Commercial businesses in global markets. The platform captures, processes, and delivers digital interaction data to enable campaign attribution, acquisition reporting, and marketing analytics, helping optimize marketing ROI and campaign performance.

GCT serves as the authoritative source for customer targeting, offer eligibility, and digital acquisition insights. It also supports partner commission and affiliate settlement processes through daily data feeds to external agencies. Additionally, the platform provides trusted data to downstream systems for analytics, machine learning, enterprise reporting, and real-time risk decisioning, making it a foundational component of the American Express digital acquisition ecosystem.

Role Summary

We are seeking a highly motivated Sr. Data Engineer I to design, build, and enhance scalable data platforms that power critical business capabilities across digital acquisition marketing. This role requires a strong engineering mindset, deep expertise in distributed data processing, and a passion for building reliable, high-performance data solutions.

As part of the GCT engineering team, you will work closely with product managers, architects, software engineers, analysts, and business stakeholders to develop modern data pipelines, optimize platform performance, and deliver high-quality data products that support real-time and batch analytics.

Key Responsibilities

• Design, develop, and maintain scalable and resilient data pipelines using modern cloud-native technologies.
• Build and optimize batch and streaming data processing frameworks.
• Develop high-quality, reusable, and maintainable code following engineering best practices.
• Design data models that support analytics, reporting, and operational use cases.
• Improve platform scalability, performance, reliability, and observability.
• Collaborate with cross-functional teams to translate business requirements into technical solutions.
• Perform data quality validation and implement automated monitoring and alerting.
• Troubleshoot production issues and drive root cause analysis.
• Participate in architecture discussions, code reviews, and technical design sessions.
• Mentor junior engineers and contribute to engineering best practices across the team.
• Drive continuous improvement through automation, CI/CD, and infrastructure-as-code practices.

Required Qualifications

• Bachelor's degree in Computer Science, Engineering, Information Systems, or a related technical field.
• 8+ years of experience in Data Engineering or Software Engineering.
• Strong programming experience with Java and/or Python.
• Hands-on experience with Apache Spark and distributed data processing.
• Experience developing ETL/ELT pipelines on cloud platforms.
• Strong SQL skills and experience with relational and distributed databases.
• Experience working with Apache Airflow or similar workflow orchestration tools.
• Experience with version control using Git and modern CI/CD practices.
• Strong understanding of software engineering principles, testing, and code quality.
• Excellent analytical, troubleshooting, and problem-solving skills.
• Strong communication and collaboration skills.

 

Preferred Qualifications

• Experience with Databricks and Delta Lake.
• Experience with AWS services such as S3, EMR, Lambda, ECS, EC2, IAM, and CloudWatch.
• Experience building event-driven or streaming applications using Kafka or similar technologies.
• Experience with containerization technologies such as Docker and Kubernetes.
• Knowledge of data governance, metadata management, and data lineage.
• Experience working in Agile/Scrum environments.
• Exposure to marketing analytics, digital acquisition, clickstream, or customer analytics platforms.
• Experience with observability and monitoring tools.
• Familiarity with DevSecOps and secure software development practices.

 

What Success Looks Like

• Deliver reliable, scalable, and high-quality data engineering solutions.
• Build production-ready pipelines that meet performance and availability goals.
• Improve platform efficiency through engineering innovation and automation.
• Maintain high standards for code quality, testing, and operational excellence.
• Collaborate effectively across engineering, product, and business teams.
• Take ownership of features from design through production support.
• Contribute to the team's technical direction and mentor fellow engineers.
• Continuously identify opportunities to simplify, modernize, and optimize the platform.

 

Key Technologies

• Java
• Python
• Apache Spark
• Databricks
• SQL
• Apache Airflow
• AWS (S3, EMR, Lambda, ECS, EC2, IAM, CloudWatch)
• Delta Lake
• Git & GitHub
• Jenkins / CI-CD
• Docker
• Kubernetes
• Kafka
• REST APIs
• Redis
• Unix/Linux
• Monitoring & Observability Tools
• Agile / Scrum