Role Overview
Reporting to the Data & Analytics Lead, the Principal Data Engineer is a hands-on role, combining the building and optimization of production data pipelines with the technical leadership needed to raise engineering standards and develop the people around them.
The role functions as a key interface between :
- Business stakeholders and department heads
- The wider data and analytics team
- Owners of source systems and data in Finance and IT.
Key Responsibilities
- Designing and building robust, production-grade data pipelines on Databricks, making appropriate use of current platform capabilities such as Spark Declarative Pipelines (SDP)
- Working “under the hood” to optimise the platform for performance and cost — tuning compute, jobs and queries, managing storage and table layout, and keeping platform spend under control
- Defining and maintaining consistent business metrics and a semantic layer (for example, using Metric Views) so reporting is built on trusted, reusable definitions
- Integrating data from core business systems, including SAP S/4HANA, and other sources, using tools such as Fivetran (both SaaS connectors and HVR)
- Setting and upholding engineering standards across the team — code quality, testing, documentation, CI/CD and data governance
- Coaching and mentoring junior and mid-level engineers, reviewing their work and helping them develop
- Partnering with business stakeholders to understand their needs, shape practical solutions and ensure the platform delivers genuine value
- Engaging data owners in Finance and IT to ensure data is well understood, valid and fit for purpose, and initiating data quality improvements where required
Requirements
Educational Background, Skills & Experience
- At least 5 or more of hands-on daily Databricks experience, covering both pipeline development and under-the-hood performance and cost optimization
- Up to date with recent platform developments, such as Spark Declarative Pipelines (SDP) and Metric Views
- Strong proficiency in programming languages commonly used in data engineering, such as Python, SQL and Spark
- Advanced experience with data manipulation, data modelling, database design and query optimization
- Experience managing or coaching junior developers
- A track record of managing and influencing business stakeholders
- Experience working with SAP S/4HANA data sets
- Experience with Fivetran, including both SaaS connectors and HVR
- Power BI semantic modelling experience
- Experience in the energy, fuels or logistics sector
- Combining deep, hands-on engineering skill with sound judgement about cost, performance and long-term maintainability
- Coaching, mentoring and raising the capability of less experienced engineers
- Collaborating, communicating confidently and influencing business stakeholders
- Breaking down complex technical concepts and explaining them simply to non-technical audiences
- Staying current with a fast-moving platform and bringing new capabilities into everyday practice
- Taking ownership and driving work independently, from concept through to production
To apply
Kindly upload a cover letter and a recent resume by close on business 16th July 2026.