[GFA] Azure Senior Data Engineer

Google Chrome Microsoft Edge Apple Safari Mozilla Firefox [GFA] Azure Senior Data Engineer Full-timeCompany DescriptionSoftware Mind develops solutions that make an impact for companies around the globe. Tech giants & unicorns, transformative projects, emerging technologies and limitless opportunities – these are a few words that describe an average day for us. Building cross-functional engineering teams that take ownership and crave more means we’re always on the lookout for talented people who bring passion and creativity to every project. Our culture embraces openness, acts with respect, shows grit & guts and combines employment with enjoyment.Job DescriptionProject – the aim you’ll haveOur customer provides innovative solutions and insights that enable our clients to manage risk and hire the best talent. Their advanced global technology platform supports fully scalable, configurable screening programs that meet the unique needs of over 33,000 clients worldwide. Headquartered in Atlanta, GA, they have an internationally distributed workforce spanning 19 countries with about 5,500 employees. Our partner perform over 93 million screens annually in over 200 countries and territories.We are seeking a Senior Data Engineer with proven expertise in Databricks PySpark development, comprehensive data modeling experience (including fact/dimension tables, SCD Type 2 and incremental loads) and event-based architecture skills to join our Data Engineering Team and drive the evolution of our Azure-based Data Analytics Platform.Position – how you’ll contributeDevelop reusable, metadata-driven data pipelines using Databricks Lakehouse architecture and PySparkDesign and implement comprehensive data modelsBuild robust ETL/ELT solutions with advanced features: Merge operations, SCD Type 2 implementations, etc.Implement incremental data loads with idempotency patterns and overlap joins optimizationAutomate and optimize data platform processes with focus on performance and reliabilityBuild integrations with data sources and consumers using event-driven patternsCooperate with infrastructure engineering team to set up cloud resourcesInitiate and implement improvements to data platform architectureQualificationsExpectations – the experience you needDatabricks expertise: proficient in Databricks Lakehouse architecture and PySpark developmentData modeling mastery: extensive experience in data model definition including fact tables, dimension tables, measures, grain analysis, SCD Type 2, surrogate keys, late-arriving records handling, overlap joins, and incremental loads with idempotencyProgramming: advanced Python and PySpark skills for ETL/ELT developmentDatabricks optimization: deep knowledge of optimize, zOrder, Liquid clustering, ACID transactions and performance tuningEvent-based architecture: proven experience in designing and implementing event-driven data solutionsAzure data platform: experience working with Azure-based datasets and data pipelinesSQL proficiency: strong SQL skills for complex data transformationsLarge-scale data processing: experienced in handling large and complex datasets efficientlyCI/CD: experience in developing automated deployment pipelinesNetworking fundamentals: understanding of basic networking conceptsAgile methodology: familiar with Scrum and agile development practicesAdditional skills – the edge you haveUnderstanding of stream processing challenges and Spark Structured StreamingExperience with Infrastructure as Code (Terraform, Bicep)Experience with containerized applications (Azure Container Apps, Kubernetes)Knowledge of Azure cloud native solutions (Azure Data Factory, Azure Function App, Azure Container Instances)Additional InformationOur offer – professional development, personal growth:Flexible employment and remote work  International projects with leading global clients International business trips  Non-corporate atmosphere Language classes Internal & external training Private...