Senior Data Engineer

We are seeking a hands-on Senior Data Engineer to build, operate, and improve an enterprise data warehouse and production data pipelines in an Azure Databricks environment.

This role works across the Bronze, Silver, and Gold layers of a medallion architecture, with a strong focus on data ingestion, transformation, modeling, quality, performance, and reliability. The Senior Data Engineer will own assigned data domains as production systems, simplify legacy transformation layers, support the transition toward event-driven ingestion, and mentor other engineers as the data platform evolves.

* Design, build, and support enterprise data warehouse pipelines and data models across Bronze, Silver, and Gold layers in Databricks. * Develop ingestion patterns and Bronze-to-Silver transformations that produce clean, standardized, and reliable data. * Create dimensional and harmonized data models, including conformance, survivorship, and Gold-layer promotion rules. * Own assigned data domains in production, including data-model integrity, pipeline reliability, ingestion SLAs, incident response, and root-cause remediation. * Build and maintain Delta Lake pipelines and help transition selected workloads from batch processing to event-driven, low-latency ingestion. * Develop streaming and high-SLA ingestion patterns using technologies such as Zerobus Ingestion and Spark Declarative Pipelines. * Implement row-level reconciliation, validation checkpoints, automated quality gates, and data-contract validation throughout the data lifecycle. * Build Gold-layer tables that support Unity Catalog metric views, lineage tracking, governed access, and downstream analytics. * Optimize pipeline performance, processing latency, platform reliability, and Databricks compute costs. * Consolidate legacy transformation layers into a simplified target architecture while validating output accuracy. * Monitor pipeline health, investigate anomalies, and resolve data-reliability incidents. * Document architecture decisions, data models, transformation logic, feed SQL, and operational procedures. * Review pull requests and maintain code quality through Azure DevOps, Git, and CI/CD practices. * Mentor Data Engineers in Databricks development patterns, SQL, Python, PySpark, testing, and documentation. * Collaborate directly with data governance, quality assurance, analytics engineering, and business-facing stakeholders to clarify requirements and resolve delivery issues.