DataEngineer

**Position Summary:** An experienced data engineer with strong Azure Databricks expertise is required for a 6 month contract to develop and deliver a scalable data pipeline that ingests files and transforms them into standardised, high-quality datasets for reporting. This is a hands-on delivery role focused on building a production-grade pipeline, suited to someone comfortable working independently, defining technical solutions, and rapidly delivering high-quality data engineering components. # **Location:** London - UK - Fenchurch **What will your job entail?** Key Responsibilities • Implement a scalable ETL pipeline in Azure Databricks, that transforms raw data into standardised schemas to support reporting and analytics, following a medallion architecture in line with internal design standards. • Leverage an existing email ingestion tool to extract multiple format source files from a shared mailbox. • Develop notebooks, jobs and pipelines in Databricks for ETL orchestration. • Design and implement robust validation rules, monitor pipeline runs and troubleshoot failures, with logging and notifications to support robust data processes. • Integrate with CI/CD flows across environments using Azure DevOps. • Produce clear technical documentation of the solution to ensure maintainability post-delivery. Experience • 2+ years hands-on experience with Azure Databricks, including data validation, transformation, and optimization. • Expertise in PySpark/Spark SQL. • Knowledge of Unity Catalog and modern Azure Databricks features, as well as data best practices and concepts. • Proficiency building end-to-end data pipelines with Delta Lake/Lakehouse architecture. • Experience implementing CI/CD pipelines (Azure DevOps preferred). • Experience supporting BI reporting layers (Power BI or similar) is nice to have. • Insurance experience is nice to have. **Disclaimer** Ryan Specialty is an Equal Opportunity Employer. We are committed to building and sustaining a diverse workforce