Deeplight AI
Financial Services
AWSGlueDataEngineer
Neural analysis suggests this role is
optimal for Mid+ candidates.
“AWS Glue Data Engineer at Deeplight AI. Skills: AWS Glue, PySpark, ETL pipeline, Lakehouse architecture. Build AWS Glue jobs. Implement loading methods”
Industry & Context.
Identifying automation opportunities
What They're Looking For.
Must Have
5+ years experience in data engineering, Hands-on experience in AWS Glue
Nice to Have
Experience working in financial services, Experience in AWS Glue, Experience in PySpark, Experience in ETL pipeline development
What You'll Do.
Implement loading methods
Design historical loading mechanisms
Optimize Glue job performance
Collaborate with platform teams
Migrate source tables
Automate production deployment
Implement source system connectivity
Ensure jobs comply with data contracts
Prepare documentation
Handover to support teams
Work with Data Architect
Coordinate with Data Assurance Lead
Partner with platform engineers
How You'll Work.
Team & Collaboration
Data Architect; Data Assurance Lead; Platform engineers; Source system owners; Data Factory Squad
Communication Scope
Compelling communicators; Persuasive advocates; Articulating technical decisions; Selling vision
Process & Methodology
Agile
Full Job Description
DeepLight AI is a specialist AI and data consultancy with extensive experience implementing intelligent enterprise systems across multiple industries, with particular depth in financial services and banking. Our team combines deep expertise in data science, statistical modeling, AI/ML technologies, workflow automation, and systems integration with a practical understanding of complex business operations. We are seeking a skilled AWS Glue Data Engineer to join our Data Factory Squad, responsible for migrating source systems into the Lakehouse ingestion zone. This role focuses on building scalable ingestion pipelines, optimizing performance, and ensuring compliance with architectural and data assurance standards. You will ideally have experience working in financial services with strong experience in AWS Glue, PySpark, and ETL pipeline development. _**Your responsibilities as the AWS Glue Data Engineer will include:**_ * **Data Ingestion Development** * Building and implementing AWS Glue jobs for Bronze layer ingestion using defined standards and templates. * Implementing correct loading methods based on source requirements (CDC, full load, delta, snapshot). * Designing and executing historical loading mechanisms to bring legacy data into the Lakehouse. * **Performance Optimisation** * Optimising Glue job performance (DPU allocation, parallelization, partitioning) according to best practices. * Collaborating with platform teams to ensure tooling and optimization alignment. * **Migration & Automation** * Aggressively migrating source tables to Bronze layer, initially using manual approaches with standards/templates, later leveraging AI-enabled acceleration. * Ensuring jobs are version-controlled and production deployment is automated via Git and Terraform. * **Governance & Monitoring** * Implementing source system connectivity into CDP in collaboration with source system owners. * Ensuring jobs comply with data contracts and are properly monitored. * Preparing document
Applying for this AWS Glue Data Engineer role?
Most applicants get filtered before a human reads their resume. See if yours makes the cut.
ANONYMOUS · UNFILTERED
What do employees actually say about Deeplight AI?
Real rants from real employees. Read before you apply.