Abacus Insights
Healthcare
SeniorDataEngineer
Neural analysis suggests this role is
optimal for Senior candidates.
“Senior Data Engineer at Abacus Insights. Skills: Data Engineering, PySpark, SparkSQL, Databricks, Snowflake, AWS, Python, SQL, Terraform, CI/CD. Architect, design, and implement high-volume batch and real-time data pipelines. Build end‑to‑end ingestion frameworks integrating with Databricks, Snowflake, AWS services, and vendor data APIs”
What You'll Achieve.
Ensure high-quality, compliant data operations across the lifecycle; Enable scalable, resilient, and high‑performance data ingestion and transformation pipelines; Improve outcomes, reduce waste, and deliver better experiences for members and providers alike
Industry & Context.
Translate complex business requirements into detailed technical specifications
Work arrangements: Standard hours: 8 hours/day, 5 days/week, Location: Pune, Hybrid (3 days a week in office), Shift: 12pm-9pm IST
What They're Looking For.
Must Have
5 to 7 years of hands‑on experience as a Data Engineer working with large‑scale, distributed data processing systems in modern cloud environments, Working knowledge of U. S. healthcare data domains—including claims, eligibility, and provider datasets—and experience applying this knowledge to complex ingestion and transformation workflows, ability to communicate complex technical concepts clearly across both technical and non‑technical stakeholders, Expert‑level proficiency in Python, SQL, and PySpark, including developing distributed data transformations and performance‑optimized queries, Demonstrated experience designing, building, and operating production‑grade ETL/ELT pipelines using Databricks, Airflow, or similar orchestration and workflow automation tools, Proven experience architecting or operating large‑scale data platforms using dbt, Kafka, Delta Lake, and event‑driven/streaming architectures, within a cloud‑native data services or platform engineering environment—requiring specialized knowledge of distributed systems, scalable data pipelines, and cloud‑scale data processing, Experience working with structured and semi‑structured data formats such as Parquet, ORC, JSON, and Avro, including schema evolution and optimization techniques, working knowledge of AWS data ecosystem components—including S3, SQS, Lambda, Glue, IAM—or equivalent cloud technologies supporting high‑volume data engineering workloads, Proficiency with Terraform, infrastructure‑as‑code methodologies, and modern CI/CD pipelines (e. g. , GitLab) supporting automated deployment and versioning of data systems, Deep expertise in SQL and compute optimization strategies, including Z‑Ordering, clustering, partitioning, pruning, and caching for large‑scale analytical and operational workloads
Nice to Have
Experience in large-scale healthcare or payer data environments, Hands-on experience with major cloud data warehouse platforms such as Snowflake (preferred), BigQuery, or Redshift, including performance tuning and data modeling for analytical environments
What You'll Do.
and implement high-volume batch and real-time data pipelines
Build end‑to‑end ingestion frameworks integrating with Databricks
Develop data modeling frameworks
Lead technical solution design for health plan clients
Translate complex business requirements into detailed technical specifications
engineering artifacts
and reusable components
Implement security automation
Establish and enforce data engineering best practices
Conduct performance profiling and optimize compute costs
Produce high-quality technical documentation
How You'll Work.
Team & Collaboration
Work directly with customers, data vendors, and internal engineering teams; Guide customers in adopting Abacus’s core data management platform; Mentor junior engineers through technical reviews, coaching, and training sessions for both internal teams and clients; Communicate complex technical concepts clearly across both technical and non‑technical stakeholders
Communication Scope
Communicate complex technical concepts clearly across both technical and non‑technical stakeholders
Full Job Description
About Us Abacus Insights is transforming how data works for health plans. Our mission is simple: make healthcare data usable, so the people responsible for care and cost decisions can act faster, with confidence. We help health plans break down data silos to create a single, trusted data foundation. That foundation powers better decisions—so plans can improve outcomes, reduce waste, and deliver better experiences for members and providers alike. Backed by $100M from top investors, we’re tackling big challenges in an industry that’s ready for change. Our platform enables GenAI use cases by delivering clean, connected, and reliable healthcare data to support automation, prioritization, and decision workflows—and it’s why we are leading the way. Our innovation begins with people. We are bold, curious, and collaborative—because the best ideas come from working together. We embrace the thoughtful use of AI and automation to drive innovation and efficiency, and we look for individuals who are curious and adaptable—those excited to leverage emerging technologies to enhance how we work—while keeping human insight, connection, and our clients at the center of every decision. Ready to make an impact? Join us and let’s build the future together. About the role We are seeking an accomplished Senior Data Engineer to join our dynamic and rapidly expanding Tech Ops division. With significant projected growth, this is an opportunity to drive meaningful technical impact. In this role, you will work directly with customers, data vendors, and internal engineering teams to design, implement, and optimize complex data integration solutions within a modern, large‑scale cloud environment. You will leverage advanced skills in distributed computing, data architecture, and cloud-native engineering to enable scalable, resilient, and high‑performance data ingestion and transformation pipelines. As a trusted technical advisor, you will guide customers in adopting Abacus’s core data management p
Applying for this Senior Data Engineer role?
Most applicants get filtered before a human reads their resume. See if yours makes the cut.
How to Apply on Greenhouse
- Create a Greenhouse profile before applying — it saves time across multiple applications.
- Upload your resume as a PDF; the parser handles it better than Word.
- Answer all knockout questions carefully — wrong answers auto-reject before a human sees you.
- Enable email notifications to track application status in real time.
ANONYMOUS · UNFILTERED
What do employees actually say about Abacus Insights?
Real rants from real employees. Read before you apply.