TechnicalDirector,DataEngineering

Diligent is looking for a Technical Director of Data Engineering to help design and build the next generation of data and AI infrastructure powering our products and internal platforms. This is not a traditional management role. We are looking for an experienced builder; someone who still enjoys writing code, debugging distributed systems, evaluating frameworks, and getting hands-on with architecture and implementation. You will work across large-scale ingestion pipelines, search systems, AI/LLM infrastructure, event-driven architectures, vector databases, analytics platforms, and real-time data processing. You should be equally comfortable discussing high-level architecture with senior leadership and diving into a failing Kubernetes pod or optimizing a Spark job. The ideal candidate has strong opinions informed by real-world experience, understands tradeoffs deeply, and can move quickly without creating unnecessary complexity. What You’ll Do Design and build scalable data platforms and distributed processing systems Develop modern ingestion, transformation, and retrieval pipelines for structured and unstructured data Build systems supporting AI/LLM applications, semantic search, RAG pipelines, vector search, and agentic workflows Work hands-on with engineering teams to implement production-grade solutions rather than producing slideware Evaluate and standardize frameworks, tooling, and infrastructure patterns across teams Improve performance, reliability, observability, and cost efficiency of data systems Partner with product and platform engineering teams to accelerate delivery of AI-native capabilities Drive pragmatic engineering decisions balancing speed, maintainability, and operational simplicity Mentor engineers technically through design reviews, architecture guidance, and pair debugging Help establish engineering standards around CI/CD, testing, data quality, monitoring, and operational excellence What We’re Looking For Strong Hands-On Engineering Experienc