SeniorDataEngineer-AWS&RAGPipelines

We're looking for a Senior Data Engineer to design and operate the cloud data infrastructure powering our AI initiatives. You'll architect production-scale data lakes on AWS, build real-time ingestion and observability pipelines, and own the vector search and embedding layers that feed our RAG systems and autonomous agents. **Requirements** **Must-Have** * Overall Experience: 7+ years in Data Engineering, Distributed Systems, or Data Architecture * AWS & Infrastructure: 4+ years architecting production-scale data lakes, storage tiers, and event streaming * AI/LLM Pipelines: 2+ years building RAG systems, managing embeddings, and orchestrating foundational models * Proficiency in AWS Data Lake Architecture & Storage * Proficiency in Real-Time Observability & Log Analytics * Proficiency in Elasticsearch & OpenSearch Optimization, Vectorization, Embeddings * Proficiency in Amazon Bedrock & Generative AI Pipelines * Proficiency in Software Engineering & API Ingestion * Production-level proficiency in one or more of: C# (.NET Core), Java, Python, or Node.js **Preferred Experience** * AWS S3 partitioning strategies, lifecycle policies, and columnar formats (Parquet, Iceberg) * AWS Glue Data Catalog and Lake Formation for multi-tenant, fine-grained access control * Query optimization over petabyte-scale datasets using Amazon Athena and Redshift Spectrum * Distributed oTel collector configuration for log, trace, and metrics capture and routing into S3 * High-volume streaming of system logs, Datadog captures, and raw server events into S3 * Real-time CDC from PostgreSQL using Debezium or AWS DMS * Amazon OpenSearch clusters with simultaneous lexical and high-dimensional vector search * OpenSearch index lifecycle management, sharding strategies, and dynamic mappings at scale * Amazon Bedrock foundational model APIs (Claude, Titan) for data enrichment, classification, and semantic parsing * Knowledge Bases for Amazon Bedrock for automatic chunking, metadata extraction, and vec