Weekday AI
LeadHadoopDataEngineer
Neural analysis suggests this role is
optimal for Lead candidates.
“Lead Hadoop Data Engineer at Weekday AI. Skills: Hadoop, Big Data, Python, SQL, Data Engineering, ETL, On-Premise Data Environment. Design, develop, and maintain scalable Big Data solutions using Hadoop ecosystem technologies. Build and optimize batch and real-time data pipelines for large-scale data ingestion and processing”
What You'll Achieve.
deliver reliable and scalable data solutions aligned with organizational goals
Industry & Context.
analytical, troubleshooting, and performance optimization skills
on-premise data environments
What They're Looking For.
Must Have
6 to 13 years of experience in Data Engineering or Big Data development, hands-on expertise in Hadoop and Big Data ecosystem technologies, Proficiency in Python scripting and advanced SQL programming, Experience working with Hadoop components such as HDFS, Hive, Spark, Sqoop, Kafka, MapReduce, Pig, or HBase, Solid understanding of distributed computing and large-scale data processing concepts, experience in building ETL/data pipeline solutions in enterprise environments, Hands-on experience managing or working within on-premise infrastructure environments, Good understanding of data warehousing concepts and relational databases, Experience with Linux/Unix environments and shell scripting, analytical, troubleshooting, and performance optimization skills, Big Data, Hadoop, Python, SQL, On-Premise Data Environment Experience
Nice to Have
Familiarity with workflow orchestration tools such as Airflow or Oozie is preferred, Experience leading technical teams or managing enterprise-scale Big Data projects, Exposure to cloud migration projects or hybrid data environments is an added advantage, Knowledge of DevOps practices, CI/CD pipelines, and containerization tools is desirable
What You'll Do.
and maintain scalable Big Data solutions using Hadoop ecosystem technologies
Build and optimize batch and real-time data pipelines for large-scale data ingestion and processing
Develop robust ETL workflows using Python
and Hadoop-based frameworks
Work extensively on distributed data processing platforms such as HDFS
and related technologies
Manage and support on-premise Big Data infrastructure and ensure high availability
Analyze complex data requirements and translate them into efficient technical solutions
Optimize query performance and data processing jobs for large datasets
Collaborate with architecture and infrastructure teams to improve system reliability and performance
technical discussions
and best practice implementation across the data engineering team
Troubleshoot production issues
and implement performance tuning strategies
and compliance standards are maintained across platforms
Mentor junior engineers and provide technical guidance on Big Data technologies and development standards
Participate in capacity planning
and Hadoop administration activities when required
How You'll Work.
Team & Collaboration
Collaborate closely with cross-functional teams including Data Science, Analytics, DevOps, and Business stakeholders; Collaborate with architecture and infrastructure teams to improve system reliability and performance; Lead code reviews, technical discussions, and best practice implementation across the data engineering team; Mentor junior engineers and provide technical guidance
Communication Scope
Excellent communication and stakeholder management abilities
Process & Methodology
managing enterprise-scale Big Data projects
Full Job Description
**This role is for one of the Weekday's clients** Min Experience: 6 years Location: Hyderabad JobType: full-time We are seeking a highly skilled and experienced Lead Hadoop Data Engineer to join our growing data engineering team. The ideal candidate will have strong expertise in Big Data technologies, Hadoop ecosystem components, Python programming, SQL development, and hands-on experience working in on-premise data environments. This role requires a strong technical leader who can design scalable data solutions, optimize large-scale data pipelines, and support enterprise-level analytics and data processing initiatives. As a Lead Hadoop Data Engineer, you will play a critical role in building and maintaining robust data infrastructure capable of handling high-volume, high-velocity, and high-variety datasets. You will collaborate closely with cross-functional teams including Data Science, Analytics, DevOps, and Business stakeholders to deliver reliable and scalable data solutions aligned with organizational goals. **Requirements** ### Key Responsibilities * Design, develop, and maintain scalable Big Data solutions using Hadoop ecosystem technologies. * Build and optimize batch and real-time data pipelines for large-scale data ingestion and processing. * Develop robust ETL workflows using Python, SQL, and Hadoop-based frameworks. * Work extensively on distributed data processing platforms such as HDFS, Hive, Spark, MapReduce, YARN, and related technologies. * Manage and support on-premise Big Data infrastructure and ensure high availability, performance, and scalability. * Analyze complex data requirements and translate them into efficient technical solutions. * Optimize query performance and data processing jobs for large datasets. * Collaborate with architecture and infrastructure teams to improve system reliability and performance. * Lead code reviews, technical discussions, and best practice implementation across the data engineering team. * Troubleshoot productio
Applying for this Lead Hadoop Data Engineer role?
Most applicants get filtered before a human reads their resume. See if yours makes the cut.
ANONYMOUS · UNFILTERED
What do employees actually say about Weekday AI?
Real rants from real employees. Read before you apply.