WebScrapingEngineer

Web Scraping Engineer Chennai/Pune About the Job At NIQ Digital shelf, scraping isn’t just about collecting data; it’s about outsmarting the obstacles that stand in the way of visibility. Every day, we process tens of billions of data points from thousands of sources, both web and mobile, and transform them into actionable insights for the world’s top retailers and brands. You'll join a team of 60+ scrapers , engineers, analysts, and data lovers from across the globe. As part of our scraping and R&D team, you’ll have space to experiment, push boundaries , and bring bold ideas to life. You’ll collaborate closely with Customer Success , Operations , and Data to ensure we extract high-quality, actionable data — fast and at scale. Responsibilities What you’ll do (role spans junior → senior; we’ll match scope to your experience): * Build and maintain web crawlers to extract structured data (products, prices, reviews). * Parse, clean, and validate messy content into reliable datasets. * Tackle dynamic, JavaScript-heavy pages and evolving site structures using modern tools. * Monitor runs, logs, and alerts to keep data fresh and accurate. Grow into (or bring experience with): * Designing distributed, scalable scraping architectures for thousands of domains. * Reverse engineering APIs/mobile traffic (mitmproxy, Charles, Burp, Frida). * Detecting/mitigating anti-bot measures (fingerprinting, behavior checks, signatures). * Implementing anomaly detection, reliability, and observability at scale. * Leading prototypes, standardizing frameworks, and mentoring teammates. ## Qualifications Must Have: * 1 to 10 years of work experience in Web scraping using Python or Dotnet or Java or JavaScript or Golang. * Comfortable using tools like Scrapy , Python Requests, Beautiful soup, Playwright/Selenium * You understand how to work with HTTP headers, cookies, session management, and are not afraid of network debugging. * You adapt quickly and aren’t scared of messy problems. When somethi