Company
Engineering
Anti-BotEngineer
Neural analysis suggests this role is
optimal for Senior candidates.
“Anti-Bot Engineer. Skills: Web scraping, Anti-bot systems, Python, Distributed systems. Design scraping systems. Build scraping systems”
Industry & Context.
Debugging; Troubleshooting
What They're Looking For.
Must Have
7+ years software engineering experience, Backend systems experience, Data engineering experience, Distributed systems experience, Large-scale production web scraping systems, Scraping Google or complex environments, Python expertise, Comparable production languages expertise, HTTP internals understanding, Browser automation frameworks experience, HTML parsing knowledge, DOM traversal knowledge, High-performance data extraction techniques, Anti-bot systems experience, Asynchronous systems design, Concurrent systems design, High-throughput workloads experience, Debugging skills, Distributed systems debugging, Failure-prone production systems debugging, Cloud-based systems at scale experience, Operational ownership experience, Adversarial environments experience
Nice to Have
Docker experience, Kubernetes experience, Distributed task queues exposure, Large-scale job orchestration systems exposure, Monitoring systems experience, Anomaly detection systems experience, Data quality validation systems experience, Search intelligence background, Advertising tech background, Competitive intelligence platforms background, High-defence web environments experience, Observability tooling familiarity, Production-grade system monitoring familiarity, AI-assisted development workflows exposure, Agentic coding tools exposure
What You'll Do.
Design scraping systems
Build scraping systems
Maintain scraping systems
Architect extraction systems
Adapt systems to changes
Build data extraction pipelines
Implement proxy rotation
Implement fingerprinting
Implement session management
Implement request routing
Monitor system health
Debug production failures
Optimise scraping infrastructure
Collaborate with data engineering
Collaborate with product teams
Operate cloud systems
Improve cloud systems
Document system architecture
Document scraping logic
Document operational procedures
Contribute to resilience
Contribute to automation
Contribute to adaptability
How You'll Work.
Team & Collaboration
Data engineering teams; Product teams
Full Job Description
## Responsibilities Design, build, and maintain large-scale Python-based scraping systems targeting highly protected websites (including Google-like environments). Architect resilient extraction systems capable of handling dynamic, JavaScript-heavy pages using browser automation and hybrid approaches. Continuously adapt systems to frequent changes in page structures, request flows, and anti-bot mechanisms. Build robust, production-grade data extraction pipelines with strong emphasis on accuracy, observability, and fault tolerance. Implement advanced strategies such as proxy rotation, fingerprinting, session management, and request routing to ensure stability at scale. Monitor system health, proactively detect anomalies, and debug complex production failures across distributed systems. Optimise scraping infrastructure for performance, cost efficiency, and reliability at scale. Collaborate with data engineering and product teams to ensure scraped data is structured, validated, and trusted. Operate and improve systems running continuously in cloud environments. Document system architecture, scraping logic, and operational procedures for long-term maintainability. Contribute to improving resilience, automation, and adaptability in adversarial environments. ## Requirements 7+ years of professional software engineering experience, with a strong focus on backend systems, data engineering, or distributed systems. Proven experience building and operating large-scale production web scraping systems. Deep hands-on experience scraping Google or similarly complex, heavily protected / anti-bot environments. Strong expertise in Python (or comparable production languages such as Go, Rust, or JavaScript). Strong understanding of HTTP internals: headers, cookies, TLS, redirects, sessions, and browser networking behaviour. Experience with browser automation frameworks such as Playwright, Selenium, Puppeteer, or equivalent. Strong knowledge of HTML parsing, DOM traversal, and high-perf
Applying for this Anti-Bot Engineer role?
Most applicants get filtered before a human reads their resume. See if yours makes the cut.
How to Apply on Lever
- Lever uses a streamlined one-page form — apply in under 5 minutes.
- LinkedIn import works well; review parsed data before submitting.
- The cover letter field is optional but visible to reviewers — use it to differentiate.
- Referral codes from employees can significantly boost visibility of your application.
ANONYMOUS · UNFILTERED
What do employees actually say about this company?
Real rants from real employees. Read before you apply.