Comcast
media and technology
AIAgenticTestEngineer
Neural analysis suggests this role is
optimal for Senior candidates.
“AI Agentic Test Engineer at Comcast. Skills: AI Agentic Test Engineer, Python, LLM-as-a-Judge, Web and API automation, CI/CD pipelines. building automated evaluation loops for AI Agents using LLM-as-a-judge patterns. maintaining robust Web and API test suites”
Industry & Context.
troubleshooting; root cause analysis
This position is ineligible for visa sponsorship., To be considered for this role, you must be legally authorized to work in the United States and not require sponsorship for employment now or in the future.
What They're Looking For.
Must Have
5 + years of experience as an SDET or Automation Engineer, technical background in building and maintaining custom automation frameworks, Ability to work independently on complex coding tasks while collaborating effectively in an Agile environment, Expert-level hands-on coding in Python, Solid understanding of Agent Evaluation techniques and using LLMs as a judge, Proven experience in Web and API automation, Legally authorized to work in the United States and not require sponsorship for employment now or in the future
Nice to Have
Multi-Stack Knowledge, TypeScript, Java, React, Streamlit, CI/CD pipeline management (Jenkins/Concourse)
What You'll Do.
building automated evaluation loops for AI Agents using LLM-as-a-judge patterns
maintaining robust Web and API test suites
Design and implement automated evaluation frameworks that utilize Large Language Models to assess the quality
and safety of Agent responses
Perform hands-on validation of AI Agents within complex environments
moving beyond static assertions to intelligent
Automate the testing of real-time data streams and LLM outputs to ensure consistency and prevent regression in Agent behavior
Develop and maintain automation across TypeScript
and React environments to support various application layers
Build high-performance automated tests for modern web interfaces and RESTful APIs
ensuring end-to-end system reliability
Help build or maintain internal quality dashboards and tools using frameworks like Streamlit
Integrate automated tests into CI/CD pipelines (e. g.
Concourse) to ensure continuous quality feedback throughout the development lifecycle
Actively participate in technical code reviews
focusing on script efficiency
and clean code principles
Use logs and observability tools to perform deep-dive root cause analysis on automation failures and system defects
How You'll Work.
Team & Collaboration
collaborating effectively in an Agile environment
Full Job Description
Make your mark at Comcast -- a Fortune 30 global media and technology company. From the connectivity and platforms we provide, to the content and experiences we create, we reach hundreds of millions of customers, viewers, and guests worldwide. Become part of our award-winning technology team that turns big ideas into cutting-edge products, platforms, and solutions that our customers love. We create space to innovate, and we recognize, reward, and invest in your ideas, while ensuring you can proudly bring your authentic self to the workplace. Join us. You’ll do the best work of your career right here at Comcast. (In most cases, Comcast prefers to have employees on-site collaborating unless the team has been designated as virtual due to the nature of their work. If a position is listed with both office locations and virtual offerings, Comcast may be willing to consider candidates who live greater than 100 miles from the office for the remote option.) **Job Summary** The AI Agentic Team designs, builds, and runs intelligent agents end to end. We own agent logic, workflows, integrations, production deployment, and ongoing performance. By combining build and operations, we deliver agentic systems that are reliable, secure, and scalable in real-world use. We are building the next generation of our chat experience, transitioning from a human-assisted workflow to a fully agentic, AI-driven system. As an AI Agentic Software Engineer, you will create re-usable AI agents that will be used across Xfinity applications. Your initial focus will be to enable agentic capabilities at our Xfinity Assistant (chat application) with a strong passion for customer experience and a commitment to building intuitive, high-quality interactions. **Job Description** _This position is ineligible for visa sponsorship. To be considered for this role, you must be legally authorized to work in the United States and not require sponsorship for employment now or in the future._ We are seeking a **AI Ag
Applying for this AI Agentic Test Engineer role?
Most applicants get filtered before a human reads their resume. See if yours makes the cut.
How to Apply on Workday
- Workday has a multi-step form — save your progress after every section.
- "Apply With LinkedIn" can fail or lose data; manual entry is more reliable.
- Watch for the "Submit for Review" final step — hitting "Save" alone does not submit.
- Job requisition numbers are useful when following up with HR by email.
ANONYMOUS · UNFILTERED
What do employees actually say about Comcast?
Real rants from real employees. Read before you apply.