Senior QA Automation Engineer

Job Details
Posted Date:
November 19, 2026
 
Job ID:
47214
 
Location:
,
 
Category:
 
Experience:
6 years
 
Remote?:
Hybrid
Job Description
  • Remote: Hybrid
  • Type: Contract
  • Job #47214

Technology Partners is currently seeking a talented Senior QA Automation Engineer.

Do you have experience building automated UI, API, and end-to-end tests for AI-powered products using Python, Playwright, and CI/CD pipelines?

Let us help you make your next big career move a reality!

What You Will Be Doing:

As a Senior QA Automation Engineer, you will own quality across an artificial intelligence (AI) role-play coaching platform and its avatar experiences. You will define test strategies, execute exploratory testing across typed and spoken sessions, and establish clear release-readiness standards with product, engineering, and avatar teams. You will expand Python-based simulated-learner automation to evaluate large language model (LLM) behavior, scoring fairness, scenario adherence, and character consistency. You will design and maintain UI, application programming interface (API), and end-to-end test suites while integrating them into continuous integration and continuous delivery (CI/CD) pipelines. You will monitor pipeline health, resolve flaky tests, automate reporting and deployments, and help shorten the release cycle. You will document testing processes and mentor teammates on effective quality practices. The employment type was not specified in the provided job description.

Key Responsibilities:

• Own test strategies for the platform and individual avatars, from smoke testing through release readiness.

• Execute exploratory testing in typed and spoken sessions, then document clear and reproducible defects.

• Build automated tests that evaluate LLM-driven avatar behavior, including character consistency, scenario adherence, accurate recaps, and fair scoring.

• Expand simulated-learner personas to cover strong, weak, and edge-case behaviors across automated test runs.

• Design, build, and maintain automated UI, API, and end-to-end test suites using appropriate testing frameworks.

• Integrate automated tests, reporting, build processes, and deployment checks into CI/CD pipelines with clear pass and fail gates.

• Analyze test results across prompt and model changes, reporting regressions with measurable counts and plain-language findings.

• Partner with product, engineering, and avatar teams while mentoring teammates and documenting testing tools and processes.

Required Skills & Experience:

• 6+ years of software Quality Assurance (QA) experience, including at least 3 years writing test automation code.

• Strong Python development skills and experience with automation frameworks such as Playwright, Selenium, Cypress, or pytest.

• Hands-on experience testing AI- or LLM-powered products whose outputs vary between runs.

• Experience building and maintaining CI/CD pipelines with GitHub Actions, Azure DevOps, GitLab CI, Jenkins, or similar tools.

• API testing experience using REST, Postman, or comparable tools, along with comfort working with Git.

• Regular experience using AI assistants such as Claude, ChatGPT, Copilot, or similar tools to create test cases, analyze transcripts, or improve QA workflows.

• Ability to translate test results into clear metrics, actionable findings, and plain-language reports; education and required credentials were not specified in the provided job description.

Desired Skills & Experience:

• Experience with LLM evaluation tools or approaches such as promptfoo, DeepEval, LangSmith, or LLM-as-judge scoring.

• Experience testing voice, speech, real-time video, or streaming applications.

• Experience with performance and load testing tools such as k6, JMeter, or Locust.

• Experience with cloud and container technologies such as Azure, AWS, and Docker, or background in e-learning, simulation, or training products; ISTQB or similar QA certification is also beneficial.

Keywords: QA Automation Engineer, Senior QA Engineer, Test Automation Engineer, Python, Playwright, LLM Testing, CI/CD, API Testing, Software Quality Assurance, AI Applications

We are interested in every qualified candidate who is eligible to work in the United States. However, we are not able to provide sponsorship at this time or accept candidates who would require a corp-to-corp agreement.

If this position sounds like you, WE SHOULD TALK! Your better future is ready, and we want to put the right tools in your hands to get you there. Let’s go!

Looking for more opportunities with Technology Partners? Check out technologypartners.net/jobs!

All offers of employment at Technology Partners are contingent upon clear results of a thorough background check and drug screening that meet corresponding laws and regulations at the city, state and federal level.

Pay ranges are influenced by candidate qualifications, experience, and role specifics, with the actual rate determined considering skills, market conditions, and are subject to change by the employer; pay negotiations follow all state and federal legal guidelines.

Share this Job!

Apply Now!

Attach a Resume file. Accepted file types are DOC, DOCX, PDF, HTML, and TXT.

We are uploading your application. It may take a few moments to read your resume. Please wait!