Exterview Logo

Exterview

QA automation engineer

Sorry, this job was removed at 09:05 p.m. (IST) on Wednesday, Nov 18, 2026
In-Office
Hyderabad, Telangana, IND
Expert/Leader
In-Office
Hyderabad, Telangana, IND
Expert/Leader

Similar Jobs

2 Days Ago
In-Office
Hyderabad, Telangana, IND
Expert/Leader
Expert/Leader
Angel or VC Firm
Designs and scales automated testing for web, API, and backend services using Playwright, Selenium, and pytest. Defines testing strategy, integrates tests and AI agents into CI/CD systems, develops autonomous QA tools, analyzes failures, improves coverage, and mentors engineers. Collaborates with product and engineering teams to establish acceptance criteria, promote quality practices, and report quality metrics. Requires advanced experience with AI-driven testing, cloud platforms, containers, databases, APIs, and modern QA methodologies.
Top Skills: AWSAzureAzure OpenaiCircleCIClaudeClaude CodeCodexConfluenceCursorDockerEmbeddingsGatlingGCPGithub ActionsGithub CopilotGitlab CiGraphQLJavaJavaScriptJenkinsJIRAJmeterK6KubernetesLanggraphModel Context ProtocolOpenaiOpenai Agents SdkPlaywrightPostmanPytestPythonRagRestRestassuredSeleniumSQLSupertestTypescriptVector Databases
12 Days Ago
In-Office
Hyderabad, Telangana, IND
Senior level
Senior level
Information Technology
Builds and maintains scalable batch and streaming data pipelines, data models, data flows, APIs, and integrations across on-premises and cloud environments. Translates business requirements into technical designs, code, testing, deployment, and maintenance tasks. Ensures data quality, governance, accessibility, and readiness for reporting, analytics, and AI/ML use cases. The role also requires advanced QA testing and automation expertise using tools such as Pytest, Selenium, Playwright, or similar platforms, along with P&C insurance and Guidewire experience.
Top Skills: AgileAi/MlAPIsAzure DevopsBig DataBlue PrismCloud ComputingGitGuidewireJIRANlpNon-Relational DatabasesPlaywrightPytestPythonRelational DatabasesRpaSeleniumSnowflakeSQL
15 Days Ago
Remote or Hybrid
India
Senior level
Senior level
Information Technology • Consulting
Design and implement an end-to-end QA automation framework for Microsoft Fabric data platforms. Validate Medallion Architecture layers, pipelines, schemas, data quality, reconciliations, Power BI semantic models, reports, and AI agents. Build regression suites using SQL, Python, PySpark, Playwright, and TypeScript; investigate root causes; monitor observability metrics; configure alerts; and integrate automated checks into Azure DevOps or other CI/CD systems. Collaborate with data engineering, BI, AI, and DevOps teams.
Top Skills: Azure DevopsAzure Log AnalyticsClaudeData WarehouseDataflows Gen2DaxDelta LakeFabric GitFabric PipelinesGithub ActionsGreat ExpectationsLakehouseMedallion ArchitectureMicrosoft FabricOnelakePlaywrightPower BIPysparkPythonSodaSpark SqlSQLT-SqlTypescript
Own the quality architecture and automation strategy for an AI-driven interview platform. Design scalable UI, API, event-driven, real-time, media, performance, and reliability testing across AI screening and interview workflows. Establish assertion strategies for probabilistic outputs, integrate automated quality gates into GitHub Actions, monitor flaky tests and quality metrics, and define production sign-off criteria. Partner with engineering teams on testability and observability while mentoring QA engineers and driving product-wide quality ownership.
The summary above was generated by AI
Role Overview

You will be the Quality Intelligence Architect of Exterview.

As a QA Automation Engineer (10+ Years), you’ll design, implement, and own the automation and quality orchestration system that validates AI-driven candidate screening, live interviews, scoring agents, and feedback pipelines.

This is quality engineering at scale: validating probabilistic AI outputs, real-time interview flows, event-driven systems, and media-heavy workflows across thousands of concurrent interviews.

You’ll work closely with Backend, AI Engineering, Prompt Engineering, and Frontend teams to ensure quality is built into the system, not tested after the fact.

Execution is tracked via Linear (tasks), Notion (PRDs & test strategy), and GitHub (automation reviews) ensuring quality decisions are transparent and measurable.

Key ResponsibilitiesAutomation Architecture
  • Design and own a scalable, maintainable automation framework for:

    • Web (Next.js)

    • APIs (GraphQL via AppSync, REST)

    • Event-driven and async workflows

  • Move QA from test execution to quality architecture ownership.

AI & Probabilistic System Validation
  • Validate:

    • Resume parsing accuracy

    • AI scoring consistency

    • Interview logic stability

  • Define assertion strategies for non-deterministic AI outputs.

Real-Time & Media Workflow Testing
  • Test live interview systems:

    • VideoSDK + Tavus video flows

    • Real-time state updates

  • Validate video upload, playback, retries, and failure handling.

API, Event & Integration Testing
  • Build API-first automation for:

    • GraphQL queries & mutations

    • Lambda & microservice APIs

  • Validate event-driven flows (notifications, interview state transitions).

Performance, Load & Reliability Testing
  • Design load and stress tests for:

    • Interview orchestration APIs

    • Video playback & report generation

  • Ensure system reliability under peak concurrency.

CI/CD & Quality Gates
  • Integrate automation into GitHub Actions.

  • Define release quality gates and production sign-off criteria.

  • Track flaky tests, failure patterns, and quality metrics.

Cross-Team Quality Leadership
  • Translate product requirements into test strategies.

  • Partner with engineering to improve testability & observability.

  • Mentor junior and mid-level QA engineers.

Success MetricsFirst 90 Days
  • Stable automation framework covering UI + API + AI workflows.

  • Flaky test rate reduced to near zero.

  • API-level load tests live for interview & video flows.

  • Release quality gates enforced in CI/CD.

12 Months
  • End-to-end automation coverage for all interview types.

  • Predictable, fast release cycles with no QA bottlenecks.

  • AI validation framework adopted as product standard.

  • QA recognized as product quality owner, not gatekeeper.

Must-Haves
  • 10+ years in QA Automation / SDET roles

  • Strong experience designing automation frameworks, not just writing scripts

  • Expertise in API testing (GraphQL & REST)

  • Experience testing real-time and async systems

  • Ability to validate AI-driven, non-deterministic outputs

  • Strong understanding of CI/CD, release quality, and production readiness

Nice-to-Haves
  • Experience testing AI/ML or agent-based systems

  • Media or video workflow testing experience

  • Startup or high-scale SaaS background

  • Prior ownership of QA strategy for a product

Tech Stack Visibility
  • Frontend: Next.js, TypeScript

  • Automation: Cypress, Playwright, Jest

  • Backend: Node.js, AWS Lambda, Serverless Framework

  • APIs: GraphQL (AWS AppSync), REST

  • Auth: AWS Cognito (OAuth, OTP)

  • Data: MongoDB Atlas

  • Media: VideoSDK, Tavus, Amazon S3

  • Communication: Twilio (SMS, WhatsApp, IVR)

  • Observability: Langfuse, internal monitoring

Assessment (PoC)Objective

Validate your ability to design quality systems, not just write test cases.

Challenge Overview

Design and implement an automation strategy for an AI-driven interview flow.

The system should validate:

  • Resume upload & parsing

  • AI resume–job match scoring

  • Interview scheduling

  • Live interview execution

  • Interview report generation

Functional Expectations1. Automation Design
  • Cover:

    • One UI flow

    • One GraphQL API flow

    • One real-time or async workflow

  • Show clear separation between test layers.

2. AI Validation Strategy
  • Define how you:

    • Assert AI scores

    • Detect drift or instability

    • Handle acceptable variance

3. Performance & Reliability
  • Include:

    • One API load test

    • Failure / retry validation

  • Avoid direct S3 URL testing; use APIs only.

Deliverables
  • Automation repo (clean structure)

  • Test strategy README

  • Example AI assertion logic

  • CI pipeline config (GitHub Actions)

  • ≤ 5 min Loom walkthrough explaining decisions

HQ

Exterview Hyderabad, Telangana, IND Office

Hyderabad, India

What you need to know about the Hyderabad Tech Scene

Because of its proximity to leading research institutions and a government committed to the city's growth, Hyderabad's tech scene is booming. With plans to establish India's first "AI city," the city is on track to become one of the world's most anticipated tech hubs, with companies like TransUnion, Schrödinger and Freshworks, among others, already calling the city home.

Sign up now Access later

Create Free Account

Please log in or sign up to report this job.

Create Free Account