You will be the Quality Intelligence Architect of Exterview.
As a QA Automation Engineer (10+ Years), you’ll design, implement, and own the automation and quality orchestration system that validates AI-driven candidate screening, live interviews, scoring agents, and feedback pipelines.
This is quality engineering at scale: validating probabilistic AI outputs, real-time interview flows, event-driven systems, and media-heavy workflows across thousands of concurrent interviews.
You’ll work closely with Backend, AI Engineering, Prompt Engineering, and Frontend teams to ensure quality is built into the system, not tested after the fact.
Execution is tracked via Linear (tasks), Notion (PRDs & test strategy), and GitHub (automation reviews) ensuring quality decisions are transparent and measurable.
Key ResponsibilitiesAutomation ArchitectureDesign and own a scalable, maintainable automation framework for:
Web (Next.js)
APIs (GraphQL via AppSync, REST)
Event-driven and async workflows
Move QA from test execution to quality architecture ownership.
Validate:
Resume parsing accuracy
AI scoring consistency
Interview logic stability
Define assertion strategies for non-deterministic AI outputs.
Test live interview systems:
VideoSDK + Tavus video flows
Real-time state updates
Validate video upload, playback, retries, and failure handling.
Build API-first automation for:
GraphQL queries & mutations
Lambda & microservice APIs
Validate event-driven flows (notifications, interview state transitions).
Design load and stress tests for:
Interview orchestration APIs
Video playback & report generation
Ensure system reliability under peak concurrency.
Integrate automation into GitHub Actions.
Define release quality gates and production sign-off criteria.
Track flaky tests, failure patterns, and quality metrics.
Translate product requirements into test strategies.
Partner with engineering to improve testability & observability.
Mentor junior and mid-level QA engineers.
Stable automation framework covering UI + API + AI workflows.
Flaky test rate reduced to near zero.
API-level load tests live for interview & video flows.
Release quality gates enforced in CI/CD.
End-to-end automation coverage for all interview types.
Predictable, fast release cycles with no QA bottlenecks.
AI validation framework adopted as product standard.
QA recognized as product quality owner, not gatekeeper.
10+ years in QA Automation / SDET roles
Strong experience designing automation frameworks, not just writing scripts
Expertise in API testing (GraphQL & REST)
Experience testing real-time and async systems
Ability to validate AI-driven, non-deterministic outputs
Strong understanding of CI/CD, release quality, and production readiness
Experience testing AI/ML or agent-based systems
Media or video workflow testing experience
Startup or high-scale SaaS background
Prior ownership of QA strategy for a product
Frontend: Next.js, TypeScript
Automation: Cypress, Playwright, Jest
Backend: Node.js, AWS Lambda, Serverless Framework
APIs: GraphQL (AWS AppSync), REST
Auth: AWS Cognito (OAuth, OTP)
Data: MongoDB Atlas
Media: VideoSDK, Tavus, Amazon S3
Communication: Twilio (SMS, WhatsApp, IVR)
Observability: Langfuse, internal monitoring
Validate your ability to design quality systems, not just write test cases.
Challenge OverviewDesign and implement an automation strategy for an AI-driven interview flow.
The system should validate:
Resume upload & parsing
AI resume–job match scoring
Interview scheduling
Live interview execution
Interview report generation
Cover:
One UI flow
One GraphQL API flow
One real-time or async workflow
Show clear separation between test layers.
Define how you:
Assert AI scores
Detect drift or instability
Handle acceptable variance
Include:
One API load test
Failure / retry validation
Avoid direct S3 URL testing; use APIs only.
Automation repo (clean structure)
Test strategy README
Example AI assertion logic
CI pipeline config (GitHub Actions)
≤ 5 min Loom walkthrough explaining decisions
