Uniphore is one of the largest B2B AI-native companies—decades-proven, built-for-scale and designed for the enterprise. The company drives business outcomes, across multiple industry verticals, and enables the largest global deployments.
Uniphore infuses AI into every part of the enterprise that impacts the customer. We deliver the only multimodal architecture centered on customers that combines Generative AI, Knowledge AI, Emotion AI, workflow automation and a co-pilot to guide you. We understand better than anyone how to capture voice, video and text and how to analyze all types of data.
As AI becomes more powerful, every part of the enterprise that impacts the customer will be disrupted. We believe the future will run on the connective tissue between people, machines and data: all in the service of creating the most human processes and experiences for customers and employees.
Job Description:
We are looking for an experienced AI Quality Manager to lead the evaluation, monitoring, and continuous improvement of our AI and machine learning systems. In this role you will define quality standards for AI outputs, build robust testing and benchmarking frameworks, partner cross-functionally with engineering and product teams, and ensure our models meet both internal performance benchmarks and external regulatory requirements.
This is a highly cross-functional leadership role sitting at the intersection of AI safety, product quality, and operational excellence.
Key responsibilities
Quality standards & governance
- Define and own quality metrics and acceptance criteria for AI systems across all product lines
- Establish and enforce AI quality policies aligned with internal standards and emerging regulatory frameworks
- Maintain a living AI quality handbook accessible to engineering, product, and legal stakeholders
Testing & evaluation
- Design and execute structured evaluation pipelines for LLM and ML model releases, including regression suites, edge case testing, and output quality audits
- Build automated monitoring systems to detect model drift, hallucination rates, bias signals, and output degradation in production
- Own the incident response process for AI quality failures, from root-cause analysis to post-mortem and remediation
Cross-functional leadership
- Partner with ML engineers, data scientists, and product managers to embed quality checkpoints into the model development lifecycle
- Lead a team of AI quality analysts and evaluation engineers, setting goals, conducting performance reviews, and mentoring career growth
- Present quality reports and risk assessments to senior leadership and external auditors
Data & tooling
- Oversee the build-out and maintenance of internal evaluation datasets, annotation pipelines, and benchmarking infrastructure
- Evaluate and procure third-party testing tools, model cards, and audit services as needed
Required qualifications
- 7+ years of experience in quality assurance, with at least 3 years focused on AI/ML systems or data-driven products
- Deep understanding of ASR (WER, keyword hit rate) and LLM evaluation methods (RLHF, human preference evaluation, automated LLM-as-judge techniques)
- Hands-on experience with Python and ML frameworks sufficient to review model code and evaluation scripts
- Proven track record owning a quality program from inception through production — including writing standards, building tooling, and managing teams
- Strong written and verbal communication skills; ability to translate complex AI risk concepts for non-technical stakeholders
- Experience with statistical analysis for model performance measurement (precision/recall, F1, BLEU, ROUGE, win-rate, etc.)
Preferred qualifications
- Experience designing structured human evaluation programs or preference studies for generative AI outputs
- Experience managing annotation vendors or running large-scale human evaluation programs
- Prior work in a high-growth AI product company or research lab
Location preference:
Uniphore is an equal opportunity employer committed to diversity in the workplace. We evaluate qualified applicants without regard to race, color, religion, sex, sexual orientation, disability, veteran status, and other protected characteristics.
For more information on how Uniphore uses AI to unify—and humanize—every enterprise experience, please visit www.uniphore.com.
