<p>Sr Quality Analyst</p> <p>Role Summary</p> <p>This role requires deep understanding of AI fundamentals, hands-on experience with LLM testing, guardrail design, data sanitization,</p> <p>and evaluation of agent trajectories. The tester will work closely with AI engineers, data scientists, and product teams to ensure safe, reliable, and high-quality AI behavior.</p> <p>5-8+ years in QA, with at least 2-3 years in AI/ML or LLM testing.</p> <p>Strong understanding of AI fundamentals: embeddings, vector stores, RAG, LLMs, agentic frameworks.</p> <p>Hands-on experience testing:</p> <ul> <li>LLM prompts and outputs </li><li>Agentic flows (e.g., LangChain, AutoGen, Semantic Kernel, AIP agents) </li><li>Model evaluation metrics </li></ul> <p>Experience writing guardrails, safety rules, and evaluation prompts.</p> <p>Knowledge of data sanitization, PII detection, and data governance.</p> <p>Familiarity with pipeline validation and MLOps workflows.</p> <p>Ability to evaluate model performance using Low-Pick Score or similar ranking metrics.</p> <p>Strong Python skills for test automation and evaluation scripting.</p> <p>New York3 - 6 Years10H29-Jan-2026YACTIVE115872-3-1</p>