<p>Product Content Engineering is a horizontal function supporting initiatives across Meta's Family of Apps. We partner closely with product and technical teams to solve problems by providing content-centered solutions, setting standards of quality, and building the frameworks that ensure AI-powered experiences actually work for people. We're looking for a people manager to lead a team within PCE's AI Content Understanding group. This team is responsible for executing evaluations and improving the quality of Meta's content understanding models - the AI systems that understand what content is about and how it should be ranked, recommended, and moderated. The work spans content quality, AI evaluation, and the search and recommendation systems that power Meta's products - building the frameworks, rubrics, and pipelines that hold AI outputs to a high standard. Your team assesses model behavior, identifies where it falls short, and works cross-functionally with engineering, product, research, and data science teams to make it better. Reporting to the AI CU lead, you'll manage a team of Content Engineers and contingent workers, owning the day-to-day execution of human evaluations, golden dataset creation, auto-eval calibration, and cross-functional delivery with engineering, data science, and product partners. You'll drive operational rigor while navigating a fast-moving environment where evaluation frameworks, model capabilities, and team processes are evolving simultaneously.</p> <p>Minimum Qualifications</p> <ul> <li>8+ years of experience in content strategy, content operations, AI evaluation, or a related field </li><li>2+ years of people management experience, including hiring, developing, and performance-managing direct reports </li><li>Experience managing cross-functional programs with engineering, data science, and product partners in fast-paced environments </li><li>1+ years working with generative AI products, AI evaluation, prompt engineering, annotation, and/or content labeling and analysis </li><li>Experience designing or operationalizing evaluation frameworks, annotation guidelines, or quality rubrics for AI/ML systems </li><li>Demonstrated ability to manage multiple concurrent workstreams with competing priorities and tight deadlines </li><li>Proven analytical skills with experience interpreting evaluation data and communicating findings to technical and non-technical audiences </li><li>Track record of building team operational processes and quality standards from the ground up or during periods of significant change </li></ul> <p>Preferred Qualifications</p> <ul> <li>Demonstrated ability to integrate AI tools to optimize/redesign workflows and drive measurable impact (e.g., efficiency gains, quality improvements) </li><li>Experience adhering to and implementing responsible, ethical AI practices (e.g., risk assessment, bias mitigation, quality and accuracy reviews) </li><li>Demonstrated ongoing AI skill development (e.g., prompt/context engineering, agent orchestration) and staying current with emerging AI technologies </li><li>Experience managing contingent worker (CW) teams, vendor relationships, or scaled annotation operations </li><li>Familiarity with AI evaluation methods such as human eval, LLM-as-judge, model benchmarking, A/B testing, or red-teaming </li><li>Experience with Python, SQL, or other tools for data analysis and evaluation automation </li><li>Background in content understanding, search quality, recommendation systems, or trust and safety </li><li>Experience managing through organizational transitions, including shifts from manual to automated workflows </li><li>BA or BS in Computer Science, Data Science, Linguistics, or related field </li></ul> <p>Responsibilities</p> <ul> <li>Manage and develop a team of Content Engineers and contingent workers, setting clear goals, providing regular feedback, and supporting career growth </li><li>Own the execution of continuous CU model evaluations - coordinating sprint planning, reviewer assignments, QA processes, and delivery timelines across multiple concurrent workstreams </li><li>Drive the creation and maintenance of golden datasets that serve as ground truth for model benchmarking and auto-eval calibration </li><li>Partner with engineering, data science, and product teams to translate evaluation insights into actionable recommendations for model improvement and prompt optimization </li><li>Lead the team's contribution to LLM-as-a-Judge (auto-eval) initiatives - ensuring human evaluation data is used to calibrate, validate, and improve automated evaluation systems </li><li>Define and maintain evaluation guidelines, rubrics, and quality standards in partnership with Lead Content Engineers, ensuring consistency across reviewers and use cases </li><li>Build repeatable operational processes for evaluation sprints, including reviewer training, calibration sessions, and escalation workflows </li><li>Manage CW workforce planning - hiring, onboarding, allocation across workstreams, and performance management </li><li>Synthesize evaluation results into structured reports and present findings to cross-functional leadership, including engineering leads and lead product stakeholders </li><li>Identify and mitigate operational risks - staffing gaps, timeline conflicts, quality regressions - before they impact delivery </li><li>If you have a proven experience of proactively identifying, scoping and implementing innovative solutions, demonstrated success in identifying issues and improving processes, to create impact and bring demonstrated experience in content judgment to AI evaluation, we encourage you to apply </li></ul> <p>About Meta</p> <p>Meta builds technologies that help people connect, find communities, and grow businesses. When Facebook launched in 2004, it changed the way people connect. Apps like Messenger, Instagram and WhatsApp further empowered billions around the world. Now, Meta is moving beyond 2D screens toward immersive experiences like augmented and virtual reality to help build the next evolution in social technology. People who choose to build their careers by building with us at Meta help shape a future that will take us beyond what digital connection makes possible today-beyond the constraints of screens, the limits of distance, and even the rules of physics.</p> <p>Equal Employment Opportunity</p> <p>Meta is proud to be an Equal Employment Opportunity employer. We do not discriminate based upon race, religion, color, national origin, sex (including pregnancy, childbirth, reproductive health decisions, or related medical conditions), sexual orientation, gender identity, gender expression, age, status as a protected veteran, status as an individual with a disability, genetic information, political views or activity, or other applicable legally protected characteristics. You may view our Equal Employment Opportunity notice here.</p> <p>Meta is committed to providing reasonable accommodations for qualified individuals with disabilities and disabled veterans in our job application procedures. If you need assistance or an accommodation due to a disability, fill out the Accommodations request form.</p>