Hiring Trust & Safety Engineers: The Complete Guide

# Trust & Safety Engineer

**Location:** San Francisco, CA (Hybrid) · **Employment Type:** Full-time · **Level:** Mid-Senior

## About [Company]

[Company] is a social platform where millions of creators share content and build communities. Trust and safety is foundational to our mission—users need to feel safe to create, share, and engage authentically.

Our platform sees 100M+ pieces of content daily across text, images, and video. We enforce policies against harmful content (hate speech, harassment, dangerous misinformation, CSAM) while protecting free expression. It's complex, nuanced, and critically important.

**Why join [Company]?**

- Work on content safety at scale—100M+ daily content pieces
- Real-world impact—your work directly protects users
- Cross-functional collaboration with policy, legal, and operations
- Join a 20-person T&S engineering team within a 500-person engineering org
- Well-funded company (Series D) with strong commitment to safety
- Comprehensive wellness support (this work matters, and so do you)

## The Role

We're looking for a Trust & Safety Engineer to help us build scalable safety systems that protect our community. You'll work on content classification, abuse detection, and the infrastructure that enables policy enforcement at scale.

This is meaningful work with real challenges. You'll need to translate nuanced policy into technical systems, handle edge cases that don't fit neatly into categories, and build tools that help human reviewers work effectively. Automation alone can't solve these problems—you'll build systems that combine ML with human judgment.

The ideal candidate combines ML skills with good judgment on sensitive content. You understand that a false positive might wrongly remove legitimate expression, and a false negative might expose users to harm. You're thoughtful about these tradeoffs.

## Important Note on Content Exposure

This role involves some exposure to disturbing content during model development, debugging, and quality review. We take this seriously and provide:

- Proactive wellness support including counseling and mental health resources
- Content exposure limits and rotation policies
- Regular wellness check-ins with managers
- Opt-out options for specific content categories
- Team culture that normalizes discussing the emotional impact

We'll discuss this honestly during the interview process. Your wellbeing matters.

## What You'll Work On

### First 30 Days
- Onboard to our T&S stack, content pipelines, and classification systems
- Understand our policies, enforcement actions, and the content landscape
- Observe content review processes to understand the problem space
- Ship your first improvement (classifier tweak, tool enhancement)

### First 90 Days
- Own a content safety area (hate speech, harassment, or misinformation)
- Build or improve classifiers for your area
- Contribute to our review tooling and operations workflows
- Collaborate with policy team on policy-to-system translation

### First Year
- Lead T&S initiatives that measurably improve user safety
- Architect improvements to our content classification infrastructure
- Develop approaches for emerging harm types
- Mentor teammates on T&S engineering practices

## Objectives of This Role

- Improve content classification accuracy while maintaining appeal overturn rate
- Reduce harmful content exposure time (faster detection, faster action)
- Build systems that scale with content volume growth
- Improve review efficiency for operations team
- Support policy changes with technical implementation

## Responsibilities

**Classification Systems (50%)**
- Build and improve content classification models (text, image, video)
- Engineer features for content safety detection
- Handle nuanced classification (context matters, bright lines are rare)
- Implement policy changes as classifier updates
- Monitor model performance and investigate regressions

**Infrastructure & Tooling (30%)**
- Design content moderation infrastructure for scale
- Build review queues, appeals systems, and enforcement tools
- Implement audit logging for transparency reports
- Optimize content pipeline latency (harmful content seen = harm caused)
- Support operations team with tooling improvements

**Collaboration & Analysis (20%)**
- Work with policy team to translate policies into technical requirements
- Analyze content trends, emerging harm types, and enforcement patterns
- Respond to emerging safety challenges (new attack vectors, current events)
- Participate in content calibration sessions (understand the problem space)
- Create documentation for compliance and transparency

## Required Skills and Qualifications

**Technical Skills**
- 4+ years of ML engineering experience
- 2+ years in trust & safety, content moderation, or abuse prevention
- Experience with text and image classification at scale
- Understanding of precision/recall tradeoffs for sensitive content
- Strong Python and SQL skills
- Experience with content moderation tooling

**Domain Knowledge**
- Understanding of online harms (hate speech, harassment, misinformation)
- Familiarity with content policy structures and enforcement
- Experience translating policy into technical systems

**Soft Skills**
- Good judgment on nuanced, ambiguous content decisions
- Strong collaboration with policy, legal, and operations teams
- Communication skills for cross-functional work
- Emotional resilience with appropriate support
- Commitment to user safety and platform integrity

## Preferred Skills and Qualifications

- Video content moderation experience
- Multi-lingual content experience
- Regulatory knowledge (DSA, KOSA, etc.)
- Experience with appeals and transparency systems
- Background in NLP for content understanding
- User research or human-computer interaction background

## Tech Stack

**ML:** Python, PyTorch, Hugging Face Transformers, custom models

**Content Pipeline:** Apache Kafka, custom content processing infrastructure

**Classification:** Multi-modal models, ensemble classifiers, human-in-the-loop

**Tooling:** Custom review tools, queue management, appeals systems

**Infrastructure:** AWS (EKS, SageMaker), Kubernetes

**Monitoring:** Datadog, custom T&S dashboards, incident response tools

## Compensation and Benefits

**Salary:** $140,000 - $190,000 (based on experience)

**Equity:** 0.04% - 0.12% (4-year vest, 1-year cliff)

**Benefits:**

*Health & Wellness*
- Medical, dental, and vision insurance (100% covered)
- Enhanced mental health support through Lyra (12 sessions/year)
- $150/month wellness stipend
- Quarterly wellness days (additional PTO for T&S team)

*Time Off*
- Unlimited PTO with 15-day minimum encouraged
- 11 paid company holidays
- 16 weeks paid parental leave
- Content exposure rotation breaks

*Professional Development*
- $2,500 annual learning budget
- Conference attendance (TrustCon, All Things Moderated)
- Internal T&S knowledge sharing

*Financial*
- 401(k) with 4% company match
- Equity refresh grants annually

*Workspace*
- $1,500 home office setup allowance
- Hybrid work (2-3 days in SF office)
- MacBook Pro and equipment of choice

## Engineering Culture

**How we work:**
- User safety is the priority—we take this responsibility seriously
- Human judgment + ML systems—neither alone is sufficient
- Close collaboration with policy, legal, and operations
- Regular content calibration to stay grounded in real problems
- Transparent metrics and honest assessment of what's working

**What we value:**
- Protecting users from harm while respecting expression
- Thoughtful tradeoffs between safety and experience
- Supporting each other—this work is hard
- Continuous improvement—the threat landscape evolves
- Work-life balance and mental health (sustainable pace)

## Interview Process

Our interview process typically takes 2 weeks.

- **Step 1: Recruiter Screen** (30 min)
  Background discussion, role overview, and honest conversation about content exposure.

- **Step 2: Technical Screen** (60 min)
  ML fundamentals and T&S concepts with a T&S engineer.

- **Step 3: T&S Deep Dive** (90 min)
  Technical discussion about content classification, policy translation, and your experience. How would you approach a specific content safety challenge?

- **Step 4: ML Coding** (90 min)
  Practical ML exercise—text classification, feature engineering, or evaluation.

- **Step 5: Cross-Functional** (60 min)
  Discussion with policy or operations partner. How do you collaborate on ambiguous cases?

- **Step 6: Team Interviews** (2 x 30 min)
  Meet potential teammates. Discussion about collaboration and working style.

---

*[Company] is an equal opportunity employer. We're committed to building an inclusive team and supporting diverse perspectives in trust & safety work.*