I spent three days at TrustCon 2026 listening to practitioners from Meta, Bumble, Pinterest, Discord, Slack, Stack Overflow, and dozens of other platforms. They're wrestling with the same core problem: how do you catch bad actors at machine speed without breaking platform trust for everyone else?

The conversations were raw. Many were technical. And they revealed that the biggest challenges facing the industry aren't usually about having access to cutting-edge tools, they're about fundamental operational and technical constraints that no one has fully solved.

1. The Human Label Bottleneck Is Real, and AI Isn't the Silver Bullet Everyone Originally Thought

Every content safety team I heard faces the same bottleneck. Human labeling doesn't scale with dynamic policy and label changes. 

A Meta ML engineer, shared his approach to semi-supervised learning, where instead of requiring thousands of human-labeled examples, they combine a small, carefully labeled dataset with a much larger unlabeled dataset with an aggregate positive rate. Using KL-divergence-based loss functions, models can learn from this unlabeled data without explicit labels.

The benefit is that when a safety policy changes, and it will, you don't need to relabel everything. You retrain only the lightweight teacher classifier head, using minimal new labels.

The limits were clear though across the multiple AI focused sessions I sat through. The way most companies are using LLMs is expensive, latency-sensitive, and unreliable at scale. The field isn't moving toward "LLM does all your content moderation." It's moving toward "LLMs are one measure in a much larger decision system."

2. Multi-Stage Systems and Context Are Non-Negotiable

Bumble's two-stage moderation system stuck with me. The example used was a hypothetical Bumble chat message saying "I'm going to destroy you". Since we’re talking about Trust & Safety here, the message begs the question… Is this a threat? The next slide revealed the context of the message being a discussion about a chess match. Ok, not a threat. They tackle situations like this in two stages:

• First stage: broad detection with a fine-tuned language model

• Second stage: uncertain cases get routed to a system that considers metadata and behavioral context

The real constraints are latency and cost. You can't afford to send everything through multiple stages. So this process keeps the automation pipeline fast and cheap by catching clear violations early, then routing everything else through richer context signals that take a touch more time, compute, and cost to determine.

The pattern appeared across multiple conference sessions. Pinterest concentrates review budget on high-exposure, high-likelihood items using ML-assisted sampling. Discord flags account integrity issues through metadata (emoji changes, IP shifts) rather than content alone. Several platforms shared how they look at non-text signals (user behavior, interaction history, device and network signals, etc). 

Automated systems execute >95% of all moderation actions for all major big tech players thereby requiring a very high confidence threshold for automatic triage, placing significant emphasis and pressure on the select signals driving these high-stakes decisions.

3. AI-Generated CSAM and Novel Abuse Patterns Are Outpacing Defenses

The child safety conversation was the most sobering. AI-generated CSAM is no longer a future problem. It's happening now. And the technical barrier to entry is dropping fast.

Open-source diffusion models are widely available. I learned there are “CSAM studios”, as the presenter called them, that produce both content and instruction sets. The abuse is moving from the dark web toward mainstream UGC spaces, becoming more interactive (real-time generation, conversational systems) and more immersive (3D, VR).

The fundamental question teams are wrestling with: are we stopping the studios (supply), or stopping the distribution and knowledge-sharing (demand)? Today's tools excel at detection. But with the use of AI, the speed of innovation in abuse tactics especially is outpacing the speed of detection tooling.

This was a consistent thread of urgency across the event. Child safety teams have a continual uphill battle and the threats are scaling faster than what traditional moderation can handle.

4. The Scale Problem Is Forcing Rethinking of What Success Means

Most safety programs were built around a containment model. They detect harm, take action, and move on. But as platforms grew, the sheer volume revealed that you can't precisely label and action everything.

Pinterest's approach to prevalence measurement is instructive. Instead of trying to perfectly classify every piece of content, they measure the fraction of user impressions going to policy-violating content daily, with confidence intervals. It's designed-based sampling but acknowledges the reality that they’re estimating prevalence at scale, not achieving perfect accuracy.

Smaller platforms shared the similar lesson. When you're a team of 1-2 writing policy, "good enough" beats perfect. The framework is to focus on the worst-of-the-worst first. Use LLMs to stress-test your bias, not make decisions. Start with hash-based detection, layer in the complex tooling as patterns emerge.

I watched this play out during a panel on Minimum Viable Policy. Midway through the session, someone two seats to my left responded to a Slack on their phone, then immediately opened their laptop.They started going through videos at 2x speed, scanned the metadata and notes, then switched to a Google Sheet to mark it as a violation or not and add their reasoning. 

It was reminiscent of old-school content flagging with an Incident -> Triage -> Decision. Took maybe 20-30 seconds a video, but I couldn’t help but imagine what it took for those videos to get to her immediate attention. But that’s what it takes when content is in the gray area of policy and systems haven’t been implemented to moderate edge cases.

5. Nobody Can Solve This Alone

This is perhaps the most important theme that cut across TrustCon. Cross-platform signal sharing and coordination are ways to disrupt abuse that moves across services. Individual platforms can build detection systems. But bad actors, scammers, and those with bad intent operate across platforms. A fraudster banned on one service signs up on another tomorrow.

Industry-led trust and safety networks, like the Global Internet Forum to Counter Terrorism (GIFCT) and Lantern by the Tech Coalition are helping and facilitating coordinated efforts. 

There’s a rising tide of open source tech that’s getting better and more accessible in the Trust & Safety industry. Teams are having to make fewer decisions between building and buying. They're choosing between buying from a vendor and adopting open source software with internal customization. The open-source layer handles the commodity parts (review queues, rules engines, escalation workflows), therefore freeing engineering to focus on platform-specific policy enforcement and behavioral signals that actually matter.

My takeaways

Every presenter at TrustCon, from Meta's ML researchers to Slack's solo policy operator, was chasing one goal: catch bad actors without breaking trust for everyone else.

It sounds straightforward. But the moment you try to do it at scale, the constraints compound. I saw this during a panel on dating app scams. Eight people (what I recall counting) lined up at the mic to ask questions of and share critiques on what the panelists were doing. Extortion. Love scams. Financial fraud. Harassment. Threats. Each requires different signals, different policies and different enforcement. The frustration was palpable.

There's no single technical solution. Detection needs contextual signals; sampling needs to reflect what users actually see. LLMs are powerful but hard to operationalize at scale, especially against novel threats and policy drift. And the paradox is that teams are stopping the overwhelming majority of bad actors despite all that. They're succeeding in an uphill battle. But the bad actors aren't slowing down. They're accelerating. And innovating…unfortunately. 

That gap, the one between what practitioners are accomplishing and what bad actors are capable of, is what I took away from TrustCon 2026. It's also why this conference exists and why no one closes that gap alone.