People keep asking why nobody in Trust and Safety ever speaks out.
Casey Newton just wrote another piece about it, wondering why so few in the field go on the record. Then he followed up by asking the same question to the wrong people: The ones with company budgets to network, the ones at the top, the ones who show up for the panels and the photo ops. The ones whose every moment at work isn’t monitored and costed like a factory line.
He asked the ladies who lunch at their annual mutual admiration conference why they don’t speak out, and seemed surprised when none of them had much to say. Maybe it’s because they’ve built whole careers on not knowing what the front lines are like. Or maybe it’s because they’re terrified of losing their position and having to do some real work.
The truth is, there are people who speak out. I’m one of them, though apparently I don’t count. The African moderators are shouting at the top of their lungs, unionising and working with groups like Foxglove to get their stories out. But they don’t get invited to the wine-and-swag conferences. They’re not useful voices in the industry’s little psychodrama of concern.
What’s being performed there is what I call safety theatre. It looks like action. It feels like progress. But it’s not built to change anything. It’s built to protect brands from legal risk by proving they have a T&S team, to create busy-work for people chasing LinkedIn “likes”, and to sell themselves to the tool lobbyists.
Omelas isn’t a metaphor here. It’s a business model. The child in the basement still suffers, but now there are snack-boxes and branded tote bags for free.
The Safety Industrial Complex
Hardly any of the narrative in this field is being written by survivors, or even by practitioners. There is some good journalism, but unless it’s followed up with legal action, it’s fleeting. A somewhat infamous PR person I used to work with put it bluntly: no matter how bad the headlines sound, they’re forgotten by next week.
Instead, the narrative of Trust and Safety is being shaped by vendors and special interest groups. Tool companies throw money, gifts, events, and soft power at anyone they think might help them close a sale. I’ve seen it first-hand and the bribery wasn’t even subtle: swag boxes, wine tastings, even entire astroturfed “T&S orgs” set up by companies trying to make their brand synonymous with responsibility.
Just like the antivirus companies of old, they sometimes feel like they invent problems simply to sell you the cure. You get these catered roundtables full of people talking about the latest panic: AI-powered CSAM, abusive messages, scam detection. And somehow, the conversation always aligns neatly with their newest product features.
Meanwhile, the actual people doing the review work are barely surviving. A few companies are selling “less exposure” or “real mental health support”, but the research behind those products rarely comes from the front lines. Their focus groups are the ladies who lunch, not the people who actually use the tools. The companies doing genuine support work aren’t part of the bribery gaggle – They’re not at the wine tastings.
The systems that harm us are also the ones writing the safety playbook, and the people who manage front-line workers are often the ones most easily sucked into the cult of the safety-snake-oil sellers. Most don’t have formal training in safety. They don’t understand complex AI, ethics, or trauma care. So they trust the vendors instead.
The rest of us are expected to nod along; and the poor buggers at the front are just expected to go along with questionable, uninformed decisions. After all, those people are disposable so who cares about them?
The Myth of Resilience
You can’t fix the trauma of this work with a monthly therapy session and a mindfulness app. I’ve been in high-level meetings that ended with the offer of free BetterHelp subscriptions to moderators, mentally picturing them ticking off “Wellness” on their HR checklist of moral responsibilities and staff health.
You can’t self-care your way through a job that shows you the worst things humans do to each other on a daily basis. The people I’ve worked with, the good ones anyway, end up leaving with C-PTSD or worse. And it’s not because they were weak. It’s because the job was impossible, and nobody told them that upfront.
I practise something I call realistic wellness with my teams. It means peer-support. It means giving people space to leave a project, or say no to a new task, without question or consequence. It means agency – real agency – in what projects they take on and how they manage their time. It also means seeing their managers actually doing the same work as them. Not sitting on LinkedIn all day. Not putting together a slideshow of their last Caribbean vacation to show their team what they are missing. Not making PowerPoints about SLAs and the money they’ve saved the company for their next performance review.
Unfortunately, servant leadership in this field comes at a price. We take on the same horrors they do, and more. I’m the last escalation point, which means I see the worst of the worst. And my teams see what it does to me. They see what happens to someone who’s been doing this for decades, someone who can never take vacation because I know I wouldn’t be able to come back.
I’m not a shining example of how to leave this work healthy; and that’s a tension I lived with every day.
Outsourcing as Moral Detachment
Outsourcing doesn’t just hurt the people on the receiving end. It hurts the people inside the company who actually give a shit about their colleagues.
When you care about your team, and half your “team” is technically employed by someone three layers removed, there’s very little you can do. You can’t talk to them directly. You can’t see how they’re doing. You’re given filtered reports by people whose job is to minimise problems, not solve them. Those people are mostly there to protect the contract, and their own company, at all costs.
This isn’t an accident. Just like Western consumers don’t want to know what happens in garment factories or slaughterhouses, Western T&S managers are kept in the dark about what their frontline workers endure. But this isn’t a factory. It’s supposed to be a team; and if your team isn’t allowed to be your team, then you’re not leading. You’re managing abstractions. The “them and us” ultimately becomes a chasm.
On two separate occasions, good outsourced moderators in the Philippines were removed from my project and fired by their company. When I asked why, I was told it was due to minor breaches. One of them had told a friend what company they were moderating for, because they were proud of their work. These weren’t major errors from my point of view. I asked for them to be returned and was told no.
That’s when it became very clear that even if these outsourced workers were working entirely on my project, their wellbeing and direction were completely out of my hands. That is not how I work. And it means I can’t extend the values of our team to the people doing exactly the same job.
Vanity Hires and Safety Theatre
Another part of this Theatre of Horror is the vanity hire. These are people brought in from other industries to generate headlines and make it look like the company is doing something big and meaningful. It usually happens just after the last vanity hire failed publicly and was quietly fired.
They’re not hired because they understand the work. Companies bring in big names from social media firms (generally people who’ve never even met a front-line moderator) – Or from law enforcement, crisis response, or journalism, and act like the skills are all interchangeable. They aren’t.
This is akin to hiring someone who managed the security protocols for a state school board and then asking them to run a nuclear submarine. Completely different risks, skills, and outcomes.
Ultimately, the goal isn’t capability. It’s optics. The industry doesn’t need people who can lead a team through everyday horrors. It needs people who can speak fluently on panels while the real work happens out of sight.
Why People Don’t Speak
In the West, most people can’t afford to speak out. Moderator pay is generally well below living wage – far too low for what the job demands. Severance is conditional and NDAs are everywhere. Even when they’re legally unenforceable, they’re terrifying if you don’t have access to legal support.
Companies know this. And now they’re hiring people who they know won’t speak. People who need the job too much. People who want to be seen as agreeable. People who know how to behave on LinkedIn.
Most of the people I managed left the field sick, traumatised, or broken. Many left because of direct abuse – not from users, but from inside the company. The customer service team quit en masse because a serious criminal act of abuse was being ignored and covered up. Some of them spoke to journalists and one of the stories was meant to run in Wired – I was asked to verify facts, which I did; but it was pulled at the last minute.
This is not an industry that wants sunlight. And even though today’s news becomes tomorrow’s fish-and-chip wrapping, they still can’t bear the idea of their internal failures being made public.
Why I Stayed
I used to wonder why I stayed, when so many of the good people I knew eventually left. People who had every reason to go, every scar to show for it, and probably left it too late for their short-term sanity.
Why did I keep going through all of it? The ever-increasing mental health debt. The knowledge that I was going to leave this job with my health destroyed. The inevitable failure. Dealing with the idiocracy and nepotism. Trying to ignore the internal abuse. Having to work alongside people who were being paid my staff’s average annual wage every single week?
Then one day, talking to my daughter, it clicked.
My grandmother died in a care home because they refused to install £50 temperature limiters on the taps. One of the untrained staff boiled her alive. The coroner ruled it manslaughter and his report was scathing – But all the company got was a nominal fine that the insurance covered. Nobody was held responsible. Nothing changed.
I wasn’t the victim. I was the bystander. Watching a system protect itself instead of the people it was supposed to care for, morally and legally. That helplessness never left me.
That’s why I stayed. Not for the company. Not for the mission. For the people who might not have anyone else to listen, or notice, or tell them they’re not going mad when they see how bad things really are.
(Wait… was I the kid in the basement?)
The Problem With Omelas
People love to reference The Ones Who Walk Away from Omelas as if it’s a story about individual ethics. But in the upper echelons of this industry, most don’t walk. They thrive. They network. They build influence and followers. They carve out comfortable careers on top of a system that chews through everyone beneath them.
Some of them know. Most of them choose not to. Either way, the child in the basement is still there.
The ones who stay and do the work aren’t at TrustCon. They’re not giving keynotes or chasing niche influencer status. They’re burnt out, traumatised, and hoping someone somewhere is trying to build something better.
Not to protect the brand. Not for engagement metrics. Not for any praise, because there isn’t any at that level. They do it because somebody has to do it. And if not them, who else?
———————————————————————————————-
Notes:
• This article was not written by AI but it was structured by GPT from a stream of consciousness mind-dump; which is a very good way of working for us ADHD folks. It also tidied up some messy sentences, but I don’t like letting it go too wild.
• Is you haven’t read Ursula LeGuin’s “The Ones Who Walk Away from Omelas” then it is online here: https://shsdavisapes.pbworks.com/f/Omelas.pdf
• The Wired article was cut in half, and was published here: https://www.thebureauinvestigates.com/stories/2024-07-22/okcupid-put-users-at-risk-of-assault-by-ignoring-safety-concerns-say-former-staff – I had very little to do with this article, but I am glad it exists, even in its cut down version.
• I thought carefully about using “Ladies who lunch”, an oft-used genderless industry insult. It is a nod to Stephen Sondheim’s song in A Little Night Music, where it wryly describes well-heeled socialites who spend their days dining and networking rather than doing substantive work. I decided that with the industry association and the apt reference, it fits well here.
• Foxglove is here: https://www.foxglove.org.uk/tag/content-moderators/
I am proud of my African (and other) colleagues for speaking out.