TikTok removed a key safety feature from millions of users as part of an A/B test, and one of those users, 16-year-old Chase Nasca, died by suicide after being flooded with harmful content. The revelation, reported by Bloomberg and detailed in a confidential document tied to a lawsuit, raises urgent questions about the ethics of algorithm testing on vulnerable populations.
What Was the TikTok Safety Feature Test?
In early 2022, TikTok developed an improved algorithm designed to reduce harmful content loops—repetitive patterns of videos about suicide, self-harm, and depression. The update aimed to detect these loops and diversify the feed to prevent users from being trapped in a spiral of damaging material.
To measure the update's impact, TikTok ran an A/B test. The new algorithm was deployed to the vast majority of users, while a control group—roughly 10% of U.S. users, about 15 million people—continued to receive the old algorithm without the safety measures. Chase Nasca's account was randomly selected for that control group.
Chase Nasca's Case: A Random Assignment with Tragic Consequences
According to the confidential document, in the two weeks before his death, TikTok recommended Chase 7,563 videos. Many of these videos reportedly focused on suicide, self-harm, and depression. Because he was in the control group, the new safeguards that might have interrupted these harmful loops were not active on his account.
Chase's story is not an isolated data point—it is a stark illustration of what can happen when safety features are withheld for experimental purposes. The fact that his inclusion in the control group was random underscores the lack of consent or awareness involved.
Why A/B Testing Safety Features Is Ethically Fraught
A/B testing is standard in digital product development. It allows companies to compare two versions of a feature and measure which performs better. But when the feature in question is a safety mechanism, the stakes are fundamentally different from testing a new button color or layout.
Withholding a safety feature from a control group means that some users are knowingly exposed to potential harm. In TikTok's case, that harm allegedly included content that promoted self-harm and suicide. The ethical review boards that oversee medical and psychological research would never approve such an experiment without informed consent and strict safeguards. Yet tech companies often apply the same testing frameworks to social features without equivalent oversight.
Key Takeaways
- 15 million U.S. users were placed in a control group that did not receive TikTok's improved safety algorithm.
- Chase Nasca, 16, was randomly assigned to that group and received thousands of harmful video recommendations before his death.
- The case highlights the ethical risks of A/B testing safety features without user consent or independent oversight.
- Regulators and lawmakers are increasingly scrutinizing algorithmic experiments that affect minors.
How TikTok's Algorithm Shapes What Teens See
TikTok's recommendation engine is designed to maximize engagement. It learns from every like, share, and watch time to serve up more of what keeps users on the app. When a user lingers on a video about depression, the algorithm may interpret that as interest and serve more similar content—creating a feedback loop.
The safety update tested in 2022 was meant to break those loops by introducing diversity into the feed and flagging repetitive harmful themes. For users in the control group, that brake was absent. The result, according to the lawsuit, was a torrent of damaging content for a teenager who was already vulnerable.
What This Means for Parents, Regulators, and Tech Companies
For parents, the case is a reminder that algorithmic curation can expose children to harmful material even on platforms that claim to have safety measures in place. It also shows that those measures may not be applied uniformly. The background to this story is set out in Instagram CEO Denies Hiding Teen Safety Feature Data in Meta Trial. A related thread runs through TikTok Blackout Challenge Inquest Reopened After Teen Death. Earlier coverage in TikTok Blackout Challenge Inquest Reopened for Teen Boy shows how the argument developed.
For regulators, the revelation adds fuel to efforts to mandate transparency and independent audits of algorithmic experiments. The Kids Online Safety Act and similar proposals aim to require platforms to default to their safest settings for minors and to disclose when they conduct experiments that could cause harm. A closer look at the same subject appears in Perez Hilton Hospitalized After Self-Harm Livestream on TikTok. Anyone following the wider picture can start with Noah Woods, 3, Died After Leaving Playground Through Fence Gap. The same theme surfaces in TikTok Drops Appeal, Pays £12.7m Fine for Under-13 Users, which takes a different angle.
For tech companies, the lesson is clear: safety features should not be treated as optional variables in an experiment. When the potential downside is user harm, the default should be the safest version, not the one that yields cleaner data. Context on how this played out before sits in Indonesia Fire Haze: Devastating Toll on Millions Exposed. A companion piece, Zyzz Death: Why Bodybuilding Influencer Died at 22, approaches the same subject from another angle.
Comparison of Algorithm Versions
| Feature | Updated Algorithm (90% of users) | Old Algorithm (10% control group) |
|---|---|---|
| Harmful content loop detection | Yes | No |
| Feed diversification | Yes | Limited |
| Repetitive theme flagging | Active | Inactive |
| User exposure to suicide/self-harm content | Reduced | Higher |
FAQ
What is A/B testing and why did TikTok use it?
A/B testing is a method where users are split into groups to compare two versions of a product feature. TikTok used it to measure whether its new algorithm reduced harmful content loops compared to the old version.
How many users were in the control group?
Approximately 10% of U.S. users—about 15 million people—were in the control group that did not receive the safety update.
What happened to Chase Nasca?
Chase Nasca was a 16-year-old TikTok user who was randomly assigned to the control group. In the two weeks before his death, TikTok recommended him over 7,500 videos, many about suicide and self-harm. He died by suicide.
What can parents do to protect their teens?
Parents can use TikTok's family safety mode, set screen time limits, and have open conversations about online content. They should also be aware that algorithmic recommendations can expose teens to harmful material even when safety features exist.