OpenAI and Anthropic are partnering with New Zealand-based crisis response firm ThroughLine to develop an AI-powered tool designed to detect and redirect users showing signs of violent extremism.
Founded by former youth worker Elliot Taylor, ThroughLine already operates a global network of 1,600 helplines across 180 countries, used by major AI platforms to connect users in mental health crises with immediate human support.
Now, the company is expanding its scope to address extremist rhetoric, working closely with The Christchurch Call - the international initiative formed after New Zealand’s 2019 terrorist attack. The proposed system would use expert-guided AI, not generic large language model training data, to identify at-risk individuals and route them to verified mental health and deradicalization services.
"We’re not just filtering content," Taylor said. "We’re responding to relationship dynamics." The tool would avoid shutting down conversations, recognizing that users often disclose dangerous thoughts to AI they would never voice to a person.
The initiative comes as governments pressure AI firms to curb online radicalization, while extremist groups migrate to unregulated platforms like Telegram. ThroughLine’s system aims to close that gap without triggering escalation.
OpenAI confirmed its partnership with ThroughLine; Anthropic and Google did not respond.
No release date has been set, but pilot testing is underway. Counterterrorism experts warn success hinges on robust follow-up infrastructure and trust in local support networks.