Meta has announced new updates aimed at protecting teenagers while they use artificial intelligence technologies on its platforms, by monitoring conversations that may indicate self-harm or the risk of suicide.
The key feature is the system's ability to send instant notifications to parents if a teenager, using an account linked to parental control tools on Instagram, has a conversation with the company's smart chatbot that includes hints about wanting to harm themselves.
Meta developed this system in collaboration with experts and parents to detect subtle verbal cues that may indicate danger, even if they are indirect.
The company explained that all conversations that the system is aware of will be subject to manual review, but it will send alerts to parents in unclear cases as a precaution. While acknowledging that this policy may lead to some false alerts, it considers this a suitable starting point with a promise to continue improving accuracy.
In addition, Meta is developing an additional tool that allows direct communication with emergency services when a conversation is detected indicating that a user is at imminent risk of suicide, a feature the company already has for monitoring posts on Facebook and Instagram.
Meta previously monitored frequent searches for dangerous terms on Instagram, but the new update expands the scope of monitoring to include conversations with artificial intelligence via Instagram and WhatsApp, in a move aimed at enhancing rapid response and effective protection for teenagers in the digital space.
