Researchers created a fake illness, and neural networks successfully convinced people of its existence. This fake diagnosis experiment revealed security vulnerabilities in chat applications and demonstrated how easily misinformation can spread.
In 2024, researcher Almira Osmanovic Tunström from the University of Gothenburg conducted an experiment to test the extent to which artificial intelligence could spread misinformation. She invented a non-existent disease called "picsonomania," fabricated its symptoms and cause (blue light), and then published fake articles in scientific databases. The results exceeded expectations, as AI systems began citing this fictitious disease as if it were real, according to Nature News.
Shortly after the fake news was published, popular AI systems like Google Gemini and Microsoft Bing began listing "picsonomania" as a rare condition caused by excessive blue light exposure. Even ChatGPT started providing users with answers about symptoms, treating "picsonomania" as if it were a real illness.
The fake articles created by the researcher were quickly cited in scientific journals. One study published in the journal Cureus even relied on these works before being retracted. This highlights how easily false data can infiltrate the scientific community.
AI) can have serious consequences, particularly the spread of misinformation in medical sources. Health experts have expressed concern that these systems do not always verify the accuracy or reliability of data, potentially contributing to the dissemination of false information that poses a threat to users' health, especially when relied upon for medical advice.
The researchers suggested developing reliability verification systems for AI models to reduce the spread of false data. Experts also emphasized the need for standards and periodic procedures to review the information provided by these systems in the medical field.
“Pixsonomania” is not just an example of how easily artificial intelligence can be misled, but also a clear indication of the urgent need to improve data filtering and verification mechanisms.
