The rapid development in the field of artificial intelligence has sparked new controversy following reports indicating that some neural networks may produce inaccurate or misleading information in certain contexts, raising questions about their reliability.
In response to a request to prepare a report on a phenomenon of interest to the user, artificial intelligence in some cases began to provide increasingly inaccurate or unreliable information.
This is because neural networks do not critically verify information sources available online; rather, they rely on their own published data, which may contain inaccurate or misleading content. Consequently, several controversial cases have been documented, particularly in journalism, where some journalists relied on AI-generated reports without adequately verifying their accuracy. Similar problems have also emerged in other fields, including scientific research.
In an attempt to address this problem, scientists from Reshetnev University in Krasnoyarsk, eastern Siberia, developed a methodology aimed at reducing unreliable or fabricated information in the responses of intelligent models. This methodology relies on systems known as RAG (Retrieval-Enhanced Generation), where a knowledge base is created from high-quality, reliable sources, which the artificial intelligence then uses to generate responses.
The researchers explained that this approach greatly reduces the likelihood of generating inaccurate information, but errors may still occur due to data entry errors, query inconsistencies, or an incomplete knowledge base.
A research team led by Associate Professor Anastasia Polyakova from the Department of Intelligent Systems and Automation analyzed cases where AI responses exhibited inaccuracies and developed a classifier to identify them. They also created automated testing instructions that generate trial queries and compare responses to reference criteria, evaluating accuracy based on semantic similarity measures.
Based on the results of the initial phases, the researchers developed a prototype real-time monitoring unit that records queries and the context of the dialogue, assesses the reliability of the responses, and assigns a confidence score to each model. If the confidence level drops or a potential error is detected, the unit sends an alert to the supervisor.
The researchers point out that this methodology is characterized by its flexibility, as it can be applied in chatbots and in government systems that rely on artificial intelligence in multiple fields, from medicine to legal and religious fields.
