Have you ever heard of hallucinations linked to AI systems ? These phenomena, still shrouded in mystery, worry decision-makers who fear their adverse effects when AI systems are deployed in their organizations. Here we explore the nature and implications of the hallucination phenomenon so that decision-makers can better grasp the subject and act accordingly.
Last December, the Amazon Q generative AI assistant experienced "severe hallucinations" when it revealed confidential data (location of data centres, disclosure of internal discount programmes, etc.) during a public preview (Schiffer & Newton, 2023). This example of controversy clearly highlights the risks behind hallucinations in AI. But what exactly do these hallucinations consist of?
A Wide Range of Definitions
First of all, it is interesting to note that there is no clear consensus in the definition of hallucinations related to artificial intelligence systems (AIS). Merriam-Webster defines themas a "plausible but false or misleading response generated by an artificial intelligence algorithm" (Merriam-Webster, 2024) where IBM equates them with the perception of "patterns or objects that are nonexistent or imperceptible to human observers, creating outputs that are nonsensical or altogether inaccurate" (IBM, 2024). In the course of research, we find that hallucinations are often characterized with one or more of three characteristics: they present users with unsatisfactory results (false, misleading, or absurd), convincingly (plausible, factual), and which appear to be unjustified by the model's training data.
Some prefer to use the term confabulation or delusion rather than hallucination to avoid the anthropomorphization of AIS and to emphasize the creative-gap filling principle, according to which generative AIS can creatively fill in their gaps to respond to a novel user request (Douglas Heaven, 2023).
At the Heart of Concerns: Large Language Models
Hallucinations have become a major concern in the AI field. Although they manifest themselves in the use of different types of AIS (computer vision, speech recognition, etc.) (Garg, 2023) (Carlini, 2018), they are particularly documented in the use of those involving Large Language Models (LLMs). Due to their accessibility and widespread deployment, these large text-specific language models (such as ChatGPT) are more likely to produce hallucinations with a significant negative impact. For example, Bard (Google's former generative AI chatbot, since changed to Gemini) falsely claimed during a demonstration that the James Webb Space Telescope was the first to take pictures of a planet outside Earth's solar system. The mistake cost Alphabet, Google's parent company, a 9% fall in its share price (Milmo, 2023).
Putting the Facts to the Test: The Most Problematic Hallucinations
A recent study (Zhang, et al., 2023) proposes a taxonomy of hallucinations related to LLMs in three categories : those contradictory to the input, the context or the established facts (see Figure 1). Another taxonomy distinguishes between two categories: factuality-related hallucinations (factual inconsistency or fact-fabrication) and faithfulness hallucinations (contextual, logical, or instructional inconsistency) (Ye, Liu, Zhang, Hua, & Jia, 2023). Hallucinations can also be categorized according to the different applications of LLMs (synthesis, translation, dialogue systems, etc.) or the different types of LLMs (multilingual, domain-specific, etc.) to provide a more complete picture.
Factuality-related hallucinations are the most documented because, depending on the context of the AIS user, its consequences can be significant. Industries where factuality is paramount (such as healthcare, legal, finance, education, etc.) are particularly likely to be resistant to such risk. In these contexts, it is therefore essential to understand the possible origin of such hallucinations in order to have the means to remedy them.
Data, Model, and Inference: the Sources of Hallucinations
LLMs are models that, in order to be effective, require a significant training phase on large bodies of data. From the initial training process to the user interaction phase, there are multiple sources of hallucinations (see Figure 2). The figure below illustrates some of the sources of hallucinations by grouping them into three types : those related to the data, the model itself, and the inference phase (interactions and queries). Some notable sources of hallucinations are related to training data biases, parametric knowledge biases, data encoding problems, or overfitting (Zhang, et al., 2023) (Huang, et al., 2023). These sources of hallucinations are identifiable throughout the life cycle of an LLM. Today, there are also tools for detecting hallucinations based on generation (evaluation of texts generated by an LLM) or discrimination (evaluation of the LLM's ability to distinguish between a truthful statement and a hallucinated statement) (Ji, et al., 2024). From the development to the deployment of an LLM, different levers of action are then emerging, allowing a decision-maker to reduce the rate of problematic hallucinations for his or her target activity.
Levers for action throughout the life cycle of LLMs
Measures to mitigate hallucinations thus exist throughout the life cycle of LLMs (see Figure 3). The most emphasized metric is the careful selection of data used to pre-train and fine-tune models. Indeed, a high-quality data set, adapted to a specific need, can significantly reduce the probability of obtaining unsatisfactory results for the user of an LLM. In addition, other measures also reduce the risk of hallucinations, whether during the model development phases (use of knowledge graphs, supervised fine-tuning, human review, etc.) or inference (prompt engineering, retrieval-augmented generation or RAG, the use of fact-checking methods, etc.) (Tonmoy, et al., 2024).
In the current state of LLMs on the market, if a decision-maker wishes to limit the negative impacts of hallucinations in his or her organization, he or she has options to initiate sound risk management of his AIS. But can the risk of hallucinations really be eliminated?
Hallucinations: Bug or Feature?
Although Sam Altman, the CEO of OpenAI, joked that he has little confidence in the answers generated by ChatGPT, he believes that the problem of hallucinations will be solved within 2 years (O'Brien & The Associated Press, 2023). Other experts, such as Emily Bender, Director of the Computational Linguistics Lab at the University of Washington, or Yann LeCun, Chief AI Scientist at Meta, think otherwise. LeCun even calls for a complete paradigm shift : according to him, Generative AIS (based on LLMs that are inherently auto-regressive, and therefore probabilistic) can never be free of hallucinations because the responses generated are structurally uncontrollable (LeCun, 2023). Indeed, LLMs do not "reason" in the same way as human beings and it is therefore obvious that a certain number of the answers generated are nonsensical or absurd. More and more experts are arguing that hallucinations are not a bug but a feature of the technology behind Generative AIS. By broadening the definition of hallucination as considered so far in our reflection (i.e. an unsatisfactory result presented in a factual way) to that of an inherent characteristic of Generative AIS, it becomes possible to take a new look at this phenomenon and to see its advantages.
Hallucinations as a source of creativity
For example, a study suggest staking a positive look at the hallucinations of LLMs by emphasizing their creative potential (Jiang, et al., 2024). In particular, the authors draw a parallel between the hallucinations of LLMs and two historical "hallucinations" that led to major scientific progress : heliocentrism, which was historically perceived as a factual "hallucination" before the scientific revolution made possible by the publication of Copernicus, and the accidental discovery of penicillin through Fleming's involuntary experiment, which can be compared to a faithfulness "hallucination”. In addition, research in cognitive science points out that human creativity is not only about retrieving information, but also about recombining and expanding existing knowledge, a phenomenon similar to the hallucinations of LLMs.
At the organizational level, it is thus possible to draw up many use cases that make it possible to take advantage of this creativity: ideation in marketing, design thinking, product development, data visualization, strategic foresight, etc. Microsoft, for example, plays on this characteristic of LLMs by offering with its AI tool Copilot the possibility of adjusting the degree of factuality or creativity of the answers generated by the chatbot.
To go further : Humans and Technology
At the end of this synthesis, we can see that AIS hallucinations question us about our perceptions of technology as well as about notions related to responsibility, trust or truth.
First of all, it seems that two pitfalls threaten any user of a Generative AIS : anthropomorphism, in this case the tendency to endow an AIS with "human" reasoning capacities (or even an expert status) or, conversely, a desire to deceive; and forgetting the gap between the current state of technology and the use we want to make of it (use an LLM as a search browser, a database, etc.). Behind the effervescence of the discourse on hallucinations, we can detect two concerns : the need for reliability and the issue of responsibility.
In the first case, some of the discourses in vogue seem excessive: is it relevant (and possible?) to demand absolutely factual answers from a Generative AIS when humans are not capable of doing so themselves ? To date, there is no universal, objective standard of truth. It is therefore a matter of the decision-maker, with the means at his disposal, weighing up as best he can the factuality and margin of error of the AIS developed or used, according to its field of application (a weighting that is certainly different in the application of the AIS to the health sector compared to that of artistic design).
As for the responsibility for the impacts caused by hallucinations, it therefore appears to be shared between the providers and developers of Generative AIS, who create and calibrate their models for an expected use, and the users of these systems who are responsible for a reasonable and critical use of the generated content. Here too, levers for action exist : the development of ethical governance of AIS at the organizational level and raising stakeholder awareness (customers, employees, etc.) about technology, to name but two.
Beyond hallucinations, in the development and deployment of AIS on an organizational scale, the application of ethics as a methodology for understanding, assessing and raising awareness of the impacts associated with AIS makes it possible to create the conditions for trust. Trust, not in a technology, but between the people who offer a tool as a service, and the people who use that tool to meet a need.
Aymeric Thiollet
Human Technology Foundation, April 22nd 2024
Schiffer, Z., & Newton, C. (2023, December). Amazon's Q has 'severe hallucinations' and leaks confidential data in public preview, employees warn. Retrieved from Platformer : https://www.platformer.news/amazons-q-has-severe-hallucinations/
Merriam-Webster. (2024, April). Hallucination. Retrieved from Merriam-Webster: https://www.merriam-webster.com/dictionary/hallucination
IBM. (2024, April). What are AI hallucinations? Retrieved from ibm.com: https://www.ibm.com/topics/ai-hallucinations
Douglas Heaven, W. (2023, May 2). Geoffrey Hinton tells us why he's now scared of the tech he helped build . Retrieved from MIT Technology Review: https://www.technologyreview.com/2023/05/02/1072528/geoffrey-hinton-google-why-scared-ai/
Garg, S. (2023, March 14). What are AI Hallucinations? Retrieved from All Things How: https://allthings.how/what-are-ai-hallucinations/
Carlini, N. (2018). Audio Adversarial Examples. Retrieved from nicholas.carlini.com: https://nicholas.carlini.com/code/audio_adversarial_examples/
Milmo, D. (2023, February 9). Google AI chatbot Bard sends shares plummeting after it gives wrong answer. Retrieved from The Guardian: https://www.theguardian.com/technology/2023/feb/09/google-ai-chatbot-bard-error-sends-shares-plummeting-in-battle-with-microsoft
Zhang, Y., Li, Y., Cui, L., Cai, D., Liu, L., Fu, T., . . . Shi, S. (2023, September 24). Siren's Song in the AI Ocean: A Survey on Hallucination in Large Language Models. Retrieved from arxiv.org: https://arxiv.org/pdf/2309.01219.pdf
Ye, H., Liu, T., Zhang, A., Hua, W., & Jia, W. (2023, September 13). Cognitive Mirage: A Review of Hallucinations in Large Language Models. Retrieved from arxiv.org: https://arxiv.org/pdf/2309.06794.pdf
Huang, L., Yu, W., Ma, W., Zhong, W., Feng, Z., Wang, H., . . . Liu, T. (2023, November 9). A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questions. Retrieved from arxiv.org: https://arxiv.org/pdf/2311.05232.pdf
Ji, Z., Lee, N., Frieske, R., Yu, T., Su, D., Xu, Y., . . . Fung, P. (2024, February 19). Survey of Hallucination in Natural Language Generation. Retrieved from arxiv.org: https://arxiv.org/pdf/2202.03629.pdf
Tonmoy, S., Zaman, S., Jain, V., Rani, A., Rawte, V., Chadha, A., & Das, A. (2024, January 8). A Comprehensive Survey of Hallucination Mitigation Techniques in Large Language Models. Retrieved from arxiv.org: https://arxiv.org/pdf/2401.01313.pdf
O'Brien, M., & The Associated Press. (2023, August 1). Tech experts are starting to doubt that ChatGPT and A.I. 'hallucinations' will ever go away: 'This isn't fixable'. Retrieved from Fortune: https://fortune.com/2023/08/01/can-ai-chatgpt-hallucinations-be-fixed-experts-doubt-altman-openai/
LeCun, Y. (2023, March 24). n.d. Retrieved from LinkedIn: https://www.linkedin.com/posts/yann-lecun_i-have-claimed-that-auto-regressive-llms-activity-7045908925660950528-hJGk?
Jiang, X., Tian, Y., Hua, F., Xu, C., Wang, Y., & Guo, J. (2024, February 2). A Survey on Large Language Model Hallucination via a Creativity Perspective. Retrieved from arxiv.org: https://arxiv.org/pdf/2402.06647v1.pdf
Dhaduk, H. (2023, October 26). The Curious Case of LLM Hallucination: Causes and Tips to Reduce its Risks. Retrieved from Simform: https://www.simform.com/blog/llm-hallucinations/
