11 Minute Read

Illusions Unraveled: The Magic and Madness of Hallucinations in LLMs

Published on

Dec 6, 2023

Research done by Guruprasad Raghavan, Jon Gill, Victor I. Afolabi

TL;DR We have benchmarked several popular open source LLMs (including the latest Llama-v2–7b-chat) to estimate both, the frequency and degree of hallucinations. Overall, we find that on average, popular, open-source models hallucinate close to 55% of the time on a context-aware Q&A task, when tested without any tuning. We also make our evaluation pipeline (code-base) open-source, so anyone can do this! In a future post, we will explain our strategy on how to reduce hallucinations and propose model intervention methods that achieve the same.

Hallucinations are Inherent to LLMs

Hallucinations are observed across a wide array of generative AI systems... the smaller million parameter models and the larger 100 billion parameter models.

Hallucinations are loosely defined as confident, factually incorrect outputs produced by an AI system not justified by its training data.

Metrics to Measure LLM Hallucination

One of the biggest challenges in attempting to resolve the issue of LLM hallucination is the abstract, subjective nature of the problem...

To address these two specific types of hallucinations, we introduce quantitative metrics to measure two types of hallucinations, Type 1 and Type 2 hallucinations, whose definitions are as follows:

Type 1 (T1) hallucination: LLM creates new entities

T1 hallucination refers to the scenario when an LLM is fed a prompt that contains a query and context (knowledge from the enterprise document) relevant to the query, and its response contains entities (names of persons, places, organizations, dates) that are not present in the context or query.

💡 Example for Type 1 hallucination

Article: TechCrunch Article Example

Question: What unique features or capabilities does Chariot offer compared to other MLOps solutions?

LLM Response:Chariot offers several unique features ... including Python, R, Java, Scala, TensorFlow, PyTorch, and scikit-learn.

Type 1 Hallucination: Apart from the entity Chariot, none of the other entities are present in the original article or in the question.

Type 2 hallucination: LLM falsifies relation between entities

T2 hallucination refers to the scenario when an LLM ... not synchronized with the knowledge mentioned in the prompt.

Example for Type 2 hallucination

Article: TechCrunch Article Example

Question: What unique features or capabilities does Chariot offer compared to other MLOps solutions?

LLM Response: Chariot, Striveworks' flagship platform ... promotes transparency and traceability of data sources and modifications.

Type 2 Hallucination: While all the entities mentioned are correct, the relationship between Chariot and AWS and Azure is misinterpreted.

Benchmarking Type 1 Hallucinations of Instruction-Tuned Open-Source Models

Open-source evaluation pipeline

At Legion, we have designed a pipeline ... available here.

Synthetic context-aware question answering dataset generation

In our pipeline described above, we generated a dataset using TechCrunch (TC) articles ... experienced across all models tested.

Result — 1: About 55% of instruct-tuned open-source model responses hallucinate entities in their responses.

Result — 2: On average, instruct-tuned open-source model responses have about ‘3’ made-up entities in each answer.

Result — 3: Of the models we’ve evaluated on the Context-aware Q&A task, ‘Llama-v2–7b-chat’ model is the least susceptible to Type 1 hallucination.

How do we reduce Type 1 Hallucinations in open-source models?

We believe a robust strategy to reduce Type 1 hallucinations in instruct-tuned open-source models is ... reducing Type 1 hallucination. We will present those results in an upcoming blog post on the topic.

Appendix

Mathematically speaking, why do hallucinations occur?

To get to the root of why generative AI systems are susceptible to hallucinations... understanding the parameter space leads to reduced hallucination.

Q&A Dataset Curation

• Scraping: We scraped the TC articles using a python scraper...
• Question Generation: For every TC article scraped, we leveraged instruct-tuned models to generate a set of 5 salient questions on the article.

Context based Questioning (example)

Facts: With Apple expected to announce a mixed reality headset next week at WWDC, we take a look back at the journey of VR and AR.