Predictive Cognitive Failure Learning (PCFL): A Pattern Recognition Framework for Early Hallucination Forecasting and Adaptive Reasoning in Large Language Models
Department of Artificial Intelligence and Data Science, Ajeenkya DY Patil School of Engineering, Pune, India · India
Abstract
When errors occur in the form of hallucinations or factual distortions in a Large Language Model (LLM), they are exacerbated by the fluency of the model’s output. Error correction techniques are generally post-hoc and result in delays to users. This paper analyses ten different studies including surveys, repeated sampling, in-context validation of dubious tokens, confidence calibration, probing and document retrieval. A gap is identified in the field where risk estimation can be performed during decoding using purely model generated signals. As a solution to this gap, we describe Predictive Cognitive Failure Learning (PCFL), where token probabilities, attention and drift between reasoning steps are used as signals to predict failure using a compact risk estimation module. This risk score is then used by a controller to decide to continue, recheck, rewrite or alert the user to the text. This is a design study and PCFL has not been implemented or assessed. Therefore, we describe how we will develop a prototype and assessment plan for future work.
Keywords
How to Cite This Article
Rights
BODHIVRUKSHA JOURNAL OF DIVERSE DISCIPLINE