Brandon D. Staple
University of Nebraska Medical Center, Omaha, NE, United States
Publications
-
Research Article
Comparison of Domain-Specific and Ensemble Large Language Models in Surgical Education: A Preliminary Performance Evaluation
Author(s): Brandon D. Staple, Elijah M. Staple, Cynthia Wallace and Bevan L. Staple*
Standard Large Language Models (sLLMs) are known for their high accuracy in answering multiple-choice questions from the Self-Assessment Neurosurgery Exam (SANS). However, their tendency to 'hallucinate' or fabricate information presents challenges for neurosurgical applications that require a high degree of precision. AtlasGPT, a Domain-specific Large Language Model (dLLM), has managed to achieve a lower Hallucination Rate (HR) through targeted fine- tuning and retrieval-augmented generation from specialized databases. Nevertheless, proprietary limitations hinder customization and broader research into hallucination mitigation, prompting an exploration of model-agnostic Ensemble Methods (EMs) that combine several sLLMs to enhance performance. This study assessed hallucination mitigation by comparing an EM consisting of three sLLMs (Gemini, Claude 3.5 Sonnet, and Mistral) with.. Read More»