LODESTAR: Trustworthy Entropy Is Navigated, Not Merely Measured

Entropy is a good confidence signal for picking answers in retrieval-augmented QA, right up until a misleading passage makes a frozen LLM confidently wrong. When that happens the lowest-entropy answer is the wrong one, and the standard minimum-entropy rule walks straight into it.

LODESTAR inserts a single learned polarizer between each retrieved passage and the question: a short natural-language prompt trained with reinforcement learning. It raises the respondent's entropy on misleading passages while leaving supporting ones near the entropy floor, so the same minimum-entropy rule now lands on the right passage. The LLM's weights are never touched, and no gold answers are used at selection time. Across 5,008 questions on five QA benchmarks, LODESTAR reaches the highest mean F₁ (0.5339) and macro exact match (0.4136) of any inference-ready selector, and cuts the share of selections captured by misleading passages from 30.3% to 26.0%.

For further details, please refer to our paper!