Self-consistency and voting

What if the model gets chain-of-thought wrong? It can happen since different reasoning paths sometimes lead to different answers. Self-consistency solves this: generate multiple CoT responses, then pick the answer that appears most often. It is like polling a room of experts and going with the majority.

Majority voting is not foolproof because if the model has a systematic bias, all runs might agree on the wrong answer. But it significantly reduces random errors. For critical decisions, combine voting with other techniques like chain-of-thought or document grounding for extra confidence.

Here is how self-consistency works: 1. Ask the same question 3-5 times with temperature > 0 (to get diverse reasoning) 2. Extract the final answer from each response 3. Take the majority vote Example: "What is 17 x 23?" Run 1: 17 x 23 = 17 x 20 + 17 x 3 = 340 + 51 = 391 Run 2: 17 x 23 = (20-3) x 23 = 460 - 69 = 391 Run 3: 17 x 23 = 17 x 25 - 17 x 2 = 425 - 34 = 391 All three paths, different reasoning, same answer: 391. High confidence.

Self-consistency also helps detect hallucinations. If you ask "What year did humans land on Mars?" three times and get three different years, the inconsistency signals that the model is making it up. Consistent answers across runs increase confidence; inconsistent answers are a red flag.

Use self-consistency when accuracy matters more than cost: medical reasoning, financial calculations, legal analysis, or any decision with real consequences. For casual tasks like writing emails or generating ideas, a single response is fine. The technique shines when wrong answers are expensive.