Chain-of-thought basics

It works because of how LLMs generate text. Each token is predicted based on all previous tokens. When the model writes out reasoning steps, those intermediate tokens become context that helps it arrive at a more accurate final answer. Without those reasoning tokens, the model jumps straight to a conclusion and often gets it wrong.

LLMs predict the next likely token (word fragment) based on everything before it. Without guidance, they jump straight to the most probable answer, which can be wrong for multi-step problems. Chain-of-thought forces the model to generate reasoning tokens first, which leads to more accurate final answers.

Not every prompt. For straightforward tasks like translation or simple lookups, the extra reasoning tokens just add cost without improving quality. Save chain-of-thought for problems that genuinely need multi-step reasoning, like math, logic, code debugging, or anything where the model might jump to a wrong conclusion.

Chain-of-thought (CoT) prompting is simple: you add "Think step by step" or "Show your reasoning" to the prompt. This forces the model to generate intermediate reasoning tokens before reaching the answer. Without CoT: "5 machines, 5 widgets, 5 minutes. 100 machines?" -> "100 minutes" (wrong) With CoT: "Think step by step. 5 machines, 5 widgets, 5 minutes. 100 machines?" -> "Each machine makes 1 widget in 5 minutes. So 100 machines each make 1 widget in 5 minutes = 100 widgets in 5 minutes." (correct)