The technique was demonstrated in 2022 in two forms: few-shot prompting, where exemplar questions are shown with worked reasoning, and the zero-shot discovery that appending "Let's think step by step" elicits similar behaviour. Both work for the same reason — the steps keep intermediate state in the text, where the model can attend to it, and allocate more computation to harder problems instead of forcing an immediate answer.
Reasoning models moved the technique from a prompting trick into the model itself. OpenAI's o-series and DeepSeek-R1 are trained with reinforcement learning to produce long internal reasoning chains before answering, and they outperform prompted chain of thought substantially on mathematics, code and science. The costs moved with it: reasoning tokens are billed and take time, so reasoning effort is now a tunable production parameter, not a free upgrade.
In production, raw reasoning should rarely reach the end user. It is verbose, it can leak system-prompt content and internal deliberation, and its fluency makes wrong conclusions look considered. The pattern that works is to reason privately and present the conclusion with a short, curated rationale. Chain of thought also has clear limits: it improves reasoning, not recall, so a factual gap needs retrieval rather than more thinking.
Useful variants grew from the basic idea. Self-consistency samples several reasoning paths and returns the majority answer, lifting accuracy further at a multiple of the token cost. Tree-of-thought methods treat reasoning as search, exploring and scoring branches before committing to one. Both belong to the same family as reasoning models: strategies that trade inference-time compute for better answers, worth paying only where a wrong answer is expensive. The discipline is to match the strategy to the task rather than applying the most expensive reasoning to every request.
For agent systems the reasoning steps double as an audit surface. An evaluation harness can grade not only the final answer but whether the agent called the right tools for the right reasons, and a trace of the deliberation is usually the fastest route to understanding why a run failed. That diagnostic value is an argument for recording reasoning in logs even when it is never shown to a user — the day you need it, it cannot be reconstructed.
Codazz builds this in production — LLM Integration.