What is Chain-of-Thought?
Chain-of-Thought (CoT) is a prompting technique that elicits intermediate natural-language steps before a model produces its final answer. Those generated steps may help on some multi-step tasks, but they are not guaranteed to be correct or to faithfully reveal the model's internal computation.
Quick Facts
| Full Name | Chain-of-Thought Prompting |
|---|---|
| Created | 2022 by Google Research |
| Specification | Official Specification |
How It Works
Wei et al. formalized few-shot Chain-of-Thought prompting in 2022 by placing worked rationales between each example input and answer. Kojima et al. then showed that a short zero-shot instruction could elicit intermediate steps on several evaluated models and benchmarks. These studies established task- and model-specific empirical gains, not a universal prompt rule or a fixed parameter threshold. A visible rationale is generated text: it can contain invalid steps, omit factors that changed the answer, or rationalize a conclusion after the fact. Modern reasoning models may also use provider-managed inference compute that is distinct from a user-requested CoT response. Production systems should compare CoT with a direct baseline, verify calculations or claims externally, avoid storing sensitive rationales by default, and measure correctness, robustness, latency, token cost, and abstention separately.
Key Characteristics
- Generates intermediate natural-language or symbolic text before a final answer
- Supports few-shot demonstrations and zero-shot elicitation variants
- Produces gains that depend on the model, task, prompt, and decoding settings
- Consumes additional output tokens, latency, and sometimes multiple samples
- May produce incorrect or unfaithful rationales even when the final answer is correct
- Requires external evidence, tools, tests, or verifiers for trustworthy decisions
Common Use Cases
- Ablation testing on multi-step tasks with exact or human-reviewed labels
- Arithmetic workflows where a calculator independently verifies the result
- Symbolic reasoning and code tasks with executable checks or formal constraints
- Few-shot demonstrations for stable task formats and domain-specific procedures
- Concise, evidence-backed explanations when users need reasons rather than hidden reasoning
Example
Loading code...Frequently Asked Questions
What is the difference between zero-shot and few-shot chain-of-thought?
Zero-shot CoT adds an instruction that elicits intermediate steps without demonstrations. Few-shot CoT includes worked input-rationale-answer examples that define a task pattern. Neither variant is universally better: compare both with a direct prompt on the same model version, labeled task set, decoding settings, latency, and token budget.
Why does chain-of-thought improve model accuracy?
CoT can provide additional generated tokens that preserve partial results and condition later tokens, and demonstrations can steer the model toward a useful solution format. That is an engineering hypothesis, not proof of human-like thought. Gains vary by model and task, and longer rationales can also introduce or conceal errors.
Is visible chain-of-thought the model's true internal reasoning?
Not necessarily. A visible rationale is another generated output channel. Faithfulness studies show that models can use information without reporting it and can produce plausible explanations that do not causally match the answer. Treat the rationale as an explanation candidate, not as an authorization record or proof of correctness.
Does chain-of-thought require a 100-billion-parameter model?
No universal parameter threshold exists. The original 2022 experiments observed stronger gains at larger scales in the evaluated model families, while later models changed through training data, instruction tuning, distillation, and reasoning-specific training. Test the exact model and endpoint instead of using parameter count as a proxy.
How should chain-of-thought prompting be evaluated?
Run an ablation that compares direct answers, decomposition, few-shot CoT, zero-shot CoT, and tool-verified variants on a versioned task set. Measure final-answer correctness, step validity where labels exist, robustness, abstention, p50 and p95 latency, token cost, privacy exposure, and cost per correct outcome.