Chain-of-Thought (CoT)
Solving complex problems through step-by-step reasoning
Chain-of-Thought (CoT) is a prompting technique proposed by Google Brain in 2022. Previously, LLMs directly generated questionβanswer, but CoT makes them explicitly generate intermediate reasoning steps. Adding just "Let's think step by step" (Zero-shot CoT) improved math problem accuracy from 17% to 78%. Few-shot CoT provides examples with reasoning processes for more accurate reasoning. This technique became the foundation of modern LLM "reasoning" capabilities and evolved into OpenAI's o1/o3 and Claude's Extended Thinking.
Key Concepts
Present the problem β complex math/logic/coding problem
Add CoT prompt β "think step by step" or provide reasoning examples
LLM generates intermediate reasoning steps as text ("First, finding A..., next B is...")
Each step's result feeds into the next step
Derive final answer
Verification: Self-Consistency β reason multiple times and select final answer by majority vote (optional)
Pros
- ✓ Dramatic accuracy improvement on complex problems
- ✓ Transparent reasoning process β debuggable
- ✓ Applied via prompt only (no model modification needed)
- ✓ Maximized effect in large models (emergent ability)
Cons
- ✗ Additional reasoning tokens increase cost/latency
- ✗ Minimal effect on small models
- ✗ Incorrect reasoning steps can lead to wrong answers
- ✗ Unnecessary overhead for simple problems