|$ curl https://forge-ai.dev/api/markdown?path=docs/ai/chain-of-thought

$cat docs/chain-of-thought-prompting.md

updated Recently·35 min read·published

Chain-of-Thought Prompting

◆AI◆Prompts◆Reasoning◆Intermediate

Introduction

Chain-of-thought (CoT) prompting is a technique that instructs large language models to produce intermediate reasoning steps before arriving at a final answer. By externalizing the reasoning process, CoT transforms the model from a black-box answer generator into a transparent reasoning system.

Introduced by Wei et al. in 2022, CoT has become one of the most impactful prompting techniques. It dramatically improves performance on arithmetic, commonsense, symbolic reasoning, and multi-step logic tasks — often doubling accuracy compared to standard prompting.

How CoT Works

Chain-of-thought exploits the autoregressive nature of LLMs. By generating intermediate tokens that represent reasoning steps, the model allocates more computation to the problem and creates a traceable path from premises to conclusion.

Approach	Trigger	Accuracy Gain	Token Overhead
Standard	Direct answer prompt	Baseline	None
Zero-Shot CoT	"Let's think step by step"	+10-30%	Low (1 sentence)
Few-Shot CoT	Examples with reasoning	+20-50%	Medium (per example)
Self-Consistency CoT	Multiple CoT paths + voting	+30-60%	High (n × tokens)

cot-comparison.txt

TEXT

1	# Standard prompt (no CoT)
2	Q: Roger has 5 tennis balls. He buys 2 more cans of tennis balls. Each can has 3 tennis balls. How many tennis balls does he have now?
3	A: 11
4
5	# Zero-shot CoT
6	Q: Roger has 5 tennis balls. He buys 2 more cans of tennis balls. Each can has 3 tennis balls. How many tennis balls does he have now?
7	A: Let's think step by step.
8	Roger starts with 5 balls.
9	He buys 2 cans × 3 balls per can = 6 balls.
10	5 + 6 = 11.
11	Therefore, Roger has 11 tennis balls.
12
13	# Few-shot CoT (with example)
14	Q: The cafeteria had 23 apples. They used 20 for lunch and bought 6 more. How many apples do they have?
15	A: They start with 23. Used 20 → 23 - 20 = 3. Bought 6 → 3 + 6 = 9. Answer: 9.
16
17	Q: Roger has 5 tennis balls. He buys 2 more cans of tennis balls. Each can has 3 tennis balls. How many tennis balls does he have now?
18	A:

ℹ

info

The magic of zero-shot CoT comes from a single phrase: "Let's think step by step." This was discovered by Kojima et al. in 2022. It works because the phrase primes the model to generate reasoning tokens before answering, effectively increasing the computational budget allocated to the problem.

Zero-Shot CoT

Zero-shot CoT requires no examples. Simply append a reasoning trigger phrase to any prompt. This is the most practical CoT variant for production because it requires minimal prompt engineering effort.

zero-shot-cot-train.txt

TEXT

1	Question: If a train travels at 60 mph for 2.5 hours, then at 80 mph for 1.5 hours, what is the total distance traveled?
2
3	Let's think step by step.
4
5	First segment: 60 mph × 2.5 hours = 150 miles
6	Second segment: 80 mph × 1.5 hours = 120 miles
7	Total distance: 150 + 120 = 270 miles
8
9	Answer: 270 miles

zero-shot-cot-discount.txt

TEXT

1	Question: A book costs $12. It is on sale for 25% off. Then an additional 10% off the sale price is applied. What is the final price?
2
3	Let's work through this.
4
5	1. Original price: $12.00
6	2. 25% off: $12.00 × 0.25 = $3.00 discount
7	3. Sale price: $12.00 - $3.00 = $9.00
8	4. Additional 10% off sale price: $9.00 × 0.10 = $0.90
9	5. Final price: $9.00 - $0.90 = $8.10
10
11	Answer: $8.10

✓

best practice

Zero-shot CoT works best with temperature=0. Higher temperatures cause the model to generate creative but incorrect reasoning chains. For tasks requiring reasoning diversity, use self-consistency instead (sample multiple zero-shot CoT chains at temperature > 0 and take majority vote).

Few-Shot CoT

Few-shot CoT provides one or more examples complete with reasoning chains. This is more effective than zero-shot CoT when the reasoning pattern is non-obvious or domain-specific.

few-shot-cot-logic.txt

TEXT

1	Solve the following logic puzzles using step-by-step reasoning.
2
3	Example:
4	There are three boxes: one contains only apples, one contains only oranges, and one contains both. All labels are wrong. You pick a fruit from the box labeled "Apples" and it is an orange. What do you know?
5	Reasoning:
6	1. The box labeled "Apples" cannot contain only apples (label is wrong).
7	2. I picked an orange from it, so it must contain both (it cannot contain only oranges because that label would be correct).
8	3. The box labeled "Oranges" cannot contain only oranges, and it cannot contain both (already found), so it contains only apples.
9	4. The box labeled "Both" contains only oranges.
10	Answer: The "Apples" box has both, "Oranges" box has apples, "Both" box has oranges.
11
12	Now solve:
13	Four people — Alice, Bob, Carol, and Dave — are sitting in a row. Alice is not at either end. Bob is to the left of Carol. Dave is to the right of Alice. Who is in the second seat from the left?
14	Reasoning:

⚠

warning

Few-shot CoT examples must be carefully curated. A single misleading or incorrect reasoning step in the examples will be amplified by the model. Always validate your CoT examples independently before including them in prompts.

Self-Consistency

Self-consistency extends CoT by sampling multiple reasoning paths and aggregating answers. The intuition: a single CoT chain might take a wrong turn, but the majority of chains will converge on the correct answer.

self-consistency-garden.txt

TEXT

1	Question: A rectangular garden is 12 meters long and 8 meters wide. A path of width 1 meter runs inside the garden along its perimeter. What is the area of the path?
2
3	Chain 1 (temperature=0.7):
4	The garden area = 12 × 8 = 96 m².
5	The inner rectangle (excluding path) = (12-2) × (8-2) = 10 × 6 = 60 m².
6	Area of path = 96 - 60 = 36 m².
7	Answer: 36 m².
8
9	Chain 2 (temperature=0.7):
10	Outer area: 12 × 8 = 96 m².
11	Path reduces length by 2m (1m each side) and width by 2m.
12	Inner dimensions: 10m × 6m = 60 m².
13	Path area: 96 - 60 = 36 m².
14	Answer: 36 m².
15
16	Chain 3 (temperature=0.7):
17	Total garden: 96 m².
18	The path runs along the perimeter. Inner garden: (12-1-1) × (8-1-1) = 10 × 6 = 60 m².
19	Path area = 96 - 60 = 36 m².
20	Answer: 36 m².
21
22	Consensus: All three chains agree on 36 m². ✓
23	Final answer: 36 m².

cot-self-consistency.py

Python

1	import openai
2	from collections import Counter
3	import re
4
5	client = openai.OpenAI()
6
7	def cot_self_consistency(question, n=5, temperature=0.5):
8	"""Run CoT with self-consistency for robust reasoning."""
9	prompt = f"{question}\n\nLet's think step by step."
10
11	responses = []
12	for _ in range(n):
13	resp = client.chat.completions.create(
14	model="gpt-4o",
15	messages=[{"role": "user", "content": prompt}],
16	temperature=temperature,
17	max_tokens=500,
18	)
19	full_text = resp.choices[0].message.content
20	responses.append(full_text)
21
22	# Extract final answers using regex
23	answers = []
24	for r in responses:
25	# Look for "Answer: X" or "Therefore, X" or final number
26	match = re.search(r'(?:Answer\|Therefore\|result\|So)[:\s]+(.+?)$',
27	r, re.MULTILINE \| re.IGNORECASE)
28	if match:
29	answers.append(match.group(1).strip())
30	else:
31	# Fallback: last sentence
32	sentences = r.strip().split('.')
33	answers.append(sentences[-1].strip())
34
35	# Majority vote
36	counter = Counter(answers)
37	final_answer = counter.most_common(1)[0][0]
38	confidence = counter.most_common(1)[0][1] / n
39
40	return {
41	"answer": final_answer,
42	"confidence": confidence,
43	"all_answers": dict(counter),
44	"traces": responses,
45	}
46
47
48	question = "A store sells shirts for $25 each. Buy 2 get 1 free. What is the average cost per shirt if you buy 6 shirts?"
49	result = cot_self_consistency(question, n=5)
50	print(f"Answer: {result['answer']} (confidence: {result['confidence']:.0%})")

🔥

pro tip

Self-consistency is particularly valuable when answers are numeric or categorical (easy to aggregate). For open-ended answers, use semantic similarity or have a separate LLM judge vote on the best response. The temperature setting for sampling should be in the 0.3-0.7 range — too low and chains will be nearly identical, too high and they become unreliable.

Advanced CoT Techniques

Beyond basic CoT, several advanced variations address specific limitations and extend applicability.

Least-to-Most Prompting

Decompose a complex problem into simpler subproblems, solve each one in order, and use previous answers as context for subsequent steps.

least-to-most.txt

TEXT

1	Question: A snail climbs 3 meters up a wall each day and slips back 2 meters each night. The wall is 20 meters high. How many days does it take to reach the top?
2
3	Step 1: How far does the snail net each full day?
4	Each day: +3m, each night: -2m. Net per full day: 1m.
5
6	Step 2: What happens on the last day?
7	On the last day, the snail reaches the top and does not slip back. So the net progress pattern changes.
8
9	Step 3: How far does the snail get before the last day?
10	If the snail needs to reach 20m, and on the last day it climbs 3m, it needs to be at 17m before the last day's climb.
11
12	Step 4: How many days to reach 17m?
13	At 1m per full day: 17 days to reach 17m.
14
15	Step 5: Add the final day.
16	On day 18, the snail climbs from 17m to 20m (3m) and reaches the top.
17
18	Answer: 18 days.

Auto-CoT

Automatically generates chain-of-thought examples by clustering questions and using the model to produce reasoning chains for representative samples.

auto-cot.py

Python

1	import openai
2	from sklearn.cluster import KMeans
3	from sentence_transformers import SentenceTransformer
4
5	client = openai.OpenAI()
6	encoder = SentenceTransformer('all-MiniLM-L6-v2')
7
8	def auto_cot(questions, k=3):
9	"""Automatically generate CoT examples from a pool of questions."""
10	# Encode questions and cluster
11	embeddings = encoder.encode(questions)
12	clusters = KMeans(n_clusters=k, random_state=0).fit(embeddings)
13
14	# Select one representative question per cluster
15	selected = []
16	for i in range(k):
17	indices = [j for j, c in enumerate(clusters.labels_) if c == i]
18	# Pick the question closest to cluster center
19	center = clusters.cluster_centers_[i]
20	closest = min(indices, key=lambda j: sum((embeddings[j] - center)**2))
21	selected.append(questions[closest])
22
23	# Generate CoT reasoning for each selected question
24	examples = []
25	for q in selected:
26	resp = client.chat.completions.create(
27	model="gpt-4o",
28	messages=[
29	{"role": "user", "content":
30	f"{q}\n\nLet's think step by step."}
31	],
32	temperature=0,
33	max_tokens=300,
34	)
35	examples.append(f"Q: {q}\nA: {resp.choices[0].message.content}")
36
37	return examples

ℹ

info

Auto-CoT is useful when you have a large pool of similar questions and need CoT examples without manual effort. The clustering ensures diversity in the selected examples, covering different reasoning patterns present in your data.

Contrastive CoT

Provides both correct and incorrect reasoning examples to teach the model what NOT to do. This is surprisingly effective at reducing specific error patterns.

contrastive-cot.txt

TEXT

1	Question: If x + 5 = 12, what is x?
2
3	Incorrect reasoning:
4	x + 5 = 12
5	x = 12 + 5 ← Wrong: should subtract 5, not add
6	x = 17
7
8	Correct reasoning:
9	x + 5 = 12
10	x = 12 - 5 ← Subtract 5 from both sides
11	x = 7
12
13	Now answer:
14	Question: If 3x - 7 = 14, what is x?
15	Reasoning:

✓

best practice

CoT is not a silver bullet. It excels at tasks that decompose into sequential steps (arithmetic, logic, planning) but may not help with tasks requiring pattern matching, intuition, or tasks where the reasoning path is not well-defined. Evaluate whether CoT actually improves your specific task before committing to it.

Limitations & Considerations

◆Token cost: CoT generates much longer outputs, increasing API costs 2-10× per query

◆Latency: More tokens means slower response times. For real-time applications, CoT may be impractical

◆CoT can produce plausible-sounding but incorrect reasoning chains (hallucinated reasoning)

◆Not all models benefit equally — CoT requires sufficient model capacity (typically 7B+ parameters)

◆Over-reasoning: for simple tasks, CoT can make the model overthink and second-guess correct answers

⚠

warning

Be cautious with CoT in user-facing applications. Showing a wrong reasoning trace erodes trust more than showing a wrong answer without explanation. Always validate the reasoning chain, not just the final answer. Consider only exposing the final answer to users and using the reasoning trace for internal debugging.

$Blueprint — Engineering Documentation·Section ID: AI-04·Revision: 1.0

X GitHub YouTube Instagram Facebook Buy me a coffee Sponsor