Reasoning & Planning

◆AI◆Advanced

Introduction

Reasoning and planning are frontier capabilities in LLM research. While standard autoregressive models are effective at pattern matching and retrieval, they struggle with multi-step logical deduction, constrained planning, and self-correction. A family of techniques — broadly called "reasoning frameworks" — addresses these limitations by structuring the model's generation process.

These techniques range from simple prompting strategies (chain-of-thought) to complex search algorithms over token sequences (tree-of-thought, graph-of-thought) to agentic loops with tool use and self-reflection (ReAct, reflexion). The unifying theme is that they allocate more computation at test time to improve reasoning quality, a paradigm called test-time compute scaling.

This guide covers the major reasoning approaches with practical implementations, performance characteristics, and guidance on when to use each technique.

Chain-of-Thought (CoT)

Chain-of-Thought prompting instructs the model to produce intermediate reasoning steps before arriving at a final answer. By externalizing the reasoning process, CoT dramatically improves performance on arithmetic, logical, and symbolic reasoning tasks. For models of sufficient capability, CoT can be elicited simply by adding "Let's think step by step" to the prompt (zero-shot CoT).

chain-of-thought.py

Python

1	from openai import OpenAI
2
3	client = OpenAI()
4
5	def chain_of_thought(prompt: str) -> str:
6	response = client.chat.completions.create(
7	model="gpt-4o",
8	messages=[
9	{
10	"role": "user",
11	"content": f"{prompt}\n\nLet's think step by step."
12	}
13	],
14	temperature=0.0,
15	max_tokens=1024
16	)
17	return response.choices[0].message.content
18
19	# Few-shot CoT with examples
20	FEW_SHOT_COT = """Q: Roger has 5 tennis balls. He buys 2 more cans of
21	tennis balls. Each can has 3 tennis balls. How many tennis balls does
22	he have now?
23	A: Roger started with 5 balls. 2 cans of 3 balls each is 6 balls.
24	5 + 6 = 11. The answer is 11.
25
26	Q: The cafeteria had 23 apples. They used 20 to make lunch and bought
27	6 more. How many apples do they have?
28	A: The cafeteria had 23 apples. They used 20, so 23 - 20 = 3.
29	They bought 6 more, so 3 + 6 = 9. The answer is 9.
30
31	Q: {question}
32	A:"""
33
34	def few_shot_cot(question: str) -> str:
35	response = client.chat.completions.create(
36	model="gpt-4o",
37	messages=[
38	{"role": "user", "content": FEW_SHOT_COT.format(question=question)}
39	],
40	temperature=0.0
41	)
42	return response.choices[0].message.content

Self-Consistency

Self-consistency improves CoT by sampling multiple reasoning paths and selecting the most common answer through majority voting. This technique is simple but effective — it reduces the variance of CoT and corrects for individual reasoning errors.

self-consistency.py

Python

1	from collections import Counter
2
3	def self_consistency(prompt: str, num_paths: int = 5) -> tuple:
4	responses = []
5	for _ in range(num_paths):
6	response = chain_of_thought(prompt)
7	responses.append(response)
8
9	# Extract final answers (assumes "The answer is X" format)
10	answers = []
11	for resp in responses:
12	if "The answer is" in resp:
13	answer = resp.split("The answer is")[-1].strip().rstrip(".")
14	answers.append(answer)
15
16	# Majority vote
17	counter = Counter(answers)
18	most_common = counter.most_common(1)[0]
19	return most_common[0], dict(counter)
20
21	# Usage
22	answer, distribution = self_consistency(
23	"A bat and a ball cost $1.10. The bat costs $1.00 more than "
24	"the ball. How much does the ball cost?",
25	num_paths=5
26	)
27	print(f"Final answer: {answer}")
28	print(f"Distribution: {distribution}")

✓

best practice

Self-consistency with 5-10 samples typically saturates in benefit — more samples yield diminishing returns. Use temperature=0.5-0.7 for sampling diverse reasoning paths. For math and logic problems, extract the final answer using structured output parsing rather than relying on "The answer is" patterns.

Tree-of-Thought (ToT)

Tree-of-Thought extends CoT by maintaining multiple parallel reasoning branches, evaluating each branch's progress, and pruning unpromising paths. The model generates several possible next steps, evaluates them, and explores the most promising ones while discarding dead ends. This is analogous to a tree search over reasoning steps.

tree-of-thought.py

Python

1	from typing import List, Optional
2
3	class TreeOfThought:
4	def __init__(self, max_branches: int = 3, max_depth: int = 5):
5	self.max_branches = max_branches
6	self.max_depth = max_depth
7
8	def solve(self, problem: str) -> str:
9	# BFS over reasoning tree
10	frontier = [{"path": [], "thoughts": [], "state": problem}]
11	best_solution = None
12
13	for depth in range(self.max_depth):
14	if not frontier:
15	break
16	new_frontier = []
17	for node in frontier:
18	# Generate candidate next steps
19	candidates = self._generate_candidates(node["state"])
20	# Evaluate each candidate
21	evaluated = self._evaluate_candidates(candidates, problem)
22	# Keep top-k
23	evaluated.sort(key=lambda x: x["score"], reverse=True)
24	for candidate in evaluated[:self.max_branches]:
25	new_path = node["path"] + [candidate["thought"]]
26	if candidate["is_solution"]:
27	return "\n".join(new_path)
28	new_frontier.append({
29	"path": new_path,
30	"thoughts": node["thoughts"] + [candidate["thought"]],
31	"state": candidate["next_state"]
32	})
33	frontier = new_frontier
34	return best_solution or "Solution not found"
35
36	def _generate_candidates(self, state: str) -> List[dict]:
37	prompt = f"""Current reasoning state: {state}
38	Generate {self.max_branches} different possible next steps.
39	Each step should make progress toward solving the problem.
40	Return as a numbered list."""
41	response = client.chat.completions.create(
42	model="gpt-4o",
43	messages=[{"role": "user", "content": prompt}],
44	temperature=0.8
45	)
46	# Parse candidates from response
47	return self._parse_candidates(response.choices[0].message.content)
48
49	def _evaluate_candidates(self, candidates: List[dict], problem: str) -> List[dict]:
50	for c in candidates:
51	eval_prompt = f"""Problem: {problem}
52	Proposed next step: {c["thought"]}
53	Evaluate if this step: (1) makes progress, (2) is logically sound.
54	Score from 1-10. If this reaches a solution, mark is_solution=true."""
55	eval_response = client.chat.completions.create(
56	model="gpt-4o",
57	messages=[{"role": "user", "content": eval_prompt}],
58	temperature=0.2
59	)
60	c["score"] = self._parse_score(eval_response.choices[0].message.content)
61	c["is_solution"] = "is_solution" in eval_response.choices[0].message.content.lower()
62	return candidates

ℹ

info

Tree-of-Thought is most valuable for problems that benefit from exploration and backtracking — puzzles, creative writing, constrained optimization. For straightforward Q&A, CoT with self-consistency is cheaper and equally effective. ToT's overhead (multiple LLM calls per depth level) makes it 5-20x more expensive than CoT, so use it selectively.

ReAct — Reasoning + Acting

ReAct (Reasoning + Acting) interleaves reasoning traces with tool-use actions. The model iteratively: (1) thinks about what to do next, (2) performs an action (search, calculation, API call), (3) observes the result, and (4) continues reasoning. This enables LLMs to gather external information, compute precise results, and verify their reasoning against real-world data.

react-agent.py

Python

1	import json
2	from typing import Callable
3
4	class ReActAgent:
5	def __init__(self, tools: dict[str, Callable]):
6	self.tools = tools
7	self.max_iterations = 10
8
9	def run(self, task: str) -> str:
10	messages = [
11	{"role": "system", "content": self._system_prompt()},
12	{"role": "user", "content": task}
13	]
14
15	for step in range(self.max_iterations):
16	response = client.chat.completions.create(
17	model="gpt-4o",
18	messages=messages,
19	temperature=0.0
20	)
21	content = response.choices[0].message.content
22	messages.append({"role": "assistant", "content": content})
23
24	# Check for final answer
25	if "FINAL ANSWER:" in content:
26	return content.split("FINAL ANSWER:")[-1].strip()
27
28	# Parse and execute action
29	action = self._parse_action(content)
30	if action:
31	tool_name = action["tool"]
32	tool_input = action["input"]
33	if tool_name in self.tools:
34	observation = self.tools[tool_name](tool_input)
35	messages.append({
36	"role": "user",
37	"content": f"Observation: {json.dumps(observation)}"
38	})
39
40	return "Max iterations reached without final answer."
41
42	def _system_prompt(self) -> str:
43	tools_desc = "\n".join([
44	f"- {name}: {fn.__doc__}"
45	for name, fn in self.tools.items()
46	])
47	return f"""You are a reasoning agent. You can use these tools:
48	{tools_desc}
49
50	For each step, respond with:
51	THOUGHT: Your reasoning about what to do next
52	ACTION: {{"tool": "tool_name", "input": "input"}}
53	After receiving an observation, continue reasoning.
54	When you have the answer, respond with: FINAL ANSWER: [your answer]"""
55
56	def _parse_action(self, text: str) -> dict \| None:
57	if "ACTION:" in text:
58	action_text = text.split("ACTION:")[-1].strip()
59	try:
60	return json.loads(action_text)
61	except json.JSONDecodeError:
62	return None
63	return None
64
65	# Example: Agent with search and calculator
66	def search_tool(query: str) -> str:
67	"""Search the web for information"""
68	return f"Search results for: {query}"
69
70	def calculator_tool(expr: str) -> str:
71	"""Evaluate a mathematical expression"""
72	return str(eval(expr))
73
74	agent = ReActAgent({
75	"search": search_tool,
76	"calculator": calculator_tool
77	})
78	result = agent.run("What is the population of Japan divided by 2?")
79	print(result)

Reflexion & Self-Verification

Reflexion equips the agent with a memory of past failures that it explicitly reflects on before attempting a task again. The agent stores feedback from previous attempts (including errors, missing information, or incorrect reasoning) in a semantic memory and uses this context to improve subsequent attempts. This turns a single-attempt process into an iterative improvement loop.

reflexion.py

Python

1	class ReflexionAgent:
2	def __init__(self, tools: dict):
3	self.tools = tools
4	self.memory = [] # List of previous attempts and feedback
5
6	def run(self, task: str, max_attempts: int = 3) -> str:
7	for attempt in range(max_attempts):
8	context = self._build_context(task)
9	response = client.chat.completions.create(
10	model="gpt-4o",
11	messages=context,
12	temperature=0.3
13	)
14	result = response.choices[0].message.content
15
16	# Self-verify the result
17	verification = self._verify(task, result)
18
19	if verification["is_correct"]:
20	return result
21	else:
22	self.memory.append({
23	"attempt": attempt,
24	"result": result,
25	"feedback": verification["feedback"]
26	})
27
28	return self._best_result()
29
30	def _build_context(self, task: str) -> list:
31	messages = [{"role": "user", "content": task}]
32	if self.memory:
33	reflections = "\n".join([
34	f"Previous attempt {m['attempt'] + 1}: {m['result']}\n"
35	f"Feedback: {m['feedback']}"
36	for m in self.memory
37	])
38	messages.append({
39	"role": "user",
40	"content": f"Before answering, consider these past attempts:\n{reflections}"
41	})
42	return [{"role": "system", "content": self._system_prompt()}] + messages
43
44	def _verify(self, task: str, result: str) -> dict:
45	verify_prompt = f"""Task: {task}
46	Proposed solution: {result}
47	Verify this solution. Is it correct? If not, explain what's wrong."""
48	response = client.chat.completions.create(
49	model="gpt-4o",
50	messages=[{"role": "user", "content": verify_prompt}],
51	temperature=0.0
52	)
53	content = response.choices[0].message.content
54	return {
55	"is_correct": "correct" in content.lower() and "not correct" not in content.lower(),
56	"feedback": content
57	}

✓

best practice

Reflexion is most effective when combined with structured verification. Instead of asking "Is this correct?", provide specific criteria: check arithmetic, verify all constraints, confirm no missing steps. The self-verification step itself can benefit from CoT — ask the model to reason through the verification before declaring correctness.

Test-Time Compute Scaling

Test-time compute scaling is the observation that allocating more computation at inference time — through longer reasoning chains, more candidate samples, or deeper search — can dramatically improve model performance on difficult problems. This creates a new dimension for model improvement beyond training.

Technique	Compute Overhead	Accuracy Gain	Best For
CoT	1-2x	+10-30%	Math, logic, multi-step
Self-consistency	5-10x	+5-15% over CoT	High-variance tasks
ToT	10-50x	+10-25% over CoT	Puzzles, creative, planning
Reflexion	3-5x per attempt	+5-20% per iteration	Coding, debugging, revision

⚠

warning

Test-time compute scaling has diminishing returns — the first 2-3 reasoning samples capture most of the gain. Beyond that, the compute-to-accuracy ratio degrades. Set a compute budget (max tokens, max iterations, max API calls) and stop when the budget is exhausted. Never let reasoning loops run unboundedly.

$Blueprint — Engineering Documentation·Section ID: AI-REASON-01·Revision: 1.0

X GitHub YouTube Instagram Facebook Buy me a coffee Sponsor