|$ curl https://forge-ai.dev/api/markdown?path=docs/ai/function-calling

$cat docs/ai-engineering-—-function-calling.md

updated Recently·35 min read·published

AI Engineering — Function Calling

◆AI Engineering◆APIs◆Intermediate to Advanced

Introduction

Function calling (also called tool use) is the ability for large language models to generate structured JSON that invokes external functions or APIs. Rather than producing text that describes an action, the model outputs a machine-readable schema that your application can parse and execute.

This capability transforms LLMs from stateless text generators into reasoning engines that can query databases, call APIs, perform calculations, and take actions in the real world. Function calling is the backbone of modern agent architectures.

Tool Definition Schema

Tools are defined using JSON Schema. Each tool has a name, description, and parameter schema. The description is critical — the model uses it to decide when and how to call the tool. Be specific about what the tool does, when it should be used, and what each parameter does.

OpenAI Tool Format

tools-definition.ts

TypeScript

1	// Tool definition using the OpenAI SDK format
2	const tools = [
3	{
4	type: "function",
5	function: {
6	name: "get_weather",
7	description: "Get current weather for a location. " +
8	"Use this when the user asks about weather, temperature, " +
9	"or climate conditions for a specific city or region.",
10	parameters: {
11	type: "object",
12	properties: {
13	location: {
14	type: "string",
15	description: "City name, e.g. 'San Francisco, CA'"
16	},
17	units: {
18	type: "string",
19	enum: ["celsius", "fahrenheit"],
20	default: "celsius",
21	description: "Temperature unit"
22	}
23	},
24	required: ["location"],
25	additionalProperties: false
26	}
27	}
28	},
29	{
30	type: "function",
31	function: {
32	name: "search_database",
33	description: "Search the internal knowledge base for " +
34	"documentation, code examples, or technical references.",
35	parameters: {
36	type: "object",
37	properties: {
38	query: {
39	type: "string",
40	description: "Search query string"
41	},
42	max_results: {
43	type: "integer",
44	default: 5,
45	description: "Maximum results to return"
46	}
47	},
48	required: ["query"],
49	additionalProperties: false
50	}
51	}
52	}
53	];

ℹ

info

Write descriptions as if instructing the model. A good description is the difference between the model calling a tool correctly and never invoking it. Include edge cases: e.g. "Use this for current weather only, not forecasts".

Making a Function Call

The flow is: send a message with tools defined, receive a response with a tool_calls array, execute the function locally, and send the result back. This is called the tool use loop.

function-call.ts

TypeScript

1	import OpenAI from "openai";
2
3	const openai = new OpenAI();
4
5	// Step 1: Send message with tools
6	const response = await openai.chat.completions.create({
7	model: "gpt-4o",
8	messages: [
9	{ role: "user", content: "What's the weather in Tokyo?" }
10	],
11	tools: weatherTools,
12	tool_choice: "auto"
13	});
14
15	// Step 2: Check for tool calls
16	const message = response.choices[0].message;
17
18	if (message.tool_calls) {
19	for (const call of message.tool_calls) {
20	const args = JSON.parse(call.function.arguments);
21
22	// Step 3: Execute the function locally
23	const result = await executeFunction(
24	call.function.name,
25	args
26	);
27
28	// Step 4: Send result back to the model
29	messages.push({
30	role: "tool",
31	tool_call_id: call.id,
32	content: JSON.stringify(result)
33	});
34	}
35
36	// Step 5: Get final response
37	const final = await openai.chat.completions.create({
38	model: "gpt-4o",
39	messages
40	});
41
42	console.log(final.choices[0].message.content);
43	}

The model may call multiple tools in a single response, or request no tool call at all. Always check message.tool_calls before assuming the response contains text content.

Parallel Function Calling

Modern LLMs can call multiple functions in a single response turn. The model identifies independent operations that can run concurrently and issues all tool calls at once. This dramatically reduces latency for multi-step operations.

parallel-calls.ts

TypeScript

1	// The model may return multiple tool_calls
2	const response = await openai.chat.completions.create({
3	model: "gpt-4o",
4	messages: [
5	{ role: "user", content:
6	"Compare weather in Tokyo, London, and NYC" }
7	],
8	tools: [weatherTool],
9	tool_choice: "auto"
10	});
11
12	// Execute all calls in parallel
13	const results = await Promise.all(
14	response.choices[0].message.tool_calls.map(async (call) => {
15	const args = JSON.parse(call.function.arguments);
16	const result = await getWeather(args.location, args.units);
17	return {
18	tool_call_id: call.id,
19	role: "tool" as const,
20	content: JSON.stringify(result)
21	};
22	})
23	);
24
25	// Send all results back
26	messages.push(
27	response.choices[0].message,
28	...results
29	);
30
31	const final = await openai.chat.completions.create({
32	model: "gpt-4o",
33	messages
34	});

✓

best practice

Always execute parallel tool calls concurrently with Promise.all. The model expects you to run them independently. Running them serially defeats the purpose and increases response latency unnecessarily.

Forced Function Calling

You can force the model to call a specific function by setting tool_choice to "required" or by specifying the exact function name. This is useful for classification, extraction, and routing tasks where you always need structured output.

tool_choice	Behavior	Use Case
"auto"	Model decides if/when to call tools	General conversation
"required"	Model must call at least one tool	Extraction, classification
"none"	Model cannot call any tool	Plain text responses only
{type:"function",function:{name:"..."}}	Model must call the specified function	Structured extraction, routing

forced-calling.ts

TypeScript

1	// Force the model to call get_weather
2	const response = await openai.chat.completions.create({
3	model: "gpt-4o",
4	messages: [
5	{ role: "user", content: "Do I need an umbrella today?" }
6	],
7	tools: [weatherTool],
8	tool_choice: {
9	type: "function",
10	function: { name: "get_weather" }
11	}
12	});
13
14	// Force the model to call any tool
15	const extraction = await openai.chat.completions.create({
16	model: "gpt-4o",
17	messages: [
18	{ role: "user", content:
19	"Extract: Name: John, Age: 30, City: Boston" }
20	],
21	tools: [extractPersonTool],
22	tool_choice: "required"
23	});

⚠

warning

Forcing a function call with tool_choice: "required" will make the model hallucinate arguments if the user's query lacks the needed information. Provide clear descriptions for each parameter so the model can ask clarifying questions or use sensible defaults when appropriate.

Streaming with Tools

When streaming, tool call deltas arrive in chunks. You must accumulate the delta to reconstruct the full tool call. The tool_calls arrive as an array with incrementally populated arguments.

streaming-tools.ts

TypeScript

1	import OpenAI from "openai";
2
3	const stream = await openai.chat.completions.create({
4	model: "gpt-4o",
5	messages: messages,
6	tools: tools,
7	stream: true
8	});
9
10	// Accumulator for tool calls
11	const toolCalls = new Map();
12
13	for await (const chunk of stream) {
14	const delta = chunk.choices[0]?.delta;
15
16	if (delta?.tool_calls) {
17	for (const call of delta.tool_calls) {
18	const idx = call.index;
19
20	if (!toolCalls.has(idx)) {
21	toolCalls.set(idx, {
22	id: call.id \|\| '',
23	function: {
24	name: call.function?.name \|\| '',
25	arguments: call.function?.arguments \|\| ''
26	}
27	});
28	} else {
29	const existing = toolCalls.get(idx);
30	if (call.id) existing.id = call.id;
31	if (call.function?.name) {
32	existing.function.name += call.function.name;
33	}
34	if (call.function?.arguments) {
35	existing.function.arguments += call.function.arguments;
36	}
37	}
38	}
39	}
40
41	// Handle text content
42	if (delta?.content) {
43	process.stdout.write(delta.content);
44	}
45	}
46
47	// After stream completes, execute tool calls
48	for (const call of toolCalls.values()) {
49	const args = JSON.parse(call.function.arguments);
50	const result = await executeTool(call.function.name, args);
51	messages.push({
52	role: "tool",
53	tool_call_id: call.id,
54	content: JSON.stringify(result)
55	});
56	}

🔥

pro tip

Use a Map keyed by call.index to accumulate streaming tool calls. The index ensures you correctly merge chunks even when multiple tools are called in parallel. Always wait for the stream to finish before executing tools.

Error Handling

Function execution can fail. Return errors as structured tool responses so the model can decide how to handle them — retry, ask for clarification, or inform the user of the failure.

error-handling.ts

TypeScript

1	// Return errors as tool responses
2	async function executeTool(name: string, args: any) {
3	try {
4	switch (name) {
5	case "get_weather":
6	return await fetchWeather(args.location, args.units);
7	case "search_database":
8	return await searchDB(args.query, args.max_results);
9	default:
10	return { error: `Unknown tool: ${name}` };
11	}
12	} catch (err) {
13	// Structured error response
14	return {
15	error: err instanceof Error ? err.message : "Unknown error",
16	code: "TOOL_EXECUTION_FAILED",
17	retryable: true,
18	timestamp: new Date().toISOString()
19	};
20	}
21	}
22
23	// Handle invalid JSON from model
24	function safeParseArguments(raw: string) {
25	try {
26	return JSON.parse(raw);
27	} catch {
28	return {
29	error: "Failed to parse function arguments",
30	raw: raw,
31	code: "INVALID_ARGUMENTS"
32	};
33	}
34	}
35
36	// Timeout protection for slow tools
37	async function executeWithTimeout(
38	fn: () => Promise<any>,
39	timeoutMs: number = 10000
40	) {
41	const timeout = new Promise((_, reject) =>
42	setTimeout(() => reject(new Error("Tool timed out")), timeoutMs)
43	);
44	return Promise.race([fn(), timeout]);
45	}

⚠

warning

Always implement timeouts for tool execution. A stuck API call can block the entire agent loop indefinitely. Set a reasonable timeout (5-30s depending on the tool) and return a timeout error to the model so it can gracefully handle the failure.

Practical Patterns

Tool Chaining

Some workflows require the output of one tool to be the input of another. The model handles this naturally across multiple turns — it calls tool A, receives the result, then calls tool B with that data.

tool-chaining.ts

TypeScript

1	// Multi-turn tool chaining
2	// Turn 1: Model calls search_database
3	// Turn 2: Model receives search results
4	// Turn 3: Model calls summarize with search results
5	// Turn 4: Model returns summary to user
6
7	const calls: ToolCall[] = [];
8
9	while (needsMoreTools(response)) {
10	for (const call of response.tool_calls) {
11	const result = await executeTool(call);
12	calls.push({ id: call.id, result });
13	}
14
15	response = await openai.chat.completions.create({
16	model: "gpt-4o",
17	messages: [...messages, response, ...calls.map(toToolMsg)]
18	});
19	}

Structured Extraction with Tools

Use forced function calling for reliable structured data extraction from unstructured text. This is more reliable than prompt-based JSON generation.

extraction-tool.ts

TypeScript

1	const extractTool = {
2	type: "function",
3	function: {
4	name: "extract_entity",
5	description: "Extract structured entities from text",
6	parameters: {
7	type: "object",
8	properties: {
9	entities: {
10	type: "array",
11	items: {
12	type: "object",
13	properties: {
14	name: { type: "string" },
15	type: { type: "string", enum: ["person", "org", "date", "product"] },
16	confidence: { type: "number", minimum: 0, maximum: 1 }
17	},
18	required: ["name", "type"]
19	}
20	}
21	},
22	required: ["entities"]
23	}
24	}
25	};

$Blueprint — Engineering Documentation·Section ID: AI-FC-01·Revision: 1.0

X GitHub YouTube Instagram Facebook Buy me a coffee Sponsor