Back to Tracker
Phase 3 — Practical Building

Function Calling & Tool Use

The architectural primitive behind every AI agent. Give LLMs controlled, structured access to real-world actions.

Tool Schema Tool Use Loop Parallel Calls Error Handling MCP ReAct
What is Function Calling?

Function calling (also called tool use) lets an LLM request the execution of external functions. The model does not execute code — it returns a structured message saying "call this function with these arguments". Your code executes it and feeds the result back.

Key mental model: The LLM is a decision-maker, not an executor. It decides what to call and with what parameters. You decide whether to execute it and how.
Without Tool Use
  • LLM can only use training knowledge
  • No real-time data (weather, stock prices)
  • No side effects (send email, write DB)
  • No structured output enforcement
With Tool Use
  • LLM can query live APIs
  • Structured output via schema enforcement
  • Controlled side effects (sandboxed)
  • Composable: chain tools to solve complex tasks
Tool Schema Design

A tool definition has 3 parts: name (how the model calls it), description (why/when to use it — the most important field), and parameters (JSON Schema).

# OpenAI format tools = [ { "type": "function", "function": { "name": "get_weather", "description": "Get current weather for a city. Use this when the user asks about weather conditions, temperature, or forecast.", "parameters": { "type": "object", "properties": { "city": { "type": "string", "description": "City name, e.g. 'London' or 'Mumbai'" }, "unit": { "type": "string", "enum": ["celsius", "fahrenheit"], "description": "Temperature unit" } }, "required": ["city"] } } } ]
Most common mistake: a vague description like "Get weather". The model decides WHEN to call your tool based entirely on the description. Write it as if explaining to a smart intern who has never seen your codebase.

Schema Best Practices

Namessnake_case verbs: get_weather, search_db, send_email
DescriptionsExplain the why and when, not just the what
EnumsUse enum for constrained choices — prevents hallucinated values
Required fieldsOnly mark truly required fields as required
Parameter descriptionsEvery parameter needs a description with an example
Avoid too many tools>20 tools confuses models — use sub-tools or routing
Anthropic formatSame concept, slightly different key names (input_schema)
Strict modeOpenAI strict:true enforces schema at API level
The Tool Use Loop — Full Lifecycle
User message → LLM → tool_call request → execute → result → LLM → final answer
import json from openai import OpenAI client = OpenAI() messages = [{"role": "user", "content": "What's the weather in Mumbai?"}] # Round 1: model decides to call a tool response = client.chat.completions.create( model="gpt-4o-mini", messages=messages, tools=tools ) msg = response.choices[0].message if msg.tool_calls: messages.append(msg) # MUST add assistant msg to history for tc in msg.tool_calls: args = json.loads(tc.function.arguments) result = get_weather(**args) # execute the actual function messages.append({ "role": "tool", "tool_call_id": tc.id, # MUST match the call ID "content": json.dumps(result) }) # Round 2: model generates final answer using tool result response2 = client.chat.completions.create( model="gpt-4o-mini", messages=messages, tools=tools ) print(response2.choices[0].message.content)
Critical: You must add the assistant message (with tool_calls) to the messages array before adding the tool result. The conversation history must be: user → assistant(tool_call) → tool(result) → assistant(final answer).
Parallel Tool Calls — Fan-Out Pattern

The model can request multiple tools in a single response. This is the fan-out pattern — parallel execution instead of sequential.

# Model returns multiple tool_calls in one response # e.g., "What's the weather in London AND Paris?" import asyncio, json from openai import AsyncOpenAI client = AsyncOpenAI() async def handle_parallel_tools(messages): response = await client.chat.completions.create( model="gpt-4o-mini", messages=messages, tools=tools ) msg = response.choices[0].message if msg.tool_calls: messages.append(msg) # Execute ALL tool calls in parallel results = await asyncio.gather(*[ execute_tool(tc.function.name, json.loads(tc.function.arguments)) for tc in msg.tool_calls ]) # Add one tool message per call (keyed by tool_call_id) for tc, result in zip(msg.tool_calls, results): messages.append({ "role": "tool", "tool_call_id": tc.id, "content": json.dumps(result) }) # Final answer using all results final = await client.chat.completions.create( model="gpt-4o-mini", messages=messages ) return final.choices[0].message.content
When to use parallel: when tool results are independent. Never when call B depends on call A's result — those must be sequential.
Tool Error Handling
Never let a Python exception propagate into the tool result. The model reads your tool output and decides what to do next. A stack trace confuses it; a structured error lets it recover gracefully.

❌ BAD

# Crashes agent loop raise Exception("API timeout after 30s") # Confuses the LLM return {"error": "ConnectionError: HTTPSConnectionPool..."}

✅ GOOD

def get_weather(city, unit="celsius"): try: data = weather_api.get(city) return json.dumps({ "status": "ok", "temp": data["temp"], "unit": unit }) except Exception as e: return json.dumps({ "status": "error", "message": "Weather service unavailable. Try a different city." })
Interview answer: "Every tool in my agent has a try/except that returns a JSON object with status: 'error' and a human-readable message. The model reads this and can either retry with different parameters, fall back to a different tool, or tell the user gracefully."
tool_choice — Controlling When the Model Calls Tools
ValueBehaviourUse When
"auto" (default)Model decides whether to call a toolTool call is optional — model uses judgment
"required"Model MUST call at least one toolYou always want structured output via tool schema
"none"Model cannot call any tool — text onlyForce a text summary step in an agentic pipeline
{"type":"function","function":{"name":"X"}}Forces a specific toolYou know exactly which step is next
Pro tip: for data extraction workflows, tool_choice="required" combined with a single extraction tool gives you stronger schema enforcement than JSON mode — the tool schema validates at the parameter level.
Model Context Protocol (MCP) — The Universal Tool Standard

MCP is an open protocol (Anthropic, 2024) that standardises how LLM applications connect to tools and data sources. Think of it as USB-C for AI tools.

Without MCP (Ad-hoc)
  • Each agent has custom tool format
  • Tools can't be shared across agents
  • OpenAI format ≠ Anthropic format
  • Re-implement for every new AI product
With MCP
  • One server works with Claude, GPT, Cursor, etc.
  • Standard transport: stdio, SSE, or HTTP
  • Exposes: Tools, Resources, Prompts
  • Claude Code, Cursor, ChatGPT all use MCP

Build a Minimal MCP Server (Python)

from mcp import FastMCP app = FastMCP("my-tools") @app.tool() def search_docs(query: str, max_results: int = 5) -> list[dict]: """Search internal documentation. Use this when the user asks about company policies, product features, or internal processes.""" return vector_db.search(query, limit=max_results) @app.tool() def create_ticket(title: str, description: str, priority: str) -> dict: """Create a Jira ticket. Use when the user wants to log a bug or task.""" ticket_id = jira.create(title, description, priority) return {"ticket_id": ticket_id, "url": f"https://jira.co/browse/{ticket_id}"} # Run with stdio transport (for Claude Code / Cursor integration) if __name__ == "__main__": app.run(transport="stdio")
Interview angle: "We built our internal tools as an MCP server. Any agent — Claude, GPT, internal bots — can use the same tools without rewriting integrations. It also decouples our AI layer from our tool layer, making it easy to update tools independently."
Interview Cheat Sheet
finish_reason"tool_calls" = model wants to use a tool; "stop" = final answer
Must add assistant msgAlways append the assistant message before the tool result
tool_call_idEach tool result must reference the original call's ID
Parallel callsMultiple tool_calls in one response — execute with asyncio.gather
Error returnAlways JSON {"status":"error","message":"human-readable"}
Max iterationsAlways set a max_steps guard in agent loops (prevent infinite)
Tool descriptionThe most important field — write why and when, not just what
MCP vs function callingMCP = universal protocol; function calling = provider-specific API