Back to Tracker
Phase 3 — Practical Building
Structured Outputs & Data Extraction
Make LLMs return predictable, parseable data — not free-form text. The skill that turns AI from a toy into a real engineering primitive.
JSON ModePydantic
Function CallingExtraction Chains
Retry LogicResume Parser
JSON Mode — The Right Way to Get Structured Output
CRITICAL
❌ Soft instruction (unreliable)
Just saying "return JSON" in the prompt. The model may add explanation text, use single quotes, or wrap in markdown code fences. json.loads() will randomly fail.
✅ JSON Mode (reliable)
response_format={"type": "json_object"} is enforced by the API — the model is constrained to output valid JSON. json.loads() will always succeed.
import json
response = client.chat.completions.create(
model="gpt-4o-mini",
messages=[
{
"role": "system",
"content": """Extract the following from the text and return as JSON:
{"name": "string", "email": "string or null", "skills": ["list", "of", "strings"]}
Rules: use null for missing fields. Return ONLY the JSON object."""
},
{"role": "user", "content": raw_text}
],
response_format={"type": "json_object"}, # ← hard constraint, always valid
temperature=0.0, # ← deterministic extraction
)
data = json.loads(response.choices[0].message.content) # always safe
Important: You MUST instruct the model to return JSON in the prompt (mention "JSON" or "json"). If you don't, the API raises a BadRequestError. The response_format parameter enforces format; the prompt tells the model what structure to use.
Pydantic Validation — Type Safety for LLM Output
from pydantic import BaseModel, Field
from typing import Optional
class WorkExperience(BaseModel):
company: str
title: str
start_date: Optional[str] = None
end_date: Optional[str] = None
responsibilities: list[str] = Field(default_factory=list)
class Resume(BaseModel):
name: str # required — missing = ValidationError
email: Optional[str] = None # missing = None, no error
phone: Optional[str] = None
skills: list[str] = Field(default_factory=list) # missing = [], no error
work_experience: list[WorkExperience] = Field(default_factory=list)
education: list[str] = Field(default_factory=list)
# Parse LLM output into the model
data = json.loads(response.choices[0].message.content)
resume = Resume(**data) # ValidationError if required fields missing or wrong types
# Access with type safety
print(resume.name) # str, guaranteed
print(resume.email) # str | None
print(resume.skills) # list[str]
for job in resume.work_experience:
print(f"{job.title} at {job.company}")
Design rule: Make fields Optional[...] = None for anything that may not appear in all documents. Only use required fields (no default) when you are 100% certain the data will always be present.
Pydantic Validators — Handle LLM Inconsistencies
from pydantic import BaseModel, Field, field_validator
class ExtractedData(BaseModel):
# Constrained float — ValidationError if outside range
confidence_score: float = Field(ge=0.0, le=1.0)
# Non-empty string
name: str = Field(min_length=1)
# List with at least 1 item
skills: list[str] = Field(min_length=1)
# Custom validator — normalize "95" → "0.95" (model sometimes returns percentages)
@field_validator("confidence_score", mode="before")
@classmethod
def normalize_confidence(cls, v):
if isinstance(v, (int, float)) and v > 1.0:
v = v / 100.0 # 95 → 0.95
if not 0.0 <= v <= 1.0:
raise ValueError(f"confidence {v} not in [0, 1] after normalization")
return round(v, 4)
# Normalize string input before type validation
@field_validator("name", mode="before")
@classmethod
def clean_name(cls, v):
if isinstance(v, str):
return v.strip()
return v
mode="before" runs your validator BEFORE Pydantic's type coercion — use this to normalize values in unexpected formats (currency strings, percentages, etc.) before validation fails.
Retry with Error Feedback — 3-Layer Reliability
When Pydantic validation fails, feed the error back to the model and ask it to fix its output:
from pydantic import ValidationError
import json
def extract_with_retry(text: str, max_retries: int = 2) -> Resume:
messages = [
{"role": "system", "content": EXTRACTION_PROMPT},
{"role": "user", "content": text}
]
for attempt in range(max_retries):
response = client.chat.completions.create(
model="gpt-4o-mini",
messages=messages,
response_format={"type": "json_object"},
temperature=0.0
)
raw_json = response.choices[0].message.content
try:
return Resume(**json.loads(raw_json))
except (json.JSONDecodeError, ValidationError) as e:
if attempt < max_retries - 1:
# Feed error back — model corrects itself ~95% of the time
messages.append({"role": "assistant", "content": raw_json})
messages.append({
"role": "user",
"content": f"Validation failed:\n{str(e)}\n\nPlease fix the JSON and return only the corrected JSON object."
})
else:
raise ValueError(f"Extraction failed after {max_retries} attempts: {e}")
3-Layer Reliability:
1. JSON Mode — guarantees syntactically valid JSON (eliminates JSONDecodeError)
2. Pydantic — guarantees correct structure and types (catches semantic errors)
3. Retry with feedback — model corrects its own mistakes (~95% success on retry)
Extraction Prompt Template
EXTRACTION_PROMPT = """You are a precise data extractor. Extract information from the provided text.
Return JSON with EXACTLY this structure:
{
"name": "full name or null",
"email": "email address or null",
"phone": "phone number or null",
"skills": ["skill1", "skill2"],
"work_experience": [
{
"company": "company name",
"title": "job title",
"start_date": "YYYY-MM or null",
"end_date": "YYYY-MM or null (use null for current role)",
"responsibilities": ["bullet1", "bullet2"]
}
]
}
Rules:
- Use null for any field not found in the text (NOT empty string, NOT "N/A")
- Dates in YYYY-MM format (e.g., "2022-03") or null
- All monetary values as numbers (float), not strings. Remove $ and commas.
- confidence_score as float 0.0-1.0 (NOT a percentage like 95)
- Return ONLY the JSON object, no explanation text
"""
Function Calling vs JSON Mode — When to Use Which
| Feature | JSON Mode | Function Calling |
|---|
| Use when | You need one structured output from text | You have multiple possible actions the model can take |
| Schema enforcement | None (Pydantic validates after) | OpenAI validates against function JSON schema |
| Complexity | Simple to implement | More setup, more powerful |
| Best for | Extraction, classification, parsing | Agents that call real functions (search, DB, APIs) |
Rule: Use JSON Mode for extraction tasks. Use Function Calling when building agents that need to decide which action to take from a set of options.
ALWAYS USE
response_format={"type": "json_object"}
temperature=0.0 # for extraction
# Mention JSON in system prompt
PYDANTIC PATTERN
class MyModel(BaseModel):
required: str
optional: Optional[str] = None
items: list[str] = Field(
default_factory=list)
data = MyModel(**json.loads(raw))
# ValidationError if structure wrong
RETRY PATTERN
for attempt in range(2):
raw = llm_call(messages)
try:
return MyModel(**json.loads(raw))
except ValidationError as e:
messages += [
{"role":"assistant","content":raw},
{"role":"user","content":
f"Fix this error: {e}"}
]
raise ValueError("Extraction failed")
VALIDATOR PATTERN
@field_validator("score", mode="before")
@classmethod
def normalize(cls, v):
if isinstance(v, (int, float)) and v > 1:
v = v / 100.0 # 95 → 0.95
return v