Personal Wiki

Structured Output (LangChain Agents)

4 min read

Structured Output (LangChain Agents)

Abstract

A LangChain reference covering structured output for agents using create_agent — the mechanism by which agents return data in validated, typed format (Pydantic models, dataclasses, TypedDicts, JSON schema) rather than unstructured natural language. LangChain exposes two strategies: ProviderStrategy uses native API-level structured output for supported providers (OpenAI, Anthropic/Claude, xAI/Grok, Gemini) — the most reliable approach when available; ToolStrategy uses tool-calling to achieve the same result for any model that supports tool calling. When a schema type is passed directly to response_format, LangChain auto-selects the strategy from the model’s capability profile (requires langchain>=1.1). The validated response appears in the structured_response key of the agent’s final state. ToolStrategy includes configurable error handling for validation failures — automatic retry with default or custom error messages, per-exception-type filtering, and custom error handler functions. Union types allow the model to select from multiple output schemas. Production-critical failure modes (multiple simultaneous structured outputs, schema validation errors) are handled with targeted ToolMessage feedback and retry prompting.

Key Concepts

  • ProviderStrategy: Structured output enforced at the model API level; requires provider support (OpenAI, Anthropic/Claude, xAI/Grok, Gemini); returns a validated Pydantic instance or dict; most reliable method
  • ToolStrategy: Structured output via tool calling; compatible with all tool-capable models; the schema is registered as a tool and the model’s tool call arguments become the structured response
  • Auto-Selection: Passing a schema type directly to response_format triggers automatic strategy selection from the model’s profile data; falls back to ToolStrategy if ProviderStrategy is unsupported
  • structured_response Key: Where the validated structured output object appears in the agent state after completion
  • Error Handling (handle_errors): ToolStrategy parameter controlling retry behavior on validation failure — True (retry all, default), str (retry all with custom message), exception type(s) (selective retry), Callable (custom handler), False (no retry)
  • Union Types in ToolStrategy: Union[SchemaA, SchemaB] allows the model to select the most contextually appropriate schema; error handling fires if multiple schemas are returned simultaneously
  • strict Mode: ProviderStrategy parameter requesting strict schema adherence at the API level; supported by OpenAI and xAI; requires langchain>=1.2

Key Claims and Findings

  • ProviderStrategy and passing a schema type directly are functionally equivalent when native structured output is supported; the fallback to ToolStrategy is automatic and transparent
  • Models using ToolStrategy can incorrectly return multiple structured outputs simultaneously — LangChain detects this (MultipleStructuredOutputsError) and prompts the model to correct its response
  • Schema validation errors (e.g. out-of-range integers) trigger automatic retry with specific error feedback rather than silent failure — preserving correctness without crashing the agent
  • Model profile data (from langchain>=1.1) drives automatic strategy selection; custom profiles can override capability detection for models without published profiles

Strategy Decision Flow

response_format = ContactInfo  (schema type passed directly)
        │
        ▼
Does model/provider support native structured output?
        │
   Yes  ├──────────────────► ProviderStrategy(ContactInfo)
        │                     API enforces schema; returns Pydantic instance
   No   └──────────────────► ToolStrategy(ContactInfo)
                              Tool call arguments become structured response
                              Retry on validation failure (handle_errors=True)

Schema Types Supported

Schema TypeProviderStrategyToolStrategyReturn Type
Pydantic BaseModel✓✓Validated Pydantic instance
Python dataclass✓✓dict
TypedDict✓✓dict
JSON Schema dict✓✓dict
Union[A, B]—✓Pydantic instance or dict

Terminology

  • Pydantic BaseModel: Preferred schema definition class; provides field validation, type coercion, and returns a validated instance
  • tool_message_content: ToolStrategy parameter customizing the ToolMessage entry written into conversation history when structured output is generated
  • StructuredOutputValidationError: Raised when output fails Pydantic schema validation; intercepted by ToolStrategy for retry
  • MultipleStructuredOutputsError: Raised when the model calls multiple structured output tools in one response; ToolStrategy prompts the model to return a single response
  • strict parameter: ProviderStrategy flag requesting strict JSON schema enforcement at the provider API level

Connections to Existing Wiki Pages

  • Retry Pattern — ToolStrategy’s automatic retry-on-validation-failure (with configurable handle_errors) is a direct instantiation of the retry pattern applied at the structured output layer
  • Circuit Breaker Pattern — structured output enforcement and circuit-breaking both address reliability; structured output enforces correctness at the model response level, circuit breakers prevent cascading failures at the service call level
  • Observability Concepts (LangSmith) — structured output calls (whether via ProviderStrategy or ToolStrategy) appear as runs within a LangSmith trace, enabling inspection of schema validation retries and structured response payloads

Graph View

Backlinks