Design Considerations of Advanced Agentic AI for Real-World Applications

Cross-posted to Agent Architecture and Design.

Abstract

This Medium article (Indrajit Kar, 2024) compares three increasingly sophisticated implementations of an agentic data-extraction workflow — CODE1 (explicit class-based sequential agents), CODE2 (LangChain dynamic tools with ReAct reasoning), and CODE5 (hybrid) — to illuminate the design trade-offs between transparency, adaptability, and scalability in agentic AI systems. The concrete use case is extracting named entities from unstructured customer reviews, integrating them with structured transaction data, and generating metadata — a representative enterprise workflow involving structured, unstructured, and vector databases. The article examines five dimensions across the three implementations: orchestration, monitoring, error handling, state management, and planning. It concludes that hybrid designs (CODE5) offer the best balance for production systems requiring both predictable workflows and dynamic adaptability.

---

Key Concepts

The Three Architectures

Architecture	Implementation	Orchestration	State	Planning
CODE1 (Architecture 1)	6 class-based agents	Hardcoded sequential; Global Agent explicitly invokes each	Explicit GraphState dict; manually updated	Static — fixed workflow, no adaptation
CODE2 (Architecture 2)	5 LangChain tools + ReAct agent	LangChain’s ZERO_SHOT_REACT_DESCRIPTION selects tools dynamically	StatefulMemory implicit; updated by tool interactions	Fully dynamic — tool selection driven by reasoning
CODE5 (Architecture 3)	6 explicit agents + 5 LangChain tools	Explicit agents for workflow; LangChain tools for per-task execution	Hybrid: explicit agents update GraphState; tools use StatefulMemory	Hybrid: structured for predictable steps, dynamic for varied inputs

Global Agent / Orchestrator Role

In all three architectures, a Global Agent is responsible for:

1. Orchestration — sequencing agent or tool invocations.
2. Monitoring — tracking which component is active and whether it has succeeded.
3. Error handling — determining retry logic and failure recovery.
4. Adaptability — deciding the next step based on state or dynamic reasoning.

CODE1’s Global Agent is a hardcoded dispatcher; CODE2’s is a LangChain ReAct agent that reasons its way through tool selection; CODE5’s combines both modes.

ReAct Loop (ZERO_SHOT_REACT_DESCRIPTION)

The LangChain ZERO_SHOT_REACT_DESCRIPTION agent type implements the ReAct (Reasoning + Acting) loop:

Thought → Action → Action Input → Observation → [repeat] → Final Answer

Tools are described in natural language. The LLM reasons about which tool to invoke based on the current context and prior observations, without any hardcoded routing. This enables flexible, unforeseen tool sequences at the cost of transparency and debuggability. See What are Multi-Agent Systems? for broader context on the ReAct pattern in multi-agent settings.

State Management: GraphState vs. StatefulMemory

• GraphState (CODE1): an explicit Python dictionary tracking column_names, chain, metadata, etc. Every transition updates it manually. Transparent but brittle — the state schema must be maintained by the developer.
• StatefulMemory (CODE2/CODE5): LangChain’s memory abstraction updates state implicitly during tool execution. Persistent across retries — if a tool fails and is retried, the memory state from the prior attempt is available. Less transparent but more resilient.

Role of Databases

The article explicitly models three database types as complementary layers in an agentic system:

Type	Examples	Role in agent workflow
Structured	PostgreSQL, MySQL	Baseline reference data; deterministic queries (customer purchase history)
Unstructured	MongoDB, Elasticsearch	Raw input to LLM for NER, sentiment analysis, full-text search
Vector	Pinecone, Weaviate, Milvus	Semantic retrieval — store embeddings of reviews/documents; cosine similarity search for contextually similar records

Vector databases serve as external semantic memory for the agent — the agent retrieves relevant context without needing to scan all unstructured data on every invocation. See What Is Agent Memory? for how this maps to the broader memory taxonomy.

---

Key Algorithms

• Sequential dispatch (CODE1): Global Agent.invoke(ColumnNameAgent) → invoke(ChainCreationAgent) → ... → invoke(MetadataExtractionAgent). State updated at each step via explicit assignment to GraphState.
• ReAct tool selection (CODE2): LLM generates Thought: + Action: + Action Input: tokens; the framework invokes the named tool; the result is fed back as Observation:; loop continues until the LLM generates Final Answer:.
• Hybrid dispatch (CODE5): Structured agents are called by explicit orchestration; within each agent, LangChain tools are invoked via ReAct for the sub-task. Retry logic exists at both levels.
• Converting class-based agents to LangChain tools: Define a tool input/output schema (typically a Pydantic model), wrap the agent’s run() method as a @tool, and register it with the LangChain agent executor. The orchestration logic that previously called the class directly is replaced by the ReAct loop.

---

Key Claims and Findings

• Static (CODE1) workflows are easiest to debug and reason about, but require developer effort to extend and cannot adapt to unexpected inputs.
• Dynamic (CODE2) workflows handle variance well but make debugging harder due to opaque LLM reasoning; logging every Thought/Action/Observation trace is essential.
• Hybrid (CODE5) is the pragmatic production choice: use explicit orchestration for well-understood, dependency-ordered steps and LangChain tools for steps that may involve variance.
• StatefulMemory is a significant reliability improvement over GraphState in error recovery: when a tool fails and is retried, prior partial state is preserved without developer-managed checkpointing.
• The article implicitly argues that as workflows grow in complexity, the overhead of maintaining GraphState schemas outweighs the transparency benefit — favouring the LangChain memory abstraction for large multi-step pipelines.
• Vector databases are not optional in production agentic systems: without semantic retrieval, agents must either pass all unstructured data in context (limited by context window) or perform expensive LLM calls to scan it.

---

Terminology

Term	Definition
Global Agent	The top-level orchestrator in an agentic system; decides the sequence of agent or tool invocations
GraphState	Explicit shared state dictionary passed between agents in a static workflow
StatefulMemory	LangChain memory abstraction that persists state across tool calls and retries implicitly
ZERO_SHOT_REACT_DESCRIPTION	LangChain agent type that implements the ReAct loop with tool descriptions as the only context
ReAct	Reasoning + Acting — prompting pattern that interleaves LLM reasoning steps with tool invocations
NER	Named Entity Recognition — extracting structured entities (names, dates, amounts) from free text
Vector database	Database storing embedding vectors, enabling approximate nearest-neighbour and cosine similarity search

---

Connections

• What are Multi-Agent Systems? — the three architectures here are concrete implementations of the multi-agent coordination patterns surveyed there
• Building Autonomous AI with NVIDIA Agentic NeMo — NeMo’s orchestration layer implements the hybrid approach (CODE5 philosophy) using LangChain-compatible tool interfaces and explicit agent pipelines
• What Is Agent Memory? — StatefulMemory used in CODE2/CODE5 maps to episodic/procedural memory in the full taxonomy; vector databases described here are the storage substrate for semantic and associative memory
• Circuit Breaker Pattern — the retry logic embedded in CODE1 and CODE5 Global Agents is the seed of a retry strategy; the circuit breaker is the production-grade extension of this pattern for external service calls
• Cross-section: Agent Architecture and Design — architecture perspective on the same material