Ch. 2 — What Is Agentic AI

Chapter 2 of Document Overview

Abstract

This chapter establishes the foundational definitions and architectural frameworks underlying Agentic AI systems, distinguishing them from traditional deterministic software through their capacity for autonomous goal-directed behavior. It introduces the core operational cycle of Perceive-Reason-Act-Learn as the mechanism enabling continuous environmental adaptation and iterative problem-solving. Furthermore, the text details the progression of agent formulations from Russell and Norvig, analyzing the evolution from simple reflex architectures to complex learning agents equipped with internal world models and performance critiquing. This technical baseline is critical for understanding the shift toward probabilistic, semantic reasoning systems that utilize Large Language Models (LLMs) as orchestration engines within a continuous feedback loop.

Key Concepts

• Agentic AI Definition: Agentic AI represents a paradigm shift where sophisticated reasoning and iterative planning allow for autonomous problem-solving in complex, multi-step environments. Unlike static programs, these systems utilize a continuous cycle known as Perceive-Reason-Act-Learn to execute self-directed tasks that meet predetermined goals. The central capability involves interacting with an environment to collect data and adapt strategies based on semantic understanding rather than rigid scripting.
• Perceive-Reason-Act-Learn Cycle: This four-step process constitutes the fundamental agent loop where each action alters the global environment, triggering new perceptions for the subsequent cycle. The cycle is non-linear and continuous, ensuring that the agent’s state is dynamically updated through a data flywheel where interaction outcomes enhance future performance. It transforms static software execution into a dynamic flow of environmental feedback and strategic adjustment.
• Traditional vs. Agentic Software: Conventional software follows a deterministic path defined by Input → Deterministic Logic → Output, ensuring fully specified behavior with predictable outcomes. Conversely, Agentic AI operates via Environment Perception → Semantic Reasoning → Context-Aware Action, resulting in probabilistic behavior that is adaptable to novel situations through goal-oriented decision-making.
• LLM as Reasoning Engine: The Large Language Model functions as the central orchestrator and reasoning engine within the agent architecture, operating within “semantic space” to understand causality and implications. It decomposes complex goals into manageable steps and may utilize Retrieval-Augmented Generation (RAG) techniques to access proprietary data during the reasoning phase.
• Local Perception vs. Global Environment: A critical design constraint is the distinction between the Global Environment, representing the complete actual state of the world, and Local Perception, which is limited to what the agent’s specific sensors can detect. Agents must map these incomplete local perceptions to local actions that modify the global state, requiring careful curation of sensor design and information feature extraction.
• Simple Reflex Agent Architecture: Characterized by condition-action rules ($Condition\to Action$), this agent type operates solely on immediate percepts without internal memory or a world model. It reacts to “what the world is like now” based on predefined rules, lacking the capacity to track temporal evolution or predict future states.
• Model-Based Reflex Agent: This architecture introduces internal state to maintain information about the world, allowing the agent to account for “how the world evolves” and “what my actions do.” By integrating a transition model with condition-action rules, it achieves more intelligent responses informed by a history of state changes rather than just immediate input.
• Goal-Based Agent: Moving beyond reflexive logic, this agent type incorporates specific goals as desired outcomes, enabling consequence prediction (“What it will be like if I do action A”). It evaluates actions based on whether they lead to goal achievement, effectively planning trajectories in semantic space to satisfy high-level objectives.
• Learning Agent Architecture: Described as the most advanced formulation, this agent includes a performance element, a critic for evaluation, a learning element for modification, and a problem generator for creating challenges. The learning element uses feedback from the critic to improve the performance element over time, enabling the “data flywheel” effect.
• Data Flywheel: This concept describes the mechanism of continuous improvement where data from agent interactions feeds back into the system to enhance models based on outcomes and feedback. It facilitates better decision-making and operational efficiency by ensuring that model parameters evolve through experience rather than remaining static.

Key Equations and Algorithms

• Simple Reflex Logic: The core decision logic for the simplest agent is represented as $Percept\stackrel{Map(Condition)}{\to }Action$. This equation signifies that the immediate input is directly mapped to a specific action via lookup rules, without internal state computation or future prediction.
• Model-Based State Update: For agents tracking internal states, the world evolution is modeled as $Stat{e}_{t+1}=f(Stat{e}_t,Actio{n}_t,Percep{t}_t)$. This functional relationship indicates that the new internal state is derived from the previous state, the action just taken, and the current perception, enabling the agent to maintain a coherent world model.
• Goal Evaluation Function: The evaluation of potential actions in a Goal-Based Agent relies on the prediction $Utility(Action)\to Goal\_Achieve$. This implies a selection process where actions are chosen based on their predicted ability to transition the current state to a state satisfying the desired goal conditions.
• Learning Agent Feedback Loop: The improvement cycle is defined by $Performanc{e}_{new}\leftarrow Learning(Performanc{e}_{old},Feedbac{k}_{critic})$. This process denotes that the performance element is modified by a learning algorithm using error signals or evaluation metrics provided by the critic component.
• The Agentic Loop Algorithm: The operational procedure is defined as $Loop:\{Perceive\to Reason\to Act\to Learn\Rightarrow Perceive\}$. This recursive algorithmic structure emphasizes that the output of the Learn phase serves as the input for the next Perceive phase, creating an infinite operational chain driven by environmental changes.

Key Claims and Findings

• Agentic AI is fundamentally distinguished from traditional software by its use of probabilistic behavior and goal-oriented decision-making rather than predetermined deterministic logic. This shift allows systems to adapt strategies based on environmental feedback instead of executing fixed rules.
• The Perceive-Reason-Act-Learn cycle is continuous by design, meaning each action modifies the global environment state, which creates new perceptions that automatically trigger the next cycle. This self-perpetuating loop is the defining characteristic of autonomous agent operation.
• Agents operate under the constraint of limited local perception, meaning they cannot observe the entire global environment but must rely on sensor-captured features. Consequently, agent design requires careful mapping of local actions to global state modifications.
• Russell and Norvig’s agent formulations demonstrate a clear progression in capability: Simple Reflex (no memory), Model-Based (internal state), Goal-Based (planning), and Learning (self-improvement). Each successive architecture adds a layer of sophistication regarding memory, planning, or adaptability.
• Retrieval-Augmented Generation (RAG) is identified as a specific technique used during the Reason phase to allow the LLM orchestrator to access proprietary data not contained in its base training weights. This expands the agent’s semantic reasoning capabilities with external knowledge sources.
• The Learning Agent is the only architecture capable of autonomous performance improvement via a performance element modification mechanism driven by a critic. This enables the “data flywheel” effect where system efficiency grows over time without human intervention.

Terminology

• Agentic AI: A software program capable of interacting with an environment to collect data and perform self-directed tasks meeting predetermined goals through autonomous reasoning.
• Semantic Reasoning: The process by which the LLM orchestrator understands tasks and solves problems within “semantic space,” a domain defined by meaning, causality, and logical implications rather than syntax.
• Percepts: The input data provided to the agent through sensors, representing the localized view of the global environment accessible to the system.
• Actuators: The hardware or software components that allow the agent to produce local actions, thereby modifying the state of the global environment.
• Simple Reflex Agent: An agent architecture that uses condition-action rules to determine behavior based solely on the current percept, lacking any internal state or memory of past events.
• Model-Based Reflex Agent: An architecture that maintains an internal state to track how the world evolves and how actions affect outcomes, utilizing transition models to inform decision-making.
• Goal-Based Agent: An agent system that utilizes desired outcomes to evaluate the consequences of potential actions, choosing behaviors that maximize the likelihood of achieving specified goals.
• Critic: A component within the Learning Agent that evaluates the agent’s performance against a standard and provides feedback to the learning element.
• Learning Element: The functional module responsible for modifying the performance element of an agent based on feedback received from the critic to improve over time.
• Performance Element: The part of the Learning Agent architecture that constitutes the actual decision-making logic, which may itself be any of the other agent formulations.
• Data Flywheel: The mechanism by which interaction data feeds back into the system to enhance models based on outcomes, enabling continuous improvement in decision-making.
• Global Environment: The complete, actual state of the world, of which the agent only perceives a subset through its limited sensors.