Agentic design pattern
Here are detailed notes for Chapter 1: Prompt Chaining from the book Agentic Design Patterns.
Chapter 1: Prompt Chaining (The Pipeline Pattern)
1. Core Concept & Definition
Definition: Prompt Chaining, also known as the Pipeline pattern, is a technique for handling complex tasks by breaking them down into a sequence of focused, manageable sub-tasks.
Mechanism: Instead of asking a Large Language Model (LLM) to do everything in one go, the output of one prompt becomes the input for the next prompt in a chain. This creates a logical workflow where each step performs a specific operation.
Analogy: It is similar to a computational pipeline in software engineering where data flows through a series of functions.
2. The Problem: Limitations of Single Prompts
The chapter explains that using a single, "monolithic" prompt for multifaceted tasks often leads to failure due to several factors:
Instruction Neglect: The model may overlook parts of the instructions when faced with too many constraints.
Context Drift: The model might lose track of the initial context as it generates long responses.
Error Propagation: Small errors early in the response can amplify as the generation continues.
Hallucination: The increased cognitive load increases the chance of the model generating incorrect information.
3. The Solution: Sequential Decomposition
Prompt chaining solves these issues by assigning a distinct "role" or specific goal to each step of the chain.
Example Scenario: Analyzing a market research report.
Bad Approach: One prompt asking to summarize, find trends, and draft an email.
Chained Approach:
Step 1 (Market Analyst): "Summarize the key findings..." -> Output: Summary.
Step 2 (Trade Analyst): "Using the summary, identify top 3 trends..." -> Output: Trends.
Step 3 (Writer): "Draft an email outlining these trends..." -> Output: Final Email.
4. Technical Implementation Strategies
Structured Output: Reliability is heavily dependent on passing clean data between steps. The chapter recommends using structured formats like JSON or XML for intermediate outputs.
Example: Step 2 should output a JSON object containing
{"trend_name": "...", "supporting_data": "..."}so Step 3 can parse it accurately without ambiguity.
Context Engineering: This involves designing the informational environment for the model. By "chaining," you are effectively engineering the context for the next step, ensuring the model has exactly what it needs (the previous step's output) and nothing else.
5. Practical Applications & Use Cases
The chapter highlights several domains where this pattern is essential:
Information Processing: Extracting text -> Summarizing -> Extracting specific entities -> Querying a database.
Conversational Agents: Maintaining "state" in a chatbot by passing the conversation history and extracted entities into the next prompt as context.
Code Generation: Decomposition into: Generate Pseudocode -> Write Initial Code -> Review/Debug -> Refine Code.
Multimodal Reasoning: Breaking down image analysis into: Extract text from image -> Link text to labels -> Interpret data.
6. Key Takeaways
Divide and Conquer: The primary strategy is decomposing complex problems into simpler, sequential steps.
Reliability: Chaining significantly improves the reliability and control of LLM outputs compared to single prompts.
Frameworks: Tools like LangChain (using LCEL), LangGraph, and Google ADK provide the infrastructure to manage these chains programmatically.
Chapter 2: Routing
1. Core Concept & Definition
Definition: The Routing pattern introduces dynamic decision-making into an agent's workflow. Unlike a fixed, linear pipeline (like Prompt Chaining), routing allows an agent to evaluate the current situation or user input and "route" the flow of control to the most appropriate specialized agent, tool, or sub-process1.
The Shift: It represents a move from deterministic execution (Step A Step B) to conditional execution (If X, then Step A; Else, Step B)2.
Analogy: It functions like a traffic controller or a switchboard operator, analyzing incoming requests and directing them to the correct department3.
2. The Problem: Limitations of Sequential Flows
Linear Rigidity: Simple sequential processing (Prompt Chaining) works well for predictable tasks but fails when the input is highly variable4.
Contextual Needs: Real-world systems must handle diverse inputs where a "one-size-fits-all" path is inefficient or impossible. For example, a "Customer Service" agent needs to handle both "technical support" and "sales" queries, which require completely different workflows5.
3. The Solution: Dynamic Decision Making
Routing solves this by inserting a decision step—often handled by an LLM or a classification algorithm—before execution6. The chapter details two primary methods for implementation:
LLM-Based Routing: The system asks an LLM to classify the input.
Example: You instruct the LLM: "Analyze this query and output only the category: 'Order Status', 'Product Info', or 'Technical Support'"7.
Embedding-Based Routing (Semantic Routing): The system converts the user's text into a vector embedding and compares it mathematically (e.g., cosine similarity) to embeddings of predefined routes. The system selects the route with the highest similarity score. This is faster and cheaper than an LLM call8.
4. Hands-On Implementation (Google ADK Example)
The chapter provides a code example using the Google Agent Developer Kit (ADK) to build a "Coordinator" agent9.
Coordinator Agent: Acts as the router. It receives all user messages10.
Sub-Agents:
Booker: Handles specific requests like "Book me a hotel"11.
Info: Handles general knowledge questions like "What is the highest mountain?"12.
Auto-Flow: The framework automatically handles the delegation. If the user asks for a hotel, the Coordinator routes the task to the Booker agent. If the user asks for random facts, it routes to the Info agent13.
5. Frameworks & Approaches
LangChain / LangGraph: Uses a graph-based structure where "conditional edges" determine the path between nodes (agents). This is ideal for complex, multi-step workflows where the routing logic needs to be explicit and visual14.
Google ADK: Focuses on an "Auto-Flow" mechanism where a coordinator agent manages sub-agents, making the routing implicit based on the sub-agents' defined roles15.
6. Key Takeaways
Adaptability: Routing allows agents to handle diverse and unpredictable inputs by adapting their behavior in real-time16.
Efficiency: By sending tasks only to the relevant specialized agent, the system avoids running unnecessary steps or tools17.
Flexibility: Logic can be implemented via simple rules, semantic embedding matching, or full LLM reasoning depending on the complexity required18.
Foundation for Autonomy: This pattern is a critical step toward true autonomy, moving beyond simple scripts to systems that can "decide" how to solve a problem19.
Chapter 3: Parallelisation
1. Core Concept & Definition
Definition: The Parallelization pattern enables an agentic system to execute multiple independent sub-tasks simultaneously rather than sequentially1.
Mechanism: Instead of waiting for one step to finish before starting the next (blocker), the system triggers multiple operations—such as LLM calls, tool usages, or entire sub-agents—at the same time2.
Goal: The primary objective is to significantly reduce overall execution time (latency) and improve efficiency for complex workflows3.
2. The Problem: Sequential Bottlenecks
Inefficiency: In a purely sequential workflow (like standard Prompt Chaining), if an agent needs to check the stock market, read the news, and look up a CEO's bio, it must do them one by one. If each step takes 5 seconds, the user waits 15 seconds.
Dependency Trap: Treating independent tasks as dependent steps creates unnecessary delays4.
3. The Solution: Concurrent Execution
Divide and Run: The pattern identifies tasks that do not depend on each other and runs them in parallel branches.
Example Scenario: A "Research Agent" tasked with analyzing a company.
Sequential Approach: Search News Summarize News Pull Stock Data Analyze Stocks.
Parallel Approach:
Branch A: Search News Summarize News.
Branch B: Pull Stock Data Analyze Stocks.
Result: Both branches run at once, and their outputs are aggregated at the end5.
4. Technical Implementation Strategies
The chapter highlights how different frameworks handle this:
LangChain (LCEL): Uses
RunnableParallelto define a dictionary of chains that execute concurrently. The results are collected into a single map6.LangGraph: Allows you to define a graph where a single state transitions into multiple nodes simultaneously, creating parallel branches7.
Google ADK: Provides a specific
ParallelAgentclass that manages the concurrent execution of sub-agents natively8.
5. Practical Applications & Use Cases
Information Gathering: An agent researching a topic can simultaneously query Google Search, check social media, and query a database to build a comprehensive view faster9.
Data Validation: When verifying a user profile, the system can run independent checks (validate email format, check address, scan for profanity) all at once10.
Multi-Modal Processing: Analysing a post by sending the text to a sentiment analysis model and the image to an object detection model simultaneously11.
A/B Testing / Creativity: Generating three different variations of a headline in parallel to let a final step choose the best one12.
6. Key Takeaways & Trade-offs
Speed vs. Cost: Parallelisation drastically reduces latency but increases the "burst" usage of computational resources and API costs13.
Complexity: It requires a mechanism to "aggregate" or synthesise the results from all parallel branches into a final cohesive answer14.
Limits: Developers must be aware of API rate limits (throttling) since parallel agents initiate multiple calls instantly15.
Chapter 4: Reflection
1. Core Concept & Definition
Definition: The Reflection pattern is a mechanism where an agent evaluates its own output or internal state to improve the quality of its final response.
The Shift: Unlike previous patterns that focus on execution flow (Chaining, Routing, Parallelization), Reflection focuses on refinement. It assumes the initial output might be suboptimal or contain errors and introduces a "check" step before finalising the result.
Analogy: It is comparable to a human writer drafting a document and then reviewing it to fix typos, clarify logic, or improve flow before publishing.
2. The Architecture: Producer-Critic Model
The chapter identifies the Producer-Critic (or Generator-Reviewer) model as the most robust implementation of this pattern. It separates the workflow into two distinct roles:
The Producer Agent: This agent focuses entirely on generating the initial content, code, or plan based on the prompt.
The Critic Agent: This agent does not generate content. Its sole purpose is to analyze the Producer's output against specific criteria (e.g., "Check for security bugs," "Verify tone," "Ensure all constraints are met") and provide feedback.
3. The Workflow: The Feedback Loop
Reflection creates a cyclical process rather than a linear one:
Generate: The Producer creates an initial draft.
Evaluate: The Critic reviews the draft and identifies specific issues.
Refine: The Producer (or a Refiner agent) uses the Critic's feedback to generate an improved version.
Repeat: This loop can continue for a fixed number of steps or until the Critic is satisfied (passes a quality threshold).
4. Key Integrations
Memory (Chapter 8): Reflection is significantly enhanced by memory. The agent needs conversational history to understand why a previous attempt failed so it doesn't repeat the same error in the next loop.
Goal Setting (Chapter 11): Goals serve as the "benchmark" for the Critic. The agent reflects on whether its current path is actually moving it closer to the defined objective.
5. Practical Applications & Use Cases
Code Generation: An agent writes code, and a "Compiler/Linter" agent (or tool) checks for errors. If errors are found, the agent "reflects" on the error message to fix the code.
Summarization: An agent drafts a summary, then reflects: "Did I miss any key points from the original text?" before outputting the final version.
Planning: Before executing a complex plan, an agent simulates the steps to identify potential bottlenecks or logical flaws.
Safety & Compliance: A "Guardrail" agent reviews a response to ensure it doesn't violate safety policies before showing it to the user.
6. Implementation Frameworks
LangGraph: Ideal for this pattern because it supports cycles (loops) where the state can circulate between a "generate" node and a "reflect" node until a condition is met.
LangChain (LCEL): Can handle simple, single-step reflection (Generate Critique Fix) but is less suited for indefinite loops.
Google ADK: Uses sequential workflows where one agent's output is passed as input to a "Reviewer" agent for critique.
7. Key Takeaways & Trade-offs
Quality vs. Latency: Reflection drastically improves output quality and reliability but introduces significant latency (time delay) and higher cost because it requires multiple LLM calls for a single user request.
Risk of Loops: Without proper exit conditions (e.g., "max_retries=3"), an agent could get stuck in an infinite loop of critiquing and refining without ever satisfying the critic.
Chapter 5: Tool Use (Function Calling)
1. Core Concept & Definition
Definition: The Tool Use pattern, technically known as Function Calling, is the mechanism that enables an agent to interact with the external world. It allows an LLM to step outside its internal text generation capabilities to execute code, query databases, or call external APIs1.
The Bridge: It bridges the gap between the LLM's "static" knowledge (training data) and the "dynamic" real world (live data like stock prices, weather, or user databases)2.
Terminology: While often called "Function Calling," the chapter suggests the broader term "Tool Calling." A "tool" can be a simple code function, a complex API endpoint, or even a delegation instruction to another specialized agent3.
2. The Workflow: How It Works
The chapter details a standard 5-step process for this pattern44:
Tool Definition: The developer defines available tools (e.g.,
get_weather,send_email) and describes them to the LLM, including parameters and data types5.Request Formulation: The LLM analyzes the user's prompt (e.g., "Send an email to Bob"). It determines which tool to use and generates a structured request (e.g., a JSON object:
{"tool": "send_email", "recipient": "Bob"})6.Client Communication: The system (the "Client") receives this structured request from the LLM and routes it to the appropriate server or API7.
Server Execution: The tool/server executes the actual action (sends the email, queries the DB)8.
Response & Context Update: The tool sends the result (e.g., "Email sent successfully" or "Error: User not found") back to the LLM. The LLM uses this new information to generate the final response to the user9.
3. Tool Use vs. Model Context Protocol (MCP)
The chapter distinguishes between standard Function Calling and the Model Context Protocol (MCP)10:
Function Calling: Typically proprietary and specific to the LLM provider (e.g., OpenAI's format vs. Google's format). The implementation differs across vendors11.
MCP: An open, standardized protocol designed to promote interoperability. It allows different LLMs to connect to different tools without needing custom adapters for every combination12.
Data Compatibility: The chapter notes that connecting is not enough; the data format must be "agent-friendly." For example, an API returning a raw PDF is less useful than one returning parsed Markdown text that the agent can read13.
4. Practical Applications & Use Cases
Data Access: E-commerce agents checking real-time product inventory or order status14.
Calculations: Financial agents using a calculator tool or stock market API to perform precise math, which LLMs often struggle with natively15.
Action Execution: Agents that can perform tasks like sending emails, booking calendar slots, or updating CRM records16.
5. Implementation Nuances (Google Vertex AI)
Extensions vs. Function Calling: The notes highlight a distinction in the Google ecosystem. Extensions are executed automatically by the platform (Vertex AI), whereas Function Calls generate a request that the client application must manually execute17.
6. Key Takeaways
Dynamic Capability: Without tools, an LLM is isolated; with tools, it becomes an agent capable of action18.
Structured Output: The reliability of this pattern relies on the LLM's ability to output strict, structured data (like JSON) that machines can parse19.
Orchestration: The "Tool Use" pattern allows agents to act as orchestrators, managing a diverse ecosystem of digital resources20.
Chapter 6: Planning
1. Core Concept & Definition
Definition: The Planning pattern endows an agent with the ability to look ahead. Instead of immediately reacting to input, the agent formulates a structured sequence of actions (a plan) to bridge the gap between its current state and a desired goal state1.
The Shift: It represents a shift from Reactive behavior (responding to immediate stimuli) to Strategic behavior (devising a strategy before acting).
Analogy: The chapter likens a planning agent to a specialist or project manager. You define the "what" (the objective and constraints), and the agent figures out the "how" (the specific steps to get there)2.
2. The Problem: Complexity Overload
Limitation of Simple Agents: Simple reactive agents or basic prompt chains often fail when faced with high-level, ambiguous goals like "Organize a team offsite" or "Research the impact of quantum computing."
Missing Logic: Without a plan, agents may execute actions out of order, miss critical dependencies (e.g., trying to book a flight before knowing the dates), or get lost in the details of a sub-task, losing sight of the main objective3.
3. The Solution: Decomposition
The core mechanism of this pattern is Decomposition. The agent breaks down a massive, insurmountable goal into smaller, manageable sub-goals or "atomic" actions that can be executed sequentially or in parallel4.
Workflow:
Goal Ingestion: The agent receives a complex user request.
Plan Generation: The LLM generates a structured plan (often a numbered list or JSON) outlining the necessary steps55.
Execution: The system iterates through the plan, executing each step one by one.
Refinement (Optional): In advanced implementations, the agent can update the plan dynamically if a step fails or new information arises6.
4. Practical Applications & Use Cases
Research & Report Generation: A "Deep Research" agent (like Google Deep Research) breaks a topic down: "Search for X" "Summarize X" "Identify new questions" "Search for Y" "Synthesize final report"777.
Onboarding: An HR agent breaks down "Onboard new employee" into: "Create email account," "Add to Slack," "Schedule intro meetings," and "Send welcome kit"8.
Competitive Analysis: Decomposing a request to "Analyse Competitor X" into gathering financial data, reviewing product launches, and checking customer sentiment9.
5. Implementation Strategy
Explicit Prompting: The chapter emphasises that you must explicitly prompt the model to plan. For example, instructing the agent: "First, create a plan. Then, execute the plan step-by-step"10.
Structured Output: Plans should ideally be output in structured formats (like JSON arrays) so the system can programmatically parse and execute them11.
6. Key Takeaways
Foresight: Planning transforms agents from simple responders into goal-oriented executors capable of foresight12.
Handling Dependencies: It is the ideal pattern for tasks where step B cannot happen until step A is complete13.
Scalability: This pattern allows agents to tackle tasks that are too complex for a single tool call or prompt14.
ReAct: The chapter references the ReAct (Reason and Act) framework as a foundational concept where the agent "thinks" (plans) before it acts15.
Chapter 7: Multi-Agent Collaboration
1. Core Concept & Definition
Definition: The Multi-Agent Collaboration pattern structures an AI system as a cooperative ensemble of distinct, specialized agents. Instead of a single "monolithic" agent trying to do everything, the problem is decomposed into sub-problems, each assigned to an agent with the specific tools and expertise required1.
The Shift: This moves beyond single-agent capabilities (like simple tool use) to a team-based approach, where agents interact, delegate, and debate to solve complex, multi-domain tasks2.
Analogy: It functions like a human company or research team. You don't ask one person to be the researcher, writer, editor, and legal compliance officer. You hire specialists for each role and have them collaborate33.
2. The Problem: The Monolithic Bottleneck
Context Limitation: A single agent often struggles with "context window" limits when trying to maintain all the instructions, rules, and history for a massive task4.
Conflicting Instructions: Complex tasks often have competing requirements (e.g., "be creative" vs. "be strictly factual"). A single agent may get confused, whereas separate "Creative" and "Reviewer" agents can hold these distinct roles without internal conflict5.
Fragility: If one part of a monolithic prompt fails, the whole system fails. In a multi-agent system, if the "Researcher" fails, the "Manager" can catch the error and ask it to try again without derailing the "Writer"6.
3. Architectures of Collaboration
The chapter outlines several ways to structure these teams:
Orchestrator (Manager/Worker): A central "Manager" agent breaks down the plan and delegates tasks to "Worker" agents. The workers report back to the manager, who synthesizes the final result. This is the most common pattern777.
Sequential Handoffs: Agent A completes a task and passes the output directly to Agent B (e.g., Researcher Writer Editor)8.
Debate / Consensus: Multiple agents with different perspectives (or even different LLMs) propose solutions and critique each other to reach a better answer than any single agent could produce alone9.
Hierarchical Teams: A "Chief Editor" manages a team of "Section Editors," who in turn manage "Writers," creating a pyramid of responsibility10.
4. Hands-On Implementation
The chapter provides code examples using:
CrewAI: Highlights how this framework is specifically designed for this pattern. You define Agents (with roles and backstories), Tasks (specific assignments), and a Crew (the team that executes the process, usually sequentially or hierarchically)111111.
Google ADK: Demonstrates creating a "Coordinator" agent that uses an LLM to identify sub-tasks and trigger specialized sub-agents121212.
LangGraph: Mentioned as a powerful tool for defining the explicit "state machine" of how agents pass control to one another (though CrewAI is noted as being higher-level/easier for this specific pattern)13.
5. Practical Applications & Use Cases
Software Development: A "Product Manager" agent defines specs, a "Developer" agent writes code, and a "QA" agent writes tests. They iterate until the tests pass14.
Content Creation: A "Trend Watcher" agent finds topics, a "Writer" drafts the post, and a "Social Media Manager" optimizes it for different platforms15.
Financial Analysis: One agent pulls stock data, another reads news sentiment, and a third synthesises a "Buy/Sell" recommendation based on both data streams16.
Customer Support: A front-line agent handles basics, while a specialist agent handles technical debugging, and a supervisor agent ensures tone and policy compliance17.
6. Key Takeaways
Specialisation: Agents perform better when they have a narrow, well-defined scope (e.g., "You are a Python expert" vs. "You are a general assistant")18.
Scalability: You can add new capabilities simply by adding a new specialist agent to the team, rather than rewriting a massive prompt19.
Inter-Agent Communication: The success of this pattern depends heavily on clear protocols for how agents talk to each other (e.g., the Agent2Agent protocol)20.
Complexity: While powerful, this pattern introduces significant complexity in debugging (who made the mistake?) and cost (many more LLM calls per user request)21.
Chapter 8: Memory Management
1. Core Concept & Definition
Definition: Memory Management is the capability that allows an agent to retain, recall, and utilize information from past interactions, observations, and learning experiences1.
The Shift: It moves an agent from being a stateless processor (where every interaction is new) to a stateful entity that can maintain conversational context and "learn" over time2.
Analogy: Similar to human cognitive processes, agents require different types of memory to function efficiently—some for immediate tasks (working memory) and some for lasting knowledge (long-term memory)3.
2. The Two Main Types of Memory
The chapter categorizes memory into two primary distinct buckets:
Short-Term Memory (Contextual):
Function: Acts like "working memory." It holds information currently being processed, such as recent messages, immediate tool outputs, and the current state of the conversation4.
Limitation: It is strictly limited by the LLM's context window. Once the window is full, older information is "forgotten" unless moved to long-term storage5.
Long-Term Memory (Persistent):
Function: Allows the system to retain information across completely different sessions or conversations6.
Mechanism: This is typically achieved by storing data in external databases (Vector DBs) and retrieving it when needed (RAG), effectively giving the agent "infinite" storage capacity beyond its context window7.
3. Deep Dive: Categories of Long-Term Memory
The chapter further breaks down long-term memory into specific cognitive functions:
Episodic Memory: The ability to recall specific past events or interactions (e.g., "What did the user ask me last Tuesday?")8.
Procedural Memory (Remembering Rules): The memory of how to perform tasks. This is often embedded in the system prompt. The chapter notes that advanced agents can update their own procedural memory using Reflection, effectively rewriting their own instructions to improve performance9.
Semantic Memory: General knowledge about the world or specific domain facts (e.g., company policies), usually accessed via a Knowledge Base10.
4. Hands-On Implementation
The chapter provides code examples for implementing memory in two major frameworks:
Google ADK Approach:
Session State (Short-Term): Uses services like
InMemorySessionServiceto track immediate variables (e.g.,login_count,task_status) during a single active session11.Memory Service (Long-Term): Uses
VertexAiRagMemoryServiceto connect the agent to a persistent RAG corpus. This allows methods likesearch_memoryto retrieve relevant past data based on vector similarity12.
LangChain Approach:
ChatMessageHistory: A class for manually tracking and storing the list of user and AI messages (
history.add_user_message(...))13.ConversationBufferMemory: A wrapper that automatically manages the conversation buffer and injects it into the prompt (via variables like
{chat_history}), allowing the LLM to "see" the conversation so far14.
5. Practical Applications & Use Cases
Conversational AI: Essential for chatbots to provide coherent answers by referencing previous user inputs rather than treating every question in isolation15.
Personalization: Long-term memory enables agents to remember user preferences (e.g., "I always prefer aisle seats") across multiple sessions16.
Task Tracking: For complex, multi-step tasks, memory is required to track which steps have been completed and what remains to be done17.
6. Key Takeaways
Context is Finite: You cannot rely solely on the context window; effective agents must offload information to persistent storage.
State Management: Distinguishing between "Session State" (temporary) and "Knowledge" (permanent) is critical for architectural design18.
Self-Evolution: Through Procedural Memory and Reflection, agents can evolve their own behavior rules, moving toward self-improving systems19.
Chapter 9: Learning and Adaptation
1. Core Concept & Definition
Definition: The Learning and Adaptation pattern enables an agent to autonomously refine its knowledge, strategies, and behaviors based on past experiences and new data1.
The Shift: It transforms an agent from a static entity (which relies solely on pre-programmed logic or fixed training data) into a dynamic system that evolves over time2.
Distinction:
Learning: The internal process of acquiring new insights or data from interactions.
Adaptation: The visible change in the agent's behavior or output resulting from that learning3.
2. The Problem: Static Rigidity
Unpredictability: Real-world environments are dynamic. A pre-programmed agent often fails when it encounters novel situations that were not anticipated during its initial design4.
Stagnation: Without learning, an agent cannot optimize its performance. It will make the same mistake twice and cannot personalize its interactions to specific users5.
3. The Solution: Evolving Systems
The pattern integrates mechanisms that allow the agent to store and analyze its experiences. The chapter highlights several approaches:
Reinforcement Learning: The agent receives "rewards" or "penalties" for its actions, gradually learning to maximize the reward.
Knowledge Base Learning (RAG): The agent uses Retrieval Augmented Generation to maintain a dynamic database of problems and solutions. By storing "successful strategies" and "past failures," it references this data in future decisions to avoid known pitfalls6.
Self-Modification: Advanced agents can actually rewrite their own internal code or prompts to improve their capabilities7.
4. Case Study: The Self-Improving Coding Agent (SICA)
The chapter features a deep dive into SICA, a system developed by researchers including Maxime Robeyns and Martin Szummer8.
Mechanism: SICA was designed to self-improve by modifying its own code based on past performance.
Results: Through this evolutionary process, SICA autonomously developed new, specialized tools for itself, such as a Smart Editor and an AST (Abstract Syntax Tree) Symbol Locator, which significantly improved its code navigation and editing abilities9.
5. Implementation Strategy
Architecture: Successful implementation often requires a hierarchy.
Sub-agents: Specialized agents perform specific tasks.
Overseer Agent: A manager agent monitors performance and guides the learning process to ensure the system stays on track10.
Context Engineering: Efficiently managing the LLM's context window is vital. You must carefully curate system prompts and historical examples to ensure the "learned" lessons are actually visible to the model during execution11.
6. Key Takeaways
Necessity: This pattern is vital for agents operating in uncertain, changing environments or applications requiring high personalization12.
Data Flow: Building learning agents requires robust pipelines to manage how interaction data is captured, processed, and fed back into the system (e.g., via machine learning tools)13.
Autonomy: True autonomy is impossible without the ability to learn; otherwise, the agent remains dependent on human developers to patch every new edge case14.
Chapter 10: Model Context Protocol (MCP)
1. Core Concept & Definition
Definition: The Model Context Protocol (MCP) is an open standard designed to facilitate seamless communication between Large Language Models (LLMs) and external systems (data sources, tools, and applications)1.
The "Universal Adapter": The chapter likens MCP to a "universal adapter" or USB port for AI. Instead of developers building custom, proprietary integrations for every new tool an agent needs to use, MCP provides a standardised way to plug any tool into any LLM2.
Goal: The primary objective is to solve the "many-to-many" problem where every LLM (Claude, Gemini, GPT) currently requires a unique connector to talk to every data source (Google Drive, Slack, GitHub). MCP unifies this into a single protocol3.
2. Architecture: Client-Host-Server
MCP operates on a distinct client-server architecture that standardizes how information flows:
MCP Host (The Application): This is the application where the AI "lives" (e.g., the Claude Desktop app, or an IDE like Cursor). The Host is responsible for managing the connection and permissions4.
MCP Client (The Agent): The LLM or agent within the host that "speaks" the protocol. It queries servers to see what they can do5.
MCP Server (The Tool/Resource): A lightweight service that sits in front of a data source (like a local database or a remote API). It exposes three specific things to the client:
Resources: Static data the agent can read (e.g., files, logs, database records)6.
Prompts: Pre-written templates that help the agent use the server effectively7.
Tools: Executable functions the agent can call (e.g., "add_row_to_db")8.
3. MCP vs. Function Calling
The chapter draws a clear distinction between standard Function Calling (Chapter 5) and MCP:
Function Calling: Is often vendor-specific (proprietary to OpenAI, Google, etc.) and requires the developer to hard-code the tool definitions directly into the agent's system prompt. It is a "1-to-1" connection9.
MCP: Is an open standard that allows for Dynamic Discovery. The agent connects to a server and asks, "What can you do?" The server replies with its list of tools and resources. This separates the tool definition from the agent's core logic, allowing agents to "hotswap" tools without code changes10.
4. Key Capabilities
Dynamic Discovery: Agents can discover new tools at runtime. If you add a new "PDF Reader" capability to your MCP server, the agent "sees" it immediately upon next connection without needing a prompt update11.
Local & Remote: MCP is designed to work both locally (securely accessing files on your laptop via
stdio) and remotely (accessing web APIs viaSSE- Server-Sent Events)12.Agent-Friendly Data: The chapter emphasizes that MCP isn't just a pipe; it encourages exposing data in formats agents can actually understand. For example, an MCP server for a PDF drive shouldn't just return raw bytes; it should return parsed text or Markdown that the LLM can process13.
5. Hands-On Implementation
The chapter highlights FastMCP as a Python library that radically simplifies building these servers.
Code Example: A developer can create a tool simply by writing a standard Python function and adding a decorator:
Python
FastMCP automatically handles the JSON-RPC communication, error handling, and tool definition generation, letting the developer focus solely on the logic14.
6. Practical Applications
IDE Integration: Tools like Cursor or Windsurf use MCP to let their coding agents read your local project files, terminal output, and git history securely15.
Enterprise Search: An internal company agent can use MCP to connect to a "Google Drive Server," a "Slack Server," and a "Notion Server" simultaneously to answer questions like "What was the decision on Project X last week?"16.
GenMedia: The notes mention "MCP Tools for Genmedia," allowing agents to standardize requests to image generation models (Imagen, Veo) just as easily as database queries17.
7. Key Takeaways
Interoperability: MCP shifts the industry away from "walled gardens" of tools toward a shared ecosystem where any tool works with any agent18.
Security: By isolating tool execution into separate "servers," MCP provides a cleaner security boundary than running arbitrary code directly inside the agent's process19.
Future-Proofing: Adopting MCP ensures your tools will work with future LLMs and agentic frameworks (like LangChain or CrewAI) that support the standard, without needing to rewrite integration code20.
Chapter 11: Goal Setting and Monitoring
1. Core Concept & Definition
Definition: This pattern involves equipping agents with explicit, high-level objectives and the mechanisms to track their progress toward them1.
The Shift: It transforms AI agents from reactive systems (which simply respond to the last user prompt) into proactive entities that maintain a sense of purpose over long interaction cycles2.
Purpose: To ensure that an agent's actions remain aligned with the user's original intent, even as the agent navigates complex, multi-step tasks3.
2. The Framework: SMART Goals
The chapter emphasises that vague instructions lead to poor performance. Instead, agent goals should follow the SMART framework:
Specific: Clear and unambiguous objectives.
Measurable: The agent must have a way to know if the goal is met (success criteria)4.
Achievable: Within the agent's capabilities and available tools.
Relevant: Aligned with the user's broader needs.
Time-bound: (Where applicable) executed within reasonable limits.
3. The Monitoring Mechanism
Setting a goal is only half the battle; the agent must also monitor its execution. This involves a continuous feedback loop:
Observation: The agent constantly observes its own actions, the state of the environment, and the outputs of the tools it uses5.
Evaluation: It compares the current state against the defined metrics or success criteria6.
Adaptation: If the monitoring reveals the agent is drifting off-track or failing, the feedback loop triggers a revision of the plan or an escalation to a human7.
4. Hands-On Implementation (Google ADK)
The chapter highlights how this is implemented technically, specifically using the Google Agent Developer Kit (ADK):
Directives: Goals are often conveyed through specific "agent instructions" or system prompts8.
State Management: Monitoring is achieved by tracking the agent's "State." The system checks variables in the session state to determine progress9.
Tool Interactions: The success or failure of tool calls (e.g., "API error" vs. "Success 200 OK") serves as a primary signal for the monitoring system10.
5. Key Takeaways
Proactivity: Goals enable agents to drive a process forward rather than waiting for the next user command11.
Accountability: Defining clear success criteria makes it possible to evaluate (and trust) the agent's performance12.
Resilience: Monitoring allows agents to "self-heal" by detecting failures early and adapting their strategy13.
Foundation for Autonomy: Without the ability to set and monitor its own goals, an agent cannot be truly autonomous14.
Chapter 12: Exception Handling and Recovery
1. Core Concept & Definition
Definition: The Exception Handling and Recovery pattern provides the necessary infrastructure for an AI agent to detect, manage, and recover from unforeseen errors during execution1.
The Shift: It moves agent design from a "happy path" mentality (assuming everything will work) to a Resilient mentality (assuming failures will occur and planning for them).
Goal: To ensure the agent can fail gracefully or self-correct without crashing the entire system or providing a broken experience to the user.
2. The Problem: Fragility in the Real World
Unpredictability: Unlike traditional software where inputs are often strictly typed, agents deal with natural language and external APIs, both of which are highly unpredictable.
Common Failures:
Tool Failures: An API might be down, rate-limited, or return unexpected data formats2.
Hallucination: The model might generate parameters that don't exist or code that doesn't compile.
Loops: An agent might get stuck in a repetitive cycle of trying the same failed action.
3. The Solution: Detect, Handle, Restore
The chapter outlines a three-phase approach to resilience:
Detection: The system must first identify that an error has occurred. This can be programmatic (e.g., catching a
404 Errorfrom an API) or semantic (e.g., a "Critic" agent realizing the output makes no sense)3.Handling: Once detected, the agent employs a specific strategy to manage the error rather than crashing.
Logging: Essential for debugging, the agent records the error state4.
Retries: The simplest mechanism—trying the action again, often with exponential backoff (waiting longer between tries) to handle temporary network blips5.
Fallbacks: If the primary method fails, switch to a secondary, perhaps less precise but more reliable method (Graceful Degradation)6.
Recovery: The final step is restoring the agent to a "stable state" so it can continue processing the rest of the user's request7.
4. Practical Applications & Use Cases
Booking Systems: If a "Flight Search" API times out, the agent catches the exception and informs the user: "I'm having trouble connecting to the airline, let me try a different provider," rather than just outputting a raw error stack trace8.
Code Execution: If an agent writes Python code that fails to run, it captures the error message (traceback), feeds it back into the context, and tries to rewrite the code to fix the bug (Self-Correction)9.
Data Retrieval: If an agent tries to read a file that is locked or missing, it implements a fallback to search for a backup file or ask the user for a new path.
5. Key Takeaways
Robustness: Resilience is not an afterthought; it is a core requirement for agents operating in dynamic production environments10.
Graceful Degradation: It is better to provide a partial answer or a polite failure message than to crash completely11.
Strategies: Effective exception handling combines multiple strategies: logging for visibility, retries for transient errors, and fallbacks for hard failures12.
Trust: Agents that handle errors transparently build higher user trust than those that break silently or confusingly.
Chapter 13: Human-in-the-Loop
1. Core Concept & Definition
Definition: The Human-in-the-Loop (HITL) pattern is a design strategy that deliberately integrates human judgment, oversight, and intervention into the workflow of an AI agent.
The Shift: It moves away from the idea of "fully autonomous" agents (which can be risky) to "augmented" systems where humans and AI work as a team.
Purpose: To combine the speed and scale of AI with the nuance, ethics, and creativity of human cognition1.
2. The Problem: The "Last Mile" of Reliability
Limitations of Autonomy: While agents are powerful, they lack "common sense" and ethical reasoning. They can make confident but catastrophic errors (e.g., a customer service agent promising a refund that violates policy)2.
High-Stakes Risks: In domains like healthcare, law, or finance, a 99% accuracy rate is not enough. The remaining 1% error rate can have legal or life-threatening consequences.
3. The Solution: Structured Intervention
The chapter outlines several mechanisms for integrating humans:
Approval (The Gatekeeper): The agent prepares a draft (e.g., an email or a code commit) but cannot execute it until a human clicks "Approve." This acts as a safety valve.
Feedback (The Teacher): The human provides corrections to the agent's output. The agent doesn't just fix the current mistake; it uses this feedback to update its memory or training data to avoid the error next time (Reinforcement Learning from Human Feedback - RLHF).
Escalation (The Safety Net): The agent autonomously detects when it is confused or when the user is angry (via sentiment analysis) and seamlessly hands the conversation over to a human operator3.
4. The Architecture of HITL
Active vs. Passive:
Active HITL: The human is a necessary step in the loop (e.g., "Review this summary before sending").
Passive HITL: The human monitors the system in the background and only intervenes if an alarm is triggered (e.g., "Alert: Agent confidence score dropped below 70%").
UI/UX Considerations: Effective HITL requires a dedicated interface for the human. It's not just a chat window; it's a dashboard where the human can see the agent's "thought process" (reasoning logs), inspect the data sources, and modify the proposed action4.
5. Practical Applications & Use Cases
Content Moderation: Agents flag potentially toxic content, but a human moderator makes the final decision on banning a user.
Medical Diagnosis: An AI analyzes X-rays and highlights potential anomalies, but a radiologist reviews the images to make the final diagnosis.
Coding Assistants: An agent generates a pull request with code changes, but a senior engineer must review and merge it5.
6. Key Takeaways & Trade-offs
Accuracy vs. Speed: HITL introduces a bottleneck (human speed) into the system. It sacrifices the instant latency of pure AI for higher reliability and trust6.
Cost: Human time is expensive. The goal is to optimize the loop so humans only see the "hard" cases, while the agent handles the routine ones autonomously.
Trust: This pattern is the primary bridge to user adoption. Users are more likely to trust an AI system if they know a human can step in when things go wrong.
Ethics: HITL ensures that accountability remains with a person, which is critical for legal compliance and ethical standards7.
Chapter 14: Knowledge Retrieval (RAG)
1. Core Concept & Definition
Definition: Knowledge Retrieval, widely known as RAG (Retrieval Augmented Generation), is a pattern designed to overcome the static nature of Large Language Models (LLMs). It enables an agent to access, retrieve, and integrate external, real-time, or domain-specific data into its responses1.
The Shift: It moves the LLM from relying solely on its pre-trained memory (which can be outdated or generic) to utilizing a dynamic "open-book" approach where it can look up information it wasn't trained on2.
2. The Problem: Static Knowledge & Hallucination
Knowledge Cutoffs: LLMs are confined to the data they were trained on. They cannot natively access real-time events (like current stock prices) or private data (like a specific company's internal wiki)3.
Hallucination: When an LLM doesn't know an answer, it may confidently invent one. Without a source of truth to check against, the model prioritizes plausibility over factual accuracy4.
3. The Solution: Grounding in Reality
RAG solves these issues by retrieving relevant facts before generating an answer.
Grounding: By providing the model with accurate context from an external source, the agent's output is "grounded" in verifiable data rather than statistical probability5.
Verification: A key advantage of RAG is the ability to provide citations. Because the agent retrieves specific documents to answer the query, it can point the user to the exact source of the information, building trust6.
4. Core Technical Components
The chapter outlines the fundamental building blocks required to implement RAG:
Embeddings: These are numerical representations of text (words, phrases, or documents) converted into vectors (lists of numbers). Embeddings capture the semantic meaning of the text, allowing the system to understand that "canine" and "dog" are related concepts even if the words are different7.
Vector Search & Text Similarity: To find the right information, the system compares the embedding of the user's query with the embeddings of the stored documents. This allows for semantic search (finding matches based on meaning) rather than just keyword matching88.
Chunking: Large documents are broken down into smaller, manageable pieces (chunks) before being embedded. This ensures that the retrieval system finds the specific paragraph relevant to the query rather than retrieving an entire massive document9.
5. Practical Applications & Use Cases
Enterprise Search: An agent answering questions about internal company policies (e.g., "What is the travel reimbursement limit?") by retrieving the latest PDF from the HR portal10.
Real-Time Decision Making: An e-commerce agent checking current inventory levels in a database before confirming a customer's order, ensuring it doesn't sell an out-of-stock item11.
Customer Support: An agent pulling the specific user manual or troubleshooting guide for a product to answer a technical question accurately12.
6. Key Takeaways
Accuracy: RAG is the primary pattern for ensuring factual accuracy and reducing hallucinations in agentic systems13.
Extensibility: It allows developers to "teach" an agent new information simply by updating its database, without needing to retrain the expensive LLM itself14.
Bridge to Reality: RAG transforms agents from creative writers into reliable research assistants capable of handling private and changing data15.
Chapter 15: Inter-Agent Communication (A2A)
1. Core Concept & Definition
Definition: The Inter-Agent Communication (A2A) pattern defines a standardized protocol that enables AI agents to discover, connect, and collaborate with one another, regardless of the underlying framework or platform they were built on1.
The Goal: To overcome the "inherent isolation" of individual agents. Instead of being standalone silos, A2A allows them to form a cooperative network2.
Interoperability: It provides a common HTTP-based framework that allows an agent built in Google ADK to seamlessly talk to an agent built in LangGraph or CrewAI3.
2. Key Architecture Components
The chapter details the specific mechanisms that make this communication possible:
Agent Card: This serves as the agent's "identity card" or directory entry. It allows other agents to discover who it is, what it does, and how to connect to it4.
Communication Methods:
Request/Response (Polling): A standard mechanism where one agent asks for information and waits for a reply55.
Webhooks: An event-driven mechanism where an agent can subscribe to updates and be notified automatically when something changes, reducing the need for constant checking66.
State Management: The protocol allows agents to share and maintain "state" (context) during interactions, ensuring that follow-up questions or tasks retain the necessary history7.
3. A2A vs. MCP
The chapter draws a critical distinction between two major protocols:
A2A (Agent-to-Agent): A high-level protocol for managing tasks and workflows between different agents (e.g., a "Manager" agent assigning a task to a "Writer" agent)8.
MCP (Model Context Protocol): A standardized interface for LLMs to access external resources and tools (e.g., an agent connecting to a database or a PDF reader)9.
Summary: A2A is for collaboration; MCP is for tool integration.
4. Tools & Observability
Visualization: The chapter highlights tools like Trickle AI, which help developers visualize the invisible traffic between agents. This is crucial for debugging multi-agent systems, allowing you to track message flows and identify bottlenecks10.
Remote Agents: The pattern encourages a modular architecture where specialized agents can operate independently on different ports or servers, enhancing scalability111111.
5. Key Takeaways
Standardization: A2A creates a "common language" for agents, preventing vendor lock-in and allowing diverse systems to work together12.
Scalability: By decoupling agents into independent services that communicate via HTTP, the system becomes significantly more scalable and easier to distribute13.
Ecosystem: It shifts the focus from building a single "super-agent" to building an ecosystem of specialised, interoperable services14.
Chapter 16: Resource-Aware Optimization
1. Core Concept & Definition
Definition: Resource-Aware Optimization is a pattern designed to make AI agents efficient, cost-effective, and fast. It ensures that agents do not just "solve the problem" but do so using the optimal amount of computational power, time, and money.
The Shift: This moves from a "brute force" approach (using the most powerful, expensive model for everything) to a Smart Selection approach (using the right tool for the job).
Goal: To maximize performance while minimizing latency (speed) and API costs.
2. The Problem: The "One Size Fits All" Trap
Overkill: Using a massive reasoning model (like Gemini 1.5 Pro or GPT-4) to answer a simple question like "What is 2+2?" is wasteful. It is slow and expensive.
Cost & Latency: In production systems with millions of users, inefficiencies compound. A slight delay or extra cost per query can make a service unviable.
3. Key Optimisation Strategies
The chapter outlines several techniques to optimize agent performance:
Dynamic Model Switching:
Concept: The agent analyzes the complexity of the user's request before answering.
Action: Simple queries are routed to smaller, faster, cheaper models (e.g., Gemini Flash). Complex reasoning tasks are routed to larger, more capable models (e.g., Gemini Pro).
Result: Drastic reduction in average cost and latency without sacrificing quality for hard questions1.
Adaptive Tool Use:
Concept: Instead of giving an agent every possible tool (which confuses it and uses up context tokens), the system dynamically gives it only the tools relevant to the current task.
Example: If the user asks about "Weather," the agent is temporarily given only weather-related tools, not financial or coding tools2.
Contextual Pruning:
Concept: Agents often suffer from "context bloat" where the conversation history gets too long. Pruning involves summarizing or deleting older, less relevant parts of the history to keep the "context window" clean and focused. This speeds up processing and improves accuracy3.
Proactive Resource Prediction:
Concept: Advanced agents can predict if a task will be "hard" or "easy" and allocate resources (like memory or CPU time) accordingly before starting4.
4. Advanced Techniques
Context Caching: Reusing the "processed" state of a long document (like a 500-page manual) across multiple queries so the model doesn't have to re-read it every time.
Graceful Degradation: If the primary high-performance model is too slow or down, the system automatically switches to a lesser model to ensure the user still gets an answer, even if it's less detailed5.
5. Practical Applications & Use Cases
Customer Support Bots: A bot uses a cheap model to say "Hello, how can I help?" but switches to an expensive model to debug a complex technical error.
Data Analysis: An agent uses a fast model to scan thousands of documents for keywords, then uses a smart model to deeply analyze only the top 10 relevant documents.
Mobile Agents: On-device agents (running on a phone) must be highly resource-aware to save battery life, offloading only the hardest tasks to the cloud.
6. Key Takeaways
Efficiency is a Feature: Speed and cost are not just operational details; they are core parts of the user experience.
Hybrid Approach: The best systems are often hybrid, combining small, fast models for routing/classification with large models for final reasoning.
Sustainability: Optimizing resources also reduces the energy footprint of AI systems6.
kBEqs: "chapter 17 agentic design patterns book notes" "reasoning techniques chapter 17 detailed notes" "chain of thought reasoning agentic design patterns chapter 17" "tree of thoughts agentic design patterns" "react reasoning framework agentic design patterns"
Chapter 17: Reasoning Techniques
1. Core Concept & Definition
Definition: Reasoning Techniques refer to the advanced cognitive frameworks that enable AI agents to "think" before they act. Instead of simply predicting the next word based on probability, these techniques force the model to explicitly break down complex problems, plan multiple steps, and evaluate its own logic11.
The Shift: This moves agent capabilities from intuitive response (fast, potentially inaccurate) to deliberate reasoning (slower, more accurate, and structured)222.
Goal: To enable agents to solve multi-step problems, handle ambiguity, and improve reliability by making their "thought process" visible and structured333.
2. Key Reasoning Architectures
The chapter details several foundational frameworks that define how modern agents reason:
Chain of Thought (CoT):
Mechanism: The agent is prompted to "think step-by-step" or produce a sequence of intermediate reasoning steps before giving the final answer4.
Benefit: It allows the model to decompose hard problems (e.g., math word problems) into manageable pieces, significantly reducing logical errors5.
Tree of Thoughts (ToT):
Mechanism: A non-linear approach where the agent explores multiple possible "branches" of reasoning simultaneously. It creates a tree of potential next steps, evaluates each one, and can backtrack if a branch looks unpromising66.
Analogy: Like a chess player considering three different moves and simulating the outcome of each before picking one7.
ReAct (Reason + Act):
Mechanism: This is the "gold standard" for autonomous agents. It combines Reasoning (thinking about what to do) with Acting (using a tool).
Loop: Thought Action (Tool Call) Observation (Tool Result) Thought ...888.
Significance: It grounds reasoning in reality by allowing the agent to update its plans based on real-world feedback from tools9.
Graph of Debates (GoD) / Chain of Debates:
Mechanism: A multi-agent approach where different agents (or the same agent playing different roles) argue for and against a specific solution10.
Process: "Proposer" agent suggests a plan, "Critic" agent attacks it, and a "Judge" agent decides. This adversarial process filters out hallucinations and weak logic11.
3. Advanced Concepts: Scaling Inference
Inference-Time Compute: The chapter discusses the "Scaling Inference Law," which suggests that an agent's performance can be improved not just by making the model bigger (training time) but by giving it more time to "think" during execution (inference time)12.
Implication: Allowing an agent to generate thousands of internal reasoning steps (hidden from the user) can allow a smaller model to outperform a larger model that answers instantly13.
4. Practical Applications
Deep Research: An agent autonomously investigating a topic by generating a search query, reading the result, realizing it's insufficient, generating a new query, and synthesising the findings—all without human help141414.
Complex Coding: An agent writing a software feature doesn't just output code; it plans the architecture, writes the code, writes a test to verify it, sees the test fail, reasons about the error, and fixes the code (Self-Correction)15.
5. Key Takeaways
Metacognition: Reasoning techniques give agents a form of "metacognition" (thinking about thinking), allowing them to spot their own errors16.
Transparency: By making the reasoning process explicit (e.g., via CoT), developers can debug why an agent made a mistake, rather than staring at a black box17.
Trade-off: Advanced reasoning improves accuracy but increases latency (time to answer) and cost (more tokens used). It should be used for complex tasks, not simple queries18.
Chapter 18: Guardrails / Safety Patterns
1. Core Concept & Definition
Definition: Guardrails (Safety Patterns) are the "protective layers" or mechanisms designed to ensure that autonomous agents operate within safe, ethical, and reliable boundaries1.
Purpose: As agents become more autonomous and integrated into critical systems, they require strict controls to prevent them from generating harmful content, executing dangerous actions, or revealing sensitive information. The goal is not to disable the agent, but to ensure it remains "robust, trustworthy, and beneficial"2.
2. Key Implementation Layers
The chapter breaks down guardrails into several distinct stages of the agent's workflow3:
Input Validation & Sanitization:
Goal: Filter out malicious or irrelevant requests before they reach the core LLM.
Example: Detecting "jailbreak" attempts (prompts designed to bypass safety rules) or blocking requests that violate usage policies (e.g., asking for illegal advice)4.
Behavioral Constraints (Prompt-Level):
Goal: Use system instructions to strictly define what the agent cannot do.
Example: "Do not provide financial advice," or "If asked about politics, politely decline"5.
Tool Use Restrictions:
Goal: Limit the agent's physical capabilities.
Example: An agent might have "read-only" access to a database but be blocked from "delete" or "update" commands to prevent accidental data loss6.
Output Filtering / Post-Processing:
Goal: Analyze the agent's final response before showing it to the user.
Example: A secondary "Reviewer" model checks the response for toxicity, bias, or PII (Personal Identifiable Information) leakage. If the check fails, the response is blocked or regenerated7.
Human Oversight (Human-in-the-Loop):
Goal: For high-stakes actions, require human approval.
Example: The agent drafts an email, but a human must click "Send"8.
3. Practical Applications & Use Cases
Educational Tutors: A guardrail ensures the AI stays on the curriculum and filters out inappropriate topics during conversations with students9
Legal/Medical Assistants: Explicitly preventing the agent from giving definitive professional advice (unauthorized practice of law/medicine) and forcing it to add disclaimers10.
HR Recruitment: Filtering out discriminatory language in job descriptions or candidate screenings to ensure fairness11.
Social Media: Automatically flagging hate speech or graphic content before it is posted12.
4. Hands-On Code Example (CrewAI)
The chapter provides a code example using CrewAI to implement a "Policy Enforcer" agent13:
Architecture: A dedicated agent (Policy Enforcer) sits in front of the primary AI.
Process:
The user's input is sent to the Policy Enforcer first.
The Enforcer uses a strict system prompt (defining rules like "No political commentary" or "No competitor mentions").
It outputs a structured JSON decision:
{"compliance_status": "compliant", "evaluation_summary": "..."}.Pydantic Guardrail: The code uses Pydantic to validate that the Enforcer's output is actually valid JSON and follows the schema.
Action: If "compliant," the request passes to the main agent. If "non-compliant," it is blocked with an explanation.
5. Key Takeaways
Defense in Depth: Effective safety is not just one prompt; it is a multi-layered approach (Input -> Prompt -> Tool -> Output)14.
Specialized Models: Smaller, faster models (like Gemini Flash) are ideal for guardrail tasks because they add minimal latency to the overall request15.
Trust: Guardrails are the essential component that allows organizations to trust agents with real-world tasks, protecting both the user and the company's reputation16.
Chapter 19: Evaluation and Monitoring
1. Core Concept & Definition
Definition: Evaluation and Monitoring is the systematic process of assessing an agent's performance, reliability, and cost-effectiveness. It shifts the development process from subjective "vibes-based" checking (e.g., "it looks correct") to rigorous, quantitative measurement1.
The Shift: Unlike traditional software, where inputs and outputs are deterministic ( always equals ), AI agents are probabilistic and non-deterministic. This chapter focuses on methodologies to validate "fuzzy" outputs and track behavior over time2.
Goal: To ensure agents meet specific requirements for accuracy, latency, and safety before deployment and to detect anomalies (drift) once they are live3.
2. The Problem: Evaluating the Unpredictable
Non-Determinism: An agent might answer "The weather is sunny" today and "It's a bright, sunny day" tomorrow. A standard string comparison test (
assert output == expected) would fail, even though both are correct.Trajectory Complexity: It is not just about the final answer. An agent might get the right answer by guessing, or it might take a very inefficient path (looping 10 times) to get there. Evaluating the process is as important as evaluating the result4.
3. Key Evaluation Metrics
The chapter highlights that evaluation must go beyond simple accuracy to include operational metrics:
Response Accuracy: Does the output match the "ground truth" or "golden answer"? This is often measured using an LLM-as-a-Judge (asking a stronger model to grade the agent's answer)5.
Latency: How much time does the agent take to complete a task? High latency kills user experience6.
Token Usage (Cost): Monitoring how many tokens are consumed per run is critical for financial viability7.
Trajectory Analysis: Examining the sequence of steps (thoughts actions observations) the agent took. This helps identify if the agent is using tools correctly or hallucinating steps8.
4. Methodologies & Testing Strategies
Golden Datasets (Evalsets): Developers create a comprehensive set of input scenarios with known "correct" outcomes (behavioral expectations). The agent is run against this set to calculate a pass rate9.
A/B Testing: Running two different versions of an agent (e.g., v1 with Prompt A vs. v2 with Prompt B) simultaneously to see which performs better on real-world traffic10.
Drift Detection: Continuous monitoring to spot if the agent's performance degrades over time (e.g., due to changes in the underlying model or data sources)11
Contractor Model: The chapter introduces the metaphor of agents as "Contractors" bound by formal agreements. They negotiate tasks and "self-validate" their work against explicit terms before considering a job done12.
5. Tools & Implementation (Google ADK)
Structured Evaluation: The Google Agent Developer Kit (ADK) provides specific structures for defining tests.
Unit Tests: For testing individual tools or functions in isolation.
Integration Tests: Using
evalsetfiles to test the full agent workflow against complex prompts13.
Visualisation: The framework often includes a web-based UI to visually inspect the agent's execution traces, making it easier to debug where the logic diverged from the ideal path14.
6. Key Takeaways
Holistic View: You cannot just test the output; you must test the journey (trajectory) and the cost (tokens/latency)15.
Automation: Manual testing scales poorly. Automated evaluation pipelines (using Evalsets) are required for professional agent engineering16.
Accountability: By implementing rigorous monitoring, agents transform from unpredictable novelties into accountable systems that can be trusted with business-critical tasks17.
Chapter 20: Prioritisation
1. Core Concept & Definition
Definition: The Prioritization pattern equips an AI agent with the ability to autonomously evaluate and rank multiple potential actions or goals. It enables the agent to decide what to do next when faced with competing demands, rather than simply executing tasks in the order they were received1.
The Shift: It moves the agent from linear execution (doing task A, then B) to strategic decision-making (determining that B is more critical than A and should be done first)2.
Goal: To ensure the agent operates effectively in dynamic environments where resources are limited and goals may conflict3.
2. The Problem: Resource Constraints & Conflict
Overload: In complex scenarios, agents often encounter numerous potential actions or user requests simultaneously. Without a filter, the agent might try to do everything at once or focus on low-value tasks4.
Inefficiency: Lacking a prioritization mechanism leads to operational delays, reduced efficiency, and failure to achieve the primary objectives5.
Dynamic Changes: Real-world conditions change rapidly. A task that was important five minutes ago might be irrelevant now, and a static plan fails to account for this6.
3. The Solution: Criteria-Based Ranking
The pattern involves assessing tasks against specific criteria to determine their execution order:
Urgency: How soon must this task be done? (Time sensitivity) 7.
Importance: How critical is this task to the overall goal? (Value impact) 8.
Dependencies: Does this task block others? (Logical order) 9.
4. Mechanism: Dynamic Re-Prioritization
Real-Time Adaptation: A key feature of this pattern is dynamic re-prioritization. The agent doesn't just prioritize once at the beginning; it continuously re-evaluates its list as new information arrives or conditions change10.
Levels of Operation:
Strategic Level: Prioritising overarching objectives (e.g., "Focus on customer satisfaction over speed").
Tactical Level: Prioritising immediate actions (e.g., "Call the API before parsing the file")11.
5. Practical Applications & Code Context
Ambiguity Handling: The chapter's code example illustrates an agent interpreting ambiguous requests and autonomously selecting the most relevant tools or actions based on a priority logic12.
Resource Management: It ensures the agent focuses its limited computational resources (tokens, API calls) on the tasks that move the needle the most13.
6. Key Takeaways
Proactivity: Prioritization transforms an agent from a passive executor into a proactive strategist14.
Human-Like Reasoning: By weighing conflicting goals, the agent demonstrates a more sophisticated, human-like reasoning process15.
Robustness: This pattern is essential for building "production-grade" agents that don't choke under pressure when faced with multiple inputs16.
Chapter 21: Exploration and Discovery
1. Core Concept & Definition
Definition: The Exploration and Discovery pattern enables intelligent agents to proactively venture into unfamiliar territories to uncover new possibilities, rather than just optimizing within a known solution space1.
The Shift: It represents a move from exploitation (using existing knowledge to solve a problem) to exploration (actively seeking new information to generate novel insights)2.
Goal: To identify "unknown unknowns" and generate new hypotheses or strategies in open-ended environments3.
2. The Problem: The Limits of Optimization
Reactive vs. Proactive: Standard agents are typically reactive; they wait for a prompt and optimize a response based on pre-existing data. This fails in domains where the solution isn't already "out there" to be found but needs to be discovered or invented4.
Stagnation: Without the ability to explore, an agent is limited to iterating on what is already known, making it unsuitable for true innovation or scientific discovery5.
3. The Solution: Autonomous Research Frameworks
The pattern involves agents that can formulate hypotheses, design experiments to test them, and iterate based on the results.
Workflow:
Hypothesis Generation: The agent scans existing literature or data to propose a new idea.
Experimentation: It designs a code experiment or simulation to test the idea.
Analysis: It reviews the results to validate or reject the hypothesis.
Evolution: It refines its understanding and starts the loop again6.
4. Hands-On Code Example: Agent Laboratory
The chapter highlights Agent Laboratory, a framework developed to augment human scientists7.
Architecture: It uses a multi-agent system where specialized roles collaborate:
Ph.D. Student / Researcher Agent: Conceptualizes the experiment and performs literature reviews8.
Software Engineer Agent: Translates the research plan into simple, executable code for data preparation9.
Machine Learning Engineer Agent: Writes the actual model training code10.
AgentRxiv: A decentralized repository mentioned in the chapter where agents can deposit and retrieve research outputs, allowing them to build upon previous findings cumulatively11.
5. Practical Applications & Use Cases
Scientific Research: Systems like Google Co-Scientist use this pattern to autonomously design and execute experiments, accelerating the pace of discovery in fields like chemistry or biology12.
Market Analysis: Exploring "blue ocean" market strategies where no data currently exists.
Creative Content: Generating entirely new genres of art or writing rather than mimicking existing styles13.
6. Key Takeaways
Innovation: This pattern is essential for tasks requiring the generation of novel hypotheses rather than just answering questions14.
Augmentation: The goal is not to replace human researchers but to offload the computationally intensive tasks (like literature review and initial code generation) so humans can focus on high-level conceptualization15.
Rule of Thumb: Use this pattern when the solution space is not fully defined and you need the agent to find "unknown unknowns"16.
Last updated