Agno Testing & Evaluation

AgentCI provides comprehensive Agno testing and evaluation for developers building multi-model agent applications. Our automated Agno CI platform discovers your Agent instances across OpenAI and Anthropic models, validates Toolkit classes, and runs thorough evaluations - without requiring any code changes.

Agno agent evals across OpenAI and Anthropic models

AgentCI automatically discovers and evaluates Agno agents, including:

  • Agent discovery: Agent() instances using OpenAI and Anthropic Claude models
  • Evaluation types: Accuracy, safety, performance testing, and Toolkit validation
  • CI/CD integration: Automated testing on pull requests via GitHub
  • Zero code changes: No test harnesses or modifications to your Python code required

Supported Agent Patterns

Agent with OpenAI Model

from agno.agent import Agent
from agno.models.openai import OpenAIChat

agent = Agent(
    name="research_assistant",
    model=OpenAIChat(id="gpt-4o"),
    instructions="You are a research assistant that helps find information.",
    description="Research assistant with web search capabilities.",
    tools=[search_web, get_weather],
    markdown=True
)

Agent with Anthropic Model

from agno.agent import Agent
from agno.models.anthropic import Claude

agent = Agent(
    name="task_coordinator",
    model=Claude(id="claude-sonnet-4-5"),
    instructions="You coordinate tasks and delegate to specialized tools.",
    description="Coordinates multiple specialized tasks.",
    tools=[search_web, calculate],
    markdown=True
)

Supported Tool Patterns

Plain Functions

import json

def get_weather(city: str) -> str:
    """Get current weather information for a city."""
    return json.dumps({
        "city": city,
        "temperature": 72,
        "condition": "sunny"
    })

@tool Decorator

from agno.tools import tool

@tool
def calculate(expression: str) -> str:
    """Evaluate a mathematical expression."""
    try:
        result = eval(expression)
        return json.dumps({"expression": expression, "result": result})
    except Exception as e:
        return json.dumps({"error": str(e)})

Toolkit Classes

from agno.tools import Toolkit

class CustomSearchTools(Toolkit):
    """Custom search toolkit."""

    def __init__(self, all: bool = False, **kwargs):
        tools = []
        if all:
            tools.append(self.search_documents)
        super().__init__(name="custom_search", tools=tools, **kwargs)

    def search_documents(self, query: str) -> str:
        """Search through documents."""
        return json.dumps({
            "query": query,
            "results": ["doc1.pdf", "doc2.txt"]
        })

What Gets Auto-Discovered

AgentCI automatically finds:

  • Agent() instances with name, model, instructions, and tools parameters
  • Both OpenAI and Anthropic model configurations
  • Plain Python functions with docstrings
  • Functions decorated with @tool
  • Toolkit subclasses with methods

Agno's flexible tool definition approach works seamlessly with AgentCI's auto-discovery.

Next Steps