When to Build Agents

Building AI agents is powerful, but not every problem needs an agent. Use this guide to decide when Vanna Agents is the right fit and how to design agents that stay safe, efficient, and trustworthy.

Deciding When to Use Agents

✅ Use Agents When

1. The task has ambiguous, multi-step complexity

• Requests can branch into different tool sequences
• Multiple data sources or verification steps are required
• Example: “Analyze sales data, compare to forecast, and propose next actions”

2. High-value outcomes justify LLM spend

• Agents consume more tokens than single-shot prompts
• Use agents when the output unblocks a human or drives revenue
• Manage cost with AgentConfig(max_tokens=..., max_tool_iterations=...)

3. You can verify results quickly

• Database queries validate against schemas
• Code changes run tests or linters
• File edits are diff-able and reversible

4. You need controlled autonomy

• Agents make progress independently but within guardrails
• Throttle with max_tool_iterations, approvals, and lifecycle hooks

❌ Skip Agents When

• The workflow is a deterministic decision tree
• A single tool call or prompt solves the task
• Mistakes are irreversible or extremely costly to detect

Designing Effective Agents

Start Simple: Compose the Essentials

from vanna import Agent, AgentConfig, MockLlmService, ToolRegistry, MemoryConversationStore

agent = Agent(
    llm_service=MockLlmService("I'm ready to help."),
    tool_registry=ToolRegistry(),
    conversation_store=MemoryConversationStore(),
    config=AgentConfig(stream_responses=False),
)

Design principle: Keep each dependency lightweight. Complexity should come from the composition, not from bespoke orchestration logic.

Share the Same Agent Backbone

from vanna import Agent, AgentConfig
from vanna.integrations.anthropic import AnthropicLlmService

def build_agent(tool_registry):
    return Agent(
        llm_service=AnthropicLlmService(model="claude-sonnet-4"),
        tool_registry=tool_registry,
        config=AgentConfig(max_tool_iterations=6),
    )

sql_agent = build_agent(sql_registry)
filesystem_agent = build_agent(filesystem_registry)

Swap registries and configs; keep the orchestration identical for predictable behavior.

Add UX Transparency Early

async for component in agent.send_message(user=user, message=prompt):
    if component.rich_component:
        if component.rich_component.type.name == "STATUS_BAR":
            update_status(component.rich_component)
        if component.rich_component.type.name == "TASK_TRACKER":
            render_checklist(component.rich_component)

    if component.simple_component:
        log_text(component.simple_component.text)

Real-time status, tasks, and progress bars turn opaque reasoning into a clear workflow.

Understanding Agent Context Limits

Agents only see what you feed them in the request:

from vanna import ToolContext

context = ToolContext(
    user=user,
    conversation_id="conv-123",
    request_id="req-456",
    metadata={"workspace_path": "/repos/acme"},
)

Design implications:

1. Keep tool descriptions concise; they’re in the LLM prompt budget.
2. Use dual outputs so the LLM gets summaries while humans see full data.
3. Pass critical routing info (workspace, org, locale) via context.metadata.

from vanna import ToolResult, UiComponent, SimpleTextComponent, DataFrameComponent

return ToolResult(
    success=True,
    result_for_llm=f"Query returned {len(rows)} rows",
    ui_component=UiComponent(
        rich_component=DataFrameComponent.from_records(rows, title="Top Customers"),
        simple_component=SimpleTextComponent(text="10 rows returned"),
    ),
)

Tool Design Best Practices

1. Strong Typing with Pydantic

from typing import Type
from pydantic import BaseModel, Field
from vanna import Tool, ToolContext, ToolResult

class CalculatorArgs(BaseModel):
    operation: str = Field(description="add | subtract | multiply | divide")
    a: float
    b: float

class CalculatorTool(Tool[CalculatorArgs]):
    @property
    def name(self) -> str:
        return "calculator"

    def get_args_schema(self) -> Type[CalculatorArgs]:
        return CalculatorArgs

    async def execute(self, context: ToolContext, args: CalculatorArgs) -> ToolResult:
        ...

2. Meaningful Errors

if args.operation not in {"add", "subtract", "multiply", "divide"}:
    message = f"Unsupported operation '{args.operation}'"
    return ToolResult(success=False, result_for_llm=message, error="INVALID_OPERATION")

3. Test Tools in Isolation

context = ToolContext(user=test_user, conversation_id="test", request_id="req")
result = await CalculatorTool().execute(context, CalculatorArgs(operation="add", a=5, b=3))
assert result.success and result.result_for_llm == "8"

Permission-Based Tool Access

from typing import List, Type
from pydantic import BaseModel, Field
from vanna import Tool, ToolContext, ToolResult, User

class SensitiveArgs(BaseModel):
    resource_id: str = Field(description="Identifier to mutate")

class SensitiveTool(Tool[SensitiveArgs]):
    @property
    def required_permissions(self) -> List[str]:
        return ["admin", "write_data"]

    def get_args_schema(self) -> Type[SensitiveArgs]:
        return SensitiveArgs

    async def execute(self, context: ToolContext, args: SensitiveArgs) -> ToolResult:
        ...

user = User(id="analyst", permissions=["read_data"])
available_tools = registry.get_schemas(user)  # Automatically filtered

The registry enforces permissions before execution—no extra checks required.

Cost and Performance Management

Token Budgets & Iteration Limits

config = AgentConfig(
    max_tokens=2000,
    max_tool_iterations=6,
    temperature=0.6,
)

Agent enforces max_tool_iterations; when exceeded it returns the best-so-far response instead of looping forever.

Async by Design

All tools are async—a good pattern is to delegate I/O or batching inside the tool. If you need parallel execution, implement a custom registry or lifecycle hook that schedules tasks concurrently while respecting shared state.

Use Cases Where Vanna Shines

1. Database analysis – RunSqlTool, VisualizeDataTool, and custom summarizers deliver validated insights quickly.
2. Code generation & refactoring – File system tools plus test runners make results easy to verify.
3. Multi-step data workflows – Chain SQL, Python, and visualization tools with deterministic checkpoints.
4. Interactive onboarding – Drive configuration flows with OTP checks, validation tools, and clear status cards.

Deployment Checklist

• Task requires reasoning across steps or systems
• max_tokens & max_tool_iterations tuned for your budget
• Tools validated independently
• required_permissions applied to sensitive tools
• Progress UI hooks in place (status, tasks, artifacts)
• Conversation storage configured with retention policy
• Evaluation suite ready for regression testing

Next Steps

• Follow the Quick Start to stand up your first agent
• Deep-dive into the architecture when you need custom integrations
• Compare deployment options in OSS vs. hosted

Remember: Start with the smallest viable agent, prove value, then harden with permissions, observability, and evaluations. Vanna Agents gives you the knobs—you decide how much autonomy to grant.