Manual Training Guide

While Vanna’s agent memory learns automatically (by default when running as an admin user) from successful interactions, you can also manually train your agent to improve its performance and ensure it has the right knowledge for your specific use cases.

How Agent Memory Works

Vanna agents learn by storing successful tool usage patterns in vector memory. When users ask similar questions later, the agent retrieves and adapts these proven examples. However, you can seed this memory with specific examples to accelerate learning and ensure coverage of important queries.

In-Chat Training (Recommended)

The easiest way to train your agent is through natural conversation in the chat interface. Simply ask questions and provide feedback:

Training Through Conversation

1. Ask a question in the chat interface
2. Review the response - if it’s correct, the agent automatically learns from it
3. Provide feedback - if the response needs improvement, tell the agent what went wrong
4. Guide corrections - the agent will remember successful corrections

Example conversation:

User: Show me total sales by country
Agent: [Runs incorrect SQL query]
Agent: [Corrects and runs proper query]
Agent: [Saves the correct query to AgentMemory]

The agent automatically saves this corrected interaction to memory for future reference.

When to Use In-Chat Training

• Best for: Business logic, complex queries, domain-specific terminology
• Advantages: Natural, contextual learning that captures intent
• Agent determines: What aspects are most important to remember

Manual Question-SQL Pair Training

For structured training data, you can directly add question-SQL pairs to the agent’s memory:

from vanna.agent_memory import ToolMemory

# Create training examples
training_examples = [
    ToolMemory(
        question="What are our top 5 selling products?",
        tool_name="run_sql",
        args={
            "sql": """
            SELECT p.ProductName, SUM(od.Quantity) as TotalSold
            FROM Products p
            JOIN OrderDetails od ON p.ProductID = od.ProductID
            GROUP BY p.ProductName
            ORDER BY TotalSold DESC
            LIMIT 5
            """
        }
    ),
    ToolMemory(
        question="Show me customer retention rates",
        tool_name="run_sql",
        args={
            "sql": """
            SELECT
                YEAR(OrderDate) as Year,
                COUNT(DISTINCT CASE WHEN CustomerID IN (
                    SELECT DISTINCT CustomerID FROM Orders
                    WHERE YEAR(OrderDate) = YEAR(o.OrderDate) - 1
                ) THEN CustomerID END) * 100.0 / COUNT(DISTINCT CustomerID) as RetentionRate
            FROM Orders o
            GROUP BY YEAR(OrderDate)
            """
        }
    )
]

# Add to agent memory
for example in training_examples:
    await agent.agent_memory.save_tool_usage(
        question=example.question,
        tool_name=example.tool_name,
        args=example.args,
        context=ToolContext(user=your_user),
        success=True
    )

Manual Documentation Addition

Add domain knowledge, business rules, and context that helps the agent understand your data:

from vanna.agent_memory import DocumentationMemory

# Add business context
documentation = [
    {
        "title": "Customer Segmentation Rules",
        "content": """
        Customers are segmented as:
        - VIP: Orders > $10,000 in past year
        - Regular: Orders > $1,000 in past year
        - New: First order within 30 days
        - Inactive: No orders in 6+ months

        Always use these definitions in customer analysis queries.
        """
    },
    {
        "title": "Product Categories",
        "content": """
        Product categories and their business meaning:
        - Electronics: High-margin, warranty required
        - Clothing: Seasonal inventory, frequent discounts
        - Books: Low overhead, long tail sales
        - Home & Garden: High shipping costs, regional preferences
        """
    }
]

# Store documentation in memory
for doc in documentation:
    await agent.agent_memory.save_documentation(
        title=doc["title"],
        content=doc["content"],
        context=ToolContext(user=your_user)
    )

When to Use Manual Documentation

• Best for: Business rules, terminology definitions, data relationships, company policies
• Advantages: Provides context that SQL alone cannot convey
• Considerations: Most effective when referenced during tool selection and response generation

Best Practices

1. Start with In-Chat Training

Let the agent learn naturally through conversation first, then supplement with manual training for gaps.

2. Focus on High-Value Queries

Prioritize training examples for:

• Frequently asked questions
• Complex business logic
• Critical KPIs and reports
• Common user mistakes

3. Include Context, Not Just SQL

When manually training, include comments explaining:

• Why this query structure was chosen
• Business rules being applied
• Edge cases handled

4. Test and Iterate

After adding training data:

• Test with similar but different questions
• Verify the agent adapts the examples correctly
• Remove or update examples that cause confusion

5. Balance Manual and Automatic Learning

• Manual training: For precision and coverage
• Automatic learning: For adaptation and scale
• Hybrid approach: Best results for most use cases

See Also

• Tool Memory - How agent memory works internally
• Built-in Tools - Available tools to train
• Custom Tools - Creating tools for specific training needs