AgentMail Docs

Appearance

Smart Email Labeling Agent ​

Fresh 🌱

Build an AI-powered agent that automatically classifies and labels incoming emails across multiple dimensions

Overview ​

Learn how to build an intelligent email classification agent that uses AI to automatically analyze and label incoming emails. This intermediate example showcases AgentMail's powerful labeling feature combined with OpenAI's GPT-4o-mini to create a sophisticated inbox automation system.

What You'll Build ​

By the end of this guide, you'll have a working smart labeling agent that:

  1. Receives incoming emails to a dedicated AgentMail inbox
  2. Analyzes each email with AI across 4 dimensions:
    • Sentiment: positive, neutral, or negative
    • Category: question, complaint, feature-request, bug-report, or praise
    • Priority: urgent, high, normal, or low
    • Department: sales, support, billing, or technical
  3. Automatically applies labels to each email for easy filtering
  4. Handles failures gracefully with retry logic and validation

Here's what happens when an email arrives:

Email: "Your product crashed! I need help ASAP!"
         ↓
   AI Analysis
         ↓
   Applied Labels:
   • negative
   • complaint
   • urgent
   • support

Prerequisites ​

Before you begin, make sure you have:

Required:

  • Python 3.8 or higher installed
  • An AgentMail account and API key
  • An OpenAI API key (for AI classification)
  • An ngrok account

Project Setup ​

Step 1: Create Project Directory ​

Create a new directory for your agent:

bash
mkdir smart-labeling-agent
cd smart-labeling-agent

Step 2: Create the Agent Code ​

Create a file named agent.py and paste the following code:

python
"""
Smart Email Labeling Agent

An AI-powered email classification agent that automatically analyzes incoming
emails across multiple dimensions and applies appropriate labels.
"""

import os
import json
import time
from dotenv import load_dotenv

load_dotenv()

from flask import Flask, request, Response
import ngrok
from agentmail import AgentMail
from openai import OpenAI

# Configuration
PORT = int(os.getenv("PORT", "8080"))
INBOX_USERNAME = os.getenv("INBOX_USERNAME", "smart-labels")
WEBHOOK_DOMAIN = os.getenv("WEBHOOK_DOMAIN")

# Initialize
app = Flask(__name__)
client = AgentMail()
openai_client = OpenAI()

def setup_agentmail():
    """Create inbox and webhook with idempotency."""
    # Create inbox
    try:
        inbox = client.inboxes.create(
            username=INBOX_USERNAME,
            client_id=f"{INBOX_USERNAME}-inbox"
        )
    except Exception as e:
        if "already exists" in str(e).lower():
            inbox_id = f"{INBOX_USERNAME}@agentmail.to"
            class SimpleInbox:
                def __init__(self, inbox_id):
                    self.inbox_id = inbox_id
            inbox = SimpleInbox(inbox_id)
        else:
            raise

    # Start ngrok
    listener = ngrok.forward(PORT, domain=WEBHOOK_DOMAIN, authtoken_from_env=True)

    # Create webhook
    try:
        client.webhooks.create(
            url=f"{listener.url()}/webhook/agentmail",
            event_types=["message.received"],
            client_id=f"{INBOX_USERNAME}-webhook"
        )
    except Exception as e:
        if "already exists" not in str(e).lower():
            raise

    print(f"Ready: {inbox.inbox_id}\n")
    return inbox, listener

def analyze_email(subject, content):
    """Use AI to classify email across multiple dimensions with retry logic."""
    valid_values = {
        "sentiment": {"positive", "neutral", "negative"},
        "category": {"question", "complaint", "feature-request", "bug-report", "praise"},
        "priority": {"urgent", "high", "normal", "low"},
        "department": {"sales", "support", "billing", "technical"}
    }

    for attempt in range(1, 4):
        try:
            if attempt > 1:
                time.sleep(1)

            response = openai_client.chat.completions.create(
                model="gpt-4o-mini",
                messages=[
                    {
                        "role": "system",
                        "content": "You are an expert email classifier. Analyze emails and return structured classifications."
                    },
                    {
                        "role": "user",
                        "content": f"""Analyze this email across 4 dimensions:

                        Subject: {subject}
                        Content: {content}

                        Classify into:
                        1. sentiment: positive | neutral | negative
                        2. category: question | complaint | feature-request | bug-report | praise
                        3. priority: urgent | high | normal | low
                        4. department: sales | support | billing | technical

                        Consider:
                        - Sentiment: Overall tone and emotion
                        - Category: Primary intent of the email
                        - Priority: Urgency indicators (ASAP, urgent, immediately, deadline mentions, emergency)
                        - Department: Best team to handle this

                        Return ONLY valid JSON with these exact keys: sentiment, category, priority, department.
                        Example: {{"sentiment": "positive", "category": "question", "priority": "normal", "department": "sales"}}
                        """
                    }
                ],
                response_format={"type": "json_object"},
                temperature=0.3
            )

            # Parse and validate
            result = json.loads(response.choices[0].message.content)

            required_keys = ["sentiment", "category", "priority", "department"]
            missing_keys = [key for key in required_keys if key not in result]
            if missing_keys:
                raise ValueError(f"Missing keys: {missing_keys}")

            invalid_values = []
            for dimension, value in result.items():
                if dimension in valid_values and value not in valid_values[dimension]:
                    invalid_values.append(f"{dimension}={value}")

            if invalid_values:
                raise ValueError(f"Invalid values: {', '.join(invalid_values)}")

            return result

        except Exception as e:
            if attempt == 3:
                raise Exception(f"AI classification failed: {e}")

def apply_labels(inbox_id, message_id, classifications):
    """Apply labels based on classification results."""
    labels = [
        f"{classifications['sentiment']}",
        f"{classifications['category']}",
        f"{classifications['priority']}",
        f"{classifications['department']}"
    ]

    # Try batch first
    try:
        client.inboxes.messages.update(
            inbox_id=inbox_id,
            message_id=message_id,
            add_labels=labels
        )
        for label in labels:
            print(f"  ✓ {label}")
        return
    except Exception:
        pass

    # Try individually
    successful = []
    for label in labels:
        try:
            client.inboxes.messages.update(
                inbox_id=inbox_id,
                message_id=message_id,
                add_labels=[label]
            )
            successful.append(label)
            print(f"  ✓ {label}")
        except Exception:
            print(f"  ✗ {label}")

    if not successful:
        raise Exception("Failed to apply labels")

@app.route('/webhook/agentmail', methods=['POST'])
def receive_webhook():
    """Webhook endpoint to receive incoming email notifications."""
    try:
        payload = request.json
        event_type = payload.get('type') or payload.get('event_type')

        # Ignore outgoing messages
        if event_type == 'message.sent':
            return Response(status=200)

        message = payload.get('message', {})
        message_id = message.get('message_id')
        inbox_id = message.get('inbox_id')
        from_field = message.get('from_', '') or message.get('from', '')

        # Validate required fields
        if not message_id or not inbox_id or not from_field:
            return Response(status=200)

        # Extract sender email
        if '<' in from_field and '>' in from_field:
            sender_email = from_field.split('<')[1].split('>')[0].strip()
        else:
            sender_email = from_field.strip()

        subject = message.get('subject', '(no subject)')
        email_body = message.get('text', '') or message.get('body', '') or message.get('html', '')

        # Log
        print(f"\n📧 {sender_email}: {subject}")

        # Analyze
        classifications = analyze_email(subject, email_body)

        print(f"  Sentiment: {classifications['sentiment']}")
        print(f"  Category: {classifications['category']}")
        print(f"  Priority: {classifications['priority']}")
        print(f"  Department: {classifications['department']}")

        # Apply labels
        apply_labels(inbox_id, message_id, classifications)
        print("Done\n")

    except Exception as e:
        print(f"Error: {e}\n")

    return Response(status=200)

if __name__ == '__main__':
    print("SMART EMAIL LABELING AGENT\n")
    inbox, listener = setup_agentmail()
    print("Waiting for emails...\n")
    app.run(port=PORT)

Step 3: Create Requirements File ​

Create a file named requirements.txt:

txt
agentmail
flask>=3.0.0
ngrok>=1.0.0
python-dotenv>=1.0.0
openai>=1.0.0

Step 4: Install Dependencies ​

Install the required Python packages:

bash
pip install -r requirements.txt

Step 5: Configure Environment Variables ​

Create a .env file with your credentials:

env
# AgentMail Configuration
AGENTMAIL_API_KEY=your_agentmail_api_key_here

# OpenAI Configuration
OPENAI_API_KEY=your_openai_api_key_here

# Ngrok Configuration
NGROK_AUTHTOKEN=your_ngrok_authtoken_here
WEBHOOK_DOMAIN=your-domain.ngrok-free.app

# Agent Settings
INBOX_USERNAME=smart-labels
PORT=8080

Code Walkthrough ​

Let's understand how the agent works by breaking down the key components.

Architecture Overview ​

Email arrives → AgentMail → Webhook → ngrok → Flask → AI Analysis → Apply Labels

1. Initialization ​

python
# Load environment variables first
load_dotenv()

# Initialize three clients
app = Flask(__name__)           # Web server for webhooks
client = AgentMail()            # AgentMail SDK
openai_client = OpenAI()        # OpenAI for AI classification

2. Setting Up Infrastructure ​

The setup_agentmail() function creates your inbox and webhook:

python
# Create inbox with idempotency
inbox = client.inboxes.create(
    username=INBOX_USERNAME,
    client_id=f"{INBOX_USERNAME}-inbox"  # Prevents duplicates
)

# Start ngrok tunnel (localhost → public URL)
listener = ngrok.forward(PORT, domain=WEBHOOK_DOMAIN)

# Register webhook with AgentMail
client.webhooks.create(
    url=f"{listener.url()}/webhook/agentmail",
    event_types=["message.received"],
    client_id=f"{INBOX_USERNAME}-webhook"
)

Idempotency with client_id:

Using client_id ensures you can safely restart the agent without creating duplicate inboxes or webhooks. If a resource already exists, AgentMail returns the existing one.

3. AI-Powered Email Analysis ​

The analyze_email() function is the core of the agent:

python
def analyze_email(subject, content):
    # Define valid classification values
    valid_values = {
        "sentiment": {"positive", "neutral", "negative"},
        "category": {"question", "complaint", "feature-request", ...},
        "priority": {"urgent", "high", "normal", "low"},
        "department": {"sales", "support", "billing", "technical"}
    }

    # Retry up to 3 times
    for attempt in range(1, 4):
        try:
            # Call OpenAI API
            response = openai_client.chat.completions.create(...)

            # Parse JSON response
            result = json.loads(response.choices[0].message.content)

            # Validate keys and values
            # ... validation logic ...

            return result
        except Exception as e:
            if attempt == 3:
                raise

Key features:

  1. Structured prompt: Clearly defines the 4 classification dimensions
  2. JSON mode: Forces OpenAI to return valid JSON
  3. Retry logic: Automatically retries up to 3 times on failures
  4. Strict validation: Ensures classifications match expected values
  5. Low temperature (0.3): Consistent, predictable classifications

Example OpenAI prompt:

Analyze this email across 4 dimensions:

Subject: Product keeps crashing!
Content: Your software is terrible. Fix it ASAP!

Classify into:
1. sentiment: positive | neutral | negative
2. category: question | complaint | feature-request | bug-report | praise
3. priority: urgent | high | normal | low
4. department: sales | support | billing | technical

Return ONLY valid JSON...

AI Response:

json
{
  "sentiment": "negative",
  "category": "complaint",
  "priority": "urgent",
  "department": "support"
}

4. Applying Labels ​

The apply_labels() function applies labels with a two-tier strategy:

python
# Create labels from classification values
labels = [
    "negative",
    "complaint",
    "urgent",
    "support"
]

# Try batch application first (most efficient)
try:
    client.inboxes.messages.update(
        inbox_id=inbox_id,
        message_id=message_id,
        add_labels=labels  # Apply all at once
    )
    return
except Exception:
    pass  # If batch fails, try individually

# Apply labels one by one
for label in labels:
    try:
        client.inboxes.messages.update(
            inbox_id=inbox_id,
            message_id=message_id,
            add_labels=[label]  # One at a time
        )
    except Exception:
        pass  # Log failure but continue

Why two tiers?

  1. Batch first: Fastest approach (1 API call for all labels)
  2. Individual fallback: If batch fails, try each label separately to save what we can
  3. Resilient: Won't fail completely if one label has an issue

5. Processing Webhooks ​

The receive_webhook() function orchestrates everything:

python
@app.route('/webhook/agentmail', methods=['POST'])
def receive_webhook():
    payload = request.json

    # Ignore outgoing messages (prevents infinite loops)
    if event_type == 'message.sent':
        return Response(status=200)

    # Extract email data
    message_id = message.get('message_id')
    inbox_id = message.get('inbox_id')
    sender_email = extract_email(from_field)
    subject = message.get('subject')
    email_body = message.get('text')

    # Classify with AI
    classifications = analyze_email(subject, email_body)

    # Apply labels
    apply_labels(inbox_id, message_id, classifications)

    # Always return 200 (tells AgentMail we received it)
    return Response(status=200)

Why always return 200?

Even if classification or labeling fails, we return 200 to tell AgentMail: "I received this webhook." If we returned an error (400/500), AgentMail would retry sending the webhook, which doesn't help with application errors.

Running the Agent ​

Start the agent:

bash
python agent.py

You should see output like this:

SMART EMAIL LABELING AGENT

Ready: smart-labels@agentmail.to

Waiting for emails...

 * Running on http://127.0.0.1:8080

Success! Your agent is now running and ready to classify emails.

Leave this terminal window open - closing it will stop the agent.

Testing Your Agent ​

Let's test the agent with different types of emails to see how it classifies them.

Example 1: Urgent Complaint ​

Send this email:

To: smart-labels@agentmail.to
Subject: Product crashed - need immediate help!
Body: Your product is TERRIBLE! It crashed 3 times today and I lost all my work.
      I need this fixed IMMEDIATELY or I want a full refund!

Console Output:

you@example.com: Product crashed - need immediate help!
  Sentiment: negative
  Category: complaint
  Priority: urgent
  Department: support
  ✓ negative
  ✓ complaint
  ✓ urgent
  ✓ support
Done

Why this classification?

  • Sentiment: negative (words: "terrible", "lost work")
  • Category: complaint (expressing dissatisfaction)
  • Priority: urgent (keywords: "IMMEDIATELY", "crashed 3 times")
  • Department: support (product issue)

Example 2: Feature Request ​

Send this email:

To: smart-labels@agentmail.to
Subject: Dark mode would be amazing!
Body: Hi! I absolutely love your product. I use it every day.
      One feature that would make it even better is dark mode support.
      Keep up the great work!

Console Output:

you@example.com: Dark mode would be amazing!
  Sentiment: positive
  Category: feature-request
  Priority: normal
  Department: technical
  ✓ positive
  ✓ feature-request
  ✓ normal
  ✓ technical
Done

Why this classification?

  • Sentiment: positive (words: "love", "amazing", "great work")
  • Category: feature-request (suggesting new functionality)
  • Priority: normal (no urgency indicators)
  • Department: technical (feature implementation)

What Happens Next? ​

View in Dashboard ​

Go to your AgentMail inbox and filter by labels to organize your emails:

Filter by sentiment:

  • Search negative to see all unhappy customers
  • Search positive to find praise and testimonials
  • Search neutral to review informational emails

Filter by priority, by department...

Combine filters for powerful queries:

  • urgent + negative → Critical customer issues
  • sales + high → Hot leads requiring fast response
  • technical + bug-report → Engineering backlog

Pro tip: You can use AgentMail's API to programmatically fetch emails by labels and build custom workflows, dashboards, and analytics.

Building on Labels ​

Once your emails are automatically labeled, you can build powerful automation on top:

Example 1: Priority Notifications ​

Scenario: Alert your team instantly when urgent issues arrive.

When an email is labeled urgent + negative, automatically send a Slack notification to your support channel. Include the sender's email and subject line so your team can respond immediately. This ensures critical customer issues never slip through the cracks.

Example 2: Sentiment Escalation ​

Scenario: Escalate negative sentiment to management.

Track all emails labeled negative and automatically notify customer success managers when sentiment trends downward. If a single customer sends 3+ negative emails in a week, trigger a personal outreach from leadership to address their concerns proactively.

Example 3: Department Routing ​

Scenario: Auto-forward emails to the right team.

Create rules that automatically forward emails based on department labels:

  • sales → Forward to sales@yourcompany.com
  • billing → Create a ticket in your billing system
  • technical → Post to #engineering Slack channel
  • support → Add to support queue with appropriate SLA

Example 4: Smart Auto-Reply ​

Scenario: Send contextual automated responses.

Instead of generic auto-replies, craft responses based on classification:

  • question → "Thanks for your question! We'll respond within 24 hours."
  • bug-report → "Thanks for reporting this. Our engineering team has been notified."
  • complaint → "We're sorry to hear that. A senior support agent will contact you within 4 hours."
  • feature-request → "Great idea! We've added this to our product roadmap."

Example 5: Analytics Dashboard ​

Scenario: Track email metrics and trends.

Build a dashboard that queries emails by labels to analyze:

  • Sentiment trends: Are customers getting happier or more frustrated?
  • Volume by department: Which team is handling the most emails?
  • Response times by priority: Are you meeting SLAs for urgent issues?
  • Common categories: What are customers asking about most?

Use this data to identify bottlenecks, improve processes, and make data-driven decisions about staffing and product priorities.

Congratulations! You've built an AI-powered email classification system. This agent showcases how AgentMail's labeling feature can power sophisticated inbox automation and analytics.