Email Reply Extraction with Talon

Fresh 🌱

Learn how to use Talon to extract new content from email replies, removing quoted text with 93.8% accuracy.

Why Talon?

Email threads accumulate quoted replies that clutter the actual content. When processing emails programmatically, you need just the new message, not the entire conversation history.

Talon solves this problem by extracting clean reply content through sophisticated pattern matching and structural analysis.

Use Cases

AI Email Agents: Extract new user messages without processing entire thread history
Email Automation: Parse replies to identify actionable content
Thread Analysis: Build conversation flows by isolating individual contributions
Inbox Management: Process only new information from replies

Why Choose Talon?

Handles Gmail, Outlook, Apple Mail, Thunderbird HTML structures

93.8% success rate across 64 real-world test cases

Supports English, Japanese, Swedish, Polish, Dutch, German

1.92ms average processing time, 488 emails/second

How Talon Works

Talon uses two complementary approaches depending on email format:

Plain Text Processing (6-Stage Pipeline)

Line Classification: Assigns markers to each line ('t'=text, 'm'=quote marker, 's'=splitter, 'e'=empty)
Pattern Matching: Applies regex to marker sequences to identify quoted blocks
Content Extraction: Removes quoted lines and returns clean text

Recognizes patterns like:

Standard quote markers (>)
Reply headers ("On [date] [name] wrote:")
Forward indicators ("-----Original Message-----")

HTML Processing (8-Stage Pipeline)

Structural Removal: Directly removes known quotation elements (Gmail divs, blockquotes, Outlook markup)
Checkpoint Fallback: For non-standard HTML, maps elements to text lines, applies text patterns, removes corresponding HTML

Processing Systems

Quotation Removal (Primary)

Removes quoted replies from thread
No initialization required
Rule-based pattern matching

Getting Started

Install via pip with required workaround for Python 3.11+:

bash

pip install talon

Required fix for cchardet dependency:

python

# Import workaround BEFORE importing talon
import sys
import chardet
sys.modules['cchardet'] = chardet

# Now safe to import talon
import talon
from talon import quotations

This workaround is required for Python 3.11+.

Basic usage for plain text and HTML:

python

from talon import quotations

email = """Great work on the project!

On Mon, Apr 11, 2011 at 6:54 PM, Bob wrote:
> Can you review the document?
> Need feedback by Friday.
"""

clean_reply = quotations.extract_from_plain(email)
# Result: "Great work on the project!"

python

from talon import quotations

html_email = """
<html><body>
<div>Thanks for the update!</div>
<div class="gmail_quote">
  <div>On Mon, Alice wrote:</div>
  <blockquote>Original message here</blockquote>
</div>
</body></html>
"""

clean_html = quotations.extract_from_html(html_email)
# Returns: <html><body><div>Thanks for the update!</div></body></html>

Performance & Accuracy

Talon has been tested on 64 real-world emails from various clients and languages.

Test Results Summary

Metric	Value
Total Tests	64 emails
Passed	60 (93.8%)
Failed	4 (6.2%)
Avg Processing Time	1.92ms
Throughput	488.6 emails/second
Min/Max Time	0.13ms - 21.55ms

Test Coverage

22 HTML emails: Gmail, Outlook, Apple Mail, Thunderbird, Mail.ru, Hotmail
42 plain text emails: Various formats and reply styles
6+ languages: English, Japanese, Swedish, Polish, Dutch, German
Mobile clients: iPhone, Android "Sent from" signatures

Processing Time by Complexity

Email Type	Avg Time	Complexity
Simple text reply	0.2-0.5ms	Low
HTML Gmail/Outlook	2-4ms	Medium
Complex threads	4-22ms	High

Speed vs Accuracy Tradeoff

Library	Avg Processing Time	Accuracy	Best For
Talon	1.92ms	93.8%	Production systems needing HTML support
qutoequail	0.96ms	~85%	Moderate accuracy requirements
Custom regex	0.1ms	~70%	Simple plain text, speed critical

Insight: For production systems, 1.92ms average is negligible. Even at worst case (21.55ms), Talon is faster than most network requests.

Known Limitations

Talon failed 4 out of 64 test cases. Here's what didn't work:

Test Case 1: Complex Email Thread with Mixed Content

Input:

text

Thank you, Sonya Johnson.

I have sent an invite for 10:30am Monday PDT (today). I
hope you can join.

Regards,

Christopher Edwards

On Mon, Jun 3, 2024 at 12:53 AM Cody Hart <omerritt@example.com> wrote:

> Hi Christopher Edwards,
>
> 10.30 AM pacific is good for me.
>
> Thanks & Regards,
>
> Cody Hart

Expected Output: First 5 lines only (up to Christopher Edwards)

Talon's Output: Returns entire email including quoted text starting with "On Mon, Jun 3..." and all "> quoted text"

Processing Time: 2.55ms

Issue: Signature placement before quotes confuses detection logic

Test Case 2: Inline Responses

Input:

text

On Tue, Apr 29, 2014 at 4:22 PM, Example Dev <sugar@example.com> wrote:

> okay. Well, here's some stuff I can write.
>
> And if I write a 2 second line you and maybe reply under this?
>
> Or if you didn't really feel like it, you could reply under this line.

I will reply under this one

>
> okay?
>

and under this.

>
> -- Tim

Expected Output: Just the inline responses (I will reply under this one and and under this.)

Talon's Output: Returns everything including "On Tue, Apr 29..." header and all quoted lines

Processing Time: 0.48ms

Issue: Interleaved inline responses not recognized as the reply pattern

Test Case 3: Gmail Forward HTML

Input:

html

<html><head></head><body><div dir="ltr">test<div><br /></div><div>blah</div>
<div><br /><div class="gmail_quote">---------- Forwarded message ----------<br />
From: <b class="gmail_sendername">Foo Bar</b>
<span dir="ltr">&lt;<a href="mailto:foo@bar.example">foo@bar.example</a>&gt;</span><br />
Date: Thu, Mar 24, 2016 at 5:17 PM<br />
Subject: The Subject<br />
To: John Doe &lt;<a href="mailto:john@doe.example">john@doe.example</a>&gt;<br />
<br /><br /><div dir="ltr">Some text<div><br /></div><div><br /></div></div>
</div><br /></div></div></body></html>

Expected Output: Just testblah (before the forward marker)

Talon's Output: Includes "---------- Forwarded message ----------" and forwarded content

Processing Time: 3.41ms

Issue: HTML forward headers not removed by Gmail quote detection

Test Case 4: Thunderbird Forward HTML

Input:

html

<html><body bgcolor="#FFFFFF" text="#000000">
<p><br /></p>
<div class="moz-forward-container"><br /><br />
-------- Forwarded Message --------
<table class="moz-email-headers-table">
  <tbody>
    <tr><th>Subject:</th><td>Re: Example subject</td></tr>
    <tr><th>Date:</th><td>Tue, 3 May 2016 14:54:27 +0200 (CEST)</td></tr>
    <tr><th>From:</th><td>John Doe &lt;johndoe@example.com&gt;</td></tr>
  </tbody>
</table>
<br /><br />
<div>Dear John,</div>
<div><br /></div>
<div>This is a test.</div>
</div></body></html>

Expected Output: Empty (no new content, just forward)

Talon's Output: Includes "-------- Forwarded Message --------" and forwarded content

Processing Time: 4.34ms

Issue: Thunderbird's moz-forward-container class not recognized

Summary: 3 of 4 failures are forwarded messages. Regular replies work with 98%+ accuracy.

Example 1: Simple Gmail Reply

Input:

text

Awesome! I haven't had another problem with it.

On Aug 22, 2011, at 7:37 PM, defunkt<reply@reply.github.com> wrote:

> Loader seems to be working well.

Talon's Output: Awesome! I haven't had another problem with it.

Processing Time: 0.2ms

What Worked: Standard "On [date] [name] wrote:" pattern detected, quote marker (>) recognized

Example 2: Outlook Reply with Separator

Input:

text

Outlook with a reply directly above line
________________________________________
From: CRM Comments [crm-comment@example.com]
Sent: Friday, 23 March 2012 5:08 p.m.
To: John S. Greene
Subject: [contact:106] John Greene

A new comment has been added to the Contact named 'John Greene':

I am replying to a comment.

Talon's Output: Outlook with a reply directly above line

Processing Time: 0.51ms

What Worked: Outlook separator line (underscores) and "From:"/"Sent:" headers detected as splitter

Example 3: HTML Outlook Reply

Input:

html

<html>
  <body>
    <div>Reply</div>
    <span id="OLK_SRC_BODY_SECTION">
      <div>
        <span>From: </span>Bob &lt;<a href="mailto:bob@example.com">bob@example.com</a>&gt;<br />
        <span>Date: </span>Tue, 01 Nov 2011 18:54:39 -0700<br />
        <span>To: </span>Rob &lt;<a href="mailto:rob@example.com">rob@example.com</a>&gt;<br />
        <span>Subject: </span>Test<br />
      </div>
      <div>Hi</div>
    </span>
  </body>
</html>

Talon's Output: Reply

Processing Time: 4.02ms

What Worked: Outlook's OLK_SRC_BODY_SECTION span ID detected and removed structurally

Tradeoff: Talon is more comprehensive but slower than plain-text-only libraries

Talon: 1.92ms average (with HTML support)
email-reply-parser: 0.03ms average (plain text only)

For production systems, 1.92ms average is negligible. Even at worst case (21.55ms), Talon is faster than most network requests.

As shown in test results, forwarded messages (especially HTML) are challenging:

Plain text forwards: Generally work well
HTML forwards: May retain forward headers
Workaround: Use plain text extraction or post-process to remove forward markers

Error Handling

Always handle potential parsing failures:

python

from talon import quotations

def safe_extract(email_body, is_html=False):
    try:
        if is_html:
            return quotations.extract_from_html(email_body)
        else:
            return quotations.extract_from_plain(email_body)
    except Exception as e:
        # Fallback to original message if extraction fails
        print(f"Talon extraction failed: {e}")
        return email_body

Testing Recommendations

Always test with your specific email formats:

python

# Create a test suite with your actual email patterns (Gmail, Outlook, Apple Mail)
test_emails = [
    "path/to/gmail_reply.html",
    "path/to/outlook_reply.txt",
    "path/to/forward.html"
]

for email_file in test_emails:
    with open(email_file) as f:
        content = f.read()
        result = quotations.extract_from(content)
        print(f"{email_file}: {len(result)} chars extracted")

Test with real emails from your users' actual email clients. Talon's accuracy is based on diverse real-world samples, but your specific use case may have unique patterns.

JavaScript Version

For TypeScript/JavaScript projects, use TalonJS - a JavaScript port of Talon with similar functionality.

Performance Comparison

Solution	Accuracy	Speed	Best For
Python Talon	93.8%	1.92ms	Highest accuracy
TalonJS	90.6%	1.88ms	TypeScript/Node.js projects

TalonJS provides 90.6% accuracy with slightly faster performance (1.88ms), making it ideal for JavaScript/TypeScript environments without needing Python dependencies.

Quick Start

bash

npm install talonjs

typescript

import * as talon from 'talonjs';

const email = `Great work on the project!

On Mon, Apr 11, 2011 at 6:54 PM, Bob wrote:
> Can you review the document?
> Need feedback by Friday.
`;

const result = talon.quotations.extractFromPlain(email);
const cleanReply = result.body.trim();
// Output: "Great work on the project!"

typescript

import * as talon from 'talonjs';

const htmlEmail = `
  <div>Thanks for the update!</div>
  <div class="gmail_quote">
    On Mon, Alice wrote: <blockquote>Original</blockquote>
  </div>
`;

const result = talon.quotations.extractFromHtml(htmlEmail);
const cleanReply = result.body.trim();
// Returns: "<div>Thanks for the update!</div>"
// Note: TalonJS returns clean HTML, not plain text

When to use TalonJS vs Python Talon:

Use TalonJS if you're building in TypeScript/JavaScript and 90.6% accuracy is sufficient
Use Python Talon if you need the highest accuracy (93.8%) or are in a Python environment
The 3.2% accuracy difference is acceptable for most use cases

Email Reply Extraction with Talon ​

Why Talon? ​

Use Cases ​

Why Choose Talon? ​

How Talon Works ​

Plain Text Processing (6-Stage Pipeline) ​

HTML Processing (8-Stage Pipeline) ​

Processing Systems ​

Getting Started ​

Performance & Accuracy ​

Test Results Summary ​

Test Coverage ​

Processing Time by Complexity ​

Speed vs Accuracy Tradeoff ​

Known Limitations ​

Test Case 1: Complex Email Thread with Mixed Content ​

Test Case 2: Inline Responses ​

Test Case 3: Gmail Forward HTML ​

Test Case 4: Thunderbird Forward HTML ​

Example 1: Simple Gmail Reply ​

Example 2: Outlook Reply with Separator ​

Example 3: HTML Outlook Reply ​

Error Handling ​

Testing Recommendations ​

JavaScript Version ​

Performance Comparison ​

Quick Start ​

Email Reply Extraction with Talon

Why Talon?

Use Cases

Why Choose Talon?

How Talon Works

Plain Text Processing (6-Stage Pipeline)

HTML Processing (8-Stage Pipeline)

Processing Systems

Getting Started

Performance & Accuracy

Test Results Summary

Test Coverage

Processing Time by Complexity

Speed vs Accuracy Tradeoff

Known Limitations

Test Case 1: Complex Email Thread with Mixed Content

Test Case 2: Inline Responses

Test Case 3: Gmail Forward HTML

Test Case 4: Thunderbird Forward HTML

Example 1: Simple Gmail Reply

Example 2: Outlook Reply with Separator

Example 3: HTML Outlook Reply

Error Handling

Testing Recommendations

JavaScript Version

Performance Comparison

Quick Start