Outlook-QA: Extract Outlook Email Content and Generate QA Corpus

This project retrieves email content from an Outlook mailbox using the Microsoft Graph API and OAuth, processes it, and generates a QA corpus suitable for model training.

Project Overview

Objective: Extract emails from an Outlook mailbox, clean the data, and generate a QA dataset.
Approach: Utilize Microsoft Graph API and OAuth protocol to access the mailbox.
Output: A structured QA corpus saved in CSV format.

Prerequisites

A Microsoft account with Outlook mailbox permissions.
An application registered in the Azure Portal to obtain an OAuth token.
A configured Python environment with required libraries installed (msgraph-sdk, requests, etc.).

Steps

1. Obtain Token

Microsoft no longer supports direct IMAP connections using account credentials. An OAuth token is required instead.

Method 1: Delegated Token Acquisition (Manual)

Visit Microsoft Graph Explorer.
Sign in and generate a token with Mail.ReadWrite permissions.

Method 2: Application Token Acquisition (Programmatic)

Register an application in the Azure Portal.
Grant the application Mail.ReadWrite permissions.
Use the client credentials flow to obtain a token.

2. Email Operations

Method 1: Microsoft Graph API

Documentation: Refer to Microsoft Graph API Documentation.
Testing: Test the API using Graph Explorer.
Example: Use the /me/messages endpoint to retrieve emails.

Method 2: Microsoft Graph SDK

Use the Python SDK (msgraph-sdk).
Sample Code: Refer to msgraph-training-python.

3. Data Processing Workflow

Step 1: Retrieve and Parse Emails

Fetch raw email data using the Graph API.
Convert the response data into JSON format.

Step 2: Data Filtering

Filtering Criteria:
1. Exclude single-session emails (no replies).
2. Remove irrelevant HTML elements (tables, advertisement images, etc.).
3. Exclude conversations with ≥ 2 messages where the last message is a forward.
4. Process and merge forwarded emails that do not display original content.

Step 3: Convert to Conversation Format

Transform email threads into a JSON conversation format.
Use an LLM to batch-extract QA pairs.
Save as a CSV file with columns: Question, Answer.

Step 4: Deduplication

Hash QA pairs and remove duplicates.

Output Format

File: qa_corpus.csv

Structure:

Question,Answer
"When is the meeting scheduled?","Tomorrow at 2 PM."

AngGaGim/Outlook-QA

ErrorLooks like something went wrong!

About

Resources

Uh oh!

Stars

Watchers

Forks

Releases

Packages 0

Contributors 2

Uh oh!

Error
Looks like something went wrong!

Packages