4.1 Preferences Module

preferences.py is a module designed to pick up on a user's preferences, for example, likes or dislikes for example. The goal is to allow for a more customized approach to conversations between the user and LLM, allowing for it to feel less "robotic" and have a more natural flow while chatting. This should also allow the app to pick up on other things relevant to the user in other ways, helping it make suggestions, decisions and use of memories or other tools.

Overview of preferences.py

This file implements a PreferenceExtractor system that can analyze text and identify what users like, dislike, love, or hate. It's designed to work with your chatbot's memory system to build user profiles.

1. Imports and Setup

The file uses several key libraries:

• spacy: For advanced natural language processing (optional fallback if not available)
• re: For regex pattern matching
• dataclasses: For clean data structures
• enum: For defining constants
• typing: For type hints, list, dict, optional
• logging: For logging

2. Enums (Constants)

ExtractionMethod:

Tracks how a preference was found:

• REGEX_PATTERN: Found using regex patterns
• SPACY_ENTITY: Found using spaCy's entity recognition
• SPACY_DEPENDENCY: Found using spaCy's dependency parsing
• SENTIMENT_ANALYSIS: Found using sentiment analysis

PreferenceType:

Defines the strength of preferences:

LIKES, DISLIKES, LOVES, HATES, NEUTRAL

3. PreferenceResult Class:

This is a dataclass that stores each extracted preference:

@dataclass
class PreferenceResult:
   user_id: str                    # Who said it
   preference_type: str            # likes/dislikes/loves/hates
   preference_category: str        # food/games/activities/etc
   preference_value: str           # the actual thing (e.g., "pizza")
   confidence: float               # how sure we are (0.0-1.0)
   extraction_method: str          # how we found it
   context: str                    # surrounding text
   original_input: str             # full original text
   notes: Optional[str] = None     # extra info@dataclass

4. PreferenceExtractor Class:

The Main Engine

This is where the magic happens! Let's break down its key methods:

init()

• Tries to load spaCy models (large first, then small, then fallback)
• Sets up regex patterns and spaCy matchers
• Defines categories (food, games, colors, etc.)
• Sets confidence thresholds

5. Key Methods Explained:

extract_preferences()

The Main Entry Point

This is the method you'd call to extract preferences from text.

What it does:

• Handles compound sentences (splits on "but")
• Runs multiple extraction methods
• Filters by confidence
• Validates and deduplicates results
• Applies sentiment hierarchy

_extract_with_regex()

Pattern Matching

Pattern matching is a simple (and possibly obvious) method for targeting words that would identify a potential preference.

This uses regex patterns like:

• "i love [something]" → confidence 0.85, type: LOVES
• "i don't like [something]" → confidence 0.8, type: DISLIKES
• "my favorite [something] is [item]" → confidence 0.9, type: LOVES

_extract_with_spacy()

Advanced NLP

NLP (Natural Language Processing) and NER (Named Entity Recognition) are AI tools that can help a user parse input text to identify possible preferences and objects that align with the identified preference. This uses spaCy's linguistic analysis to find preferences in more complex sentences.

_parse_compound_objects()

Handle Lists

Splits text like "I like pizza and pasta" into separate preferences for "pizza" and "pasta".

_is_valid_preference_value()

Quality Control

Filters out noise like:

• Too short/long text
• Just sentiment words ("love", "hate")
• Mostly stop words ("the", "and", "but")

_categorize_preference()

Classification

Tries to classify preferences into categories like "food", "games", "activities", etc. It will need to be updated periodically to add more categories as they become relevant, likely after reviewing chat logs and other collected data from usage.

6. How It All Works Together

Here's an example flow when you call extract_preferences("I love pizza but hate mushrooms"):

• Split compound sentence: "I love pizza" + "I hate mushrooms"
• Run regex patterns on each part
• Run spaCy analysis (if available)
• Parse objects: Extract "pizza" and "mushrooms"
• Validate: Check if "pizza" and "mushrooms" are valid preferences
• Categorize: Classify both as "food"
• Create results: Two PreferenceResult objects
• Return: List of validated, high-confidence preferences

7. Why This Design is Smart

• Hybrid approach: Uses both simple regex AND advanced NLP
• Confidence scoring: Knows how sure it is about each preference
• Graceful degradation: Works even if spaCy isn't installed
• Extensible: Easy to add new patterns or categories
• Validation: Filters out noise and invalid results

This module is built with simple yet robust, production-ready code that handles edge cases and provides multiple fallback options. It may require periodic updates for other cases and exceptions if ever needed, but it covers most of the bases of finding user preferences within the nuances of the use of the English language around grammatical rules and syntax.

8. Confidence Scoring System

The PreferenceExtractor uses a sophisticated confidence scoring system to ensure only high-quality preferences are extracted and stored.

Confidence Thresholds by Pattern Type:

• Explicit favorites: 0.9 - "My favorite food is pizza"
• Strong emotions (love/hate): 0.85 - "I love pizza" / "I hate mushrooms"
• Clear dislikes: 0.8 - "I don't like olives"
• Moderate preferences: 0.75 - "I like pasta" / "I enjoy cooking"

Why These Scores Matter:

• Default minimum threshold: 0.6 filters out uncertain extractions
• Higher confidence = more reliable for building user profiles
• Prevents noise from ambiguous statements like "pizza is okay" (would score ~0.4)

Confidence Boosting Factors:

• Intensity words: "really love" gets higher confidence than just "love"
• Explicit patterns: "My favorite X is Y" gets maximum confidence
• Clear negation: "don't like" is more confident than implied dislike

Example Confidence Scores:

"I absolutely love chocolate" → 0.85 (LOVES)
"My favorite game is Minecraft" → 0.9 (LOVES) 
"I don't really like horror movies" → 0.8 (DISLIKES)
"Pizza is pretty good" → 0.4 (filtered out - below threshold)

9. spaCy Fallback System

The system uses a hybrid approach that gracefully handles different installation scenarios:

Three-Tier Loading Strategy:

1. First attempt: Load en_core_web_lg (large model, best accuracy)
2. Fallback: Load en_core_web_sm (small model, good performance)
3. Final fallback: Regex-only mode (no spaCy required)

What Happens Without spaCy:

• Regex patterns still work - covers 80% of common preference expressions
• Performance impact: Minimal - regex is actually faster
• Accuracy trade-off: May miss complex grammatical structures
• No dependency issues - works on any Python installation

spaCy Advantages When Available:

• Advanced parsing: Handles complex sentence structures
• Entity recognition: Better at identifying compound nouns ("chocolate ice cream")
• Dependency parsing: Understands relationships between words
• Contextual understanding: Can handle more nuanced expressions

Performance Comparison:

# Regex-only mode:
"I love pizza" ✅ (0.85 confidence)
"Pizza is my favorite food" ✅ (0.9 confidence)

# spaCy enhancement:  
"The pizza I had yesterday was amazing" ✅ (spaCy catches this)
"I'm not a fan of spicy food" ✅ (spaCy understands negation better)

When to Use Each Mode:

• Production servers: spaCy recommended for best accuracy
• Lightweight deployments: Regex-only is sufficient for basic needs
• Development/testing: Either works fine

10. Future Updates to PreferenceExtractor:

1. Adding new preference categories.
2. Integrating this module with the updated MemoryManager module.
3. Integrating this module (and the MemoryManager module) into the main functioning of the app.