diff --git a/.markdownlint.json b/.markdownlint.json new file mode 100644 index 0000000..34333c4 --- /dev/null +++ b/.markdownlint.json @@ -0,0 +1,14 @@ +{ + "MD010": false, + "MD013": false, + "MD022": false, + "MD024": false, + "MD029": false, + "MD031": false, + "MD032": false, + "MD033": false, + "MD036": false, + "MD038": false, + "MD040": false, + "MD041": false +} diff --git a/CONTRIBUTION.md b/CONTRIBUTION.md new file mode 100644 index 0000000..79a1b73 --- /dev/null +++ b/CONTRIBUTION.md @@ -0,0 +1,821 @@ +# Contributing to HighLevel API SDK + +Thank you for your interest in contributing to the HighLevel API SDK! This document provides guidelines for contributing and detailed information about the TOON integration for AI/LLM token cost reduction. + +## Table of Contents + +- [Getting Started](#getting-started) +- [TOON Integration](#toon-integration) + - [What is TOON?](#what-is-toon) + - [Why TOON?](#why-toon) + - [Where TOON is Used](#where-toon-is-used) + - [How TOON Works](#how-toon-works) + - [Adding TOON to New Services](#adding-toon-to-new-services) +- [Development Workflow](#development-workflow) +- [Code Standards](#code-standards) +- [Testing](#testing) +- [Pull Request Process](#pull-request-process) + +--- + +## Getting Started + +### Prerequisites + +- Node.js >= 18.0.0 +- TypeScript 5.3.0+ +- Git +- A HighLevel developer account + +### Setup + +1. Fork and clone the repository: +```bash +git clone https://github.com/GoHighLevel/highlevel-api-sdk.git +cd highlevel-api-sdk +``` + +2. Install dependencies: +```bash +npm install +``` + +3. Build the project: +```bash +npm run build +``` + +4. Create a new branch for your feature: +```bash +git checkout -b feature/your-feature-name +``` + +--- + +## TOON Integration + +### What is TOON? + +**TOON (Token-Oriented Object Notation)** is a compact, human-readable serialization format designed specifically for passing structured data to Large Language Models with significantly reduced token usage. It's a lossless, drop-in representation of JSON data optimized for LLM contexts. + +**Official Repository:** [github.com/toon-format/toon](https://github.com/toon-format/toon) +**Specification:** [TOON Spec v1.4](https://github.com/toon-format/spec/blob/main/SPEC.md) +**NPM Package:** [@toon-format/toon](https://www.npmjs.com/package/@toon-format/toon) + +**Key Benefits:** +- ๐Ÿ’ธ **30-60% Token Reduction** - Typically 30-60% fewer tokens on large uniform arrays vs formatted JSON +- ๐Ÿคฟ **LLM-Friendly Guardrails** - Explicit lengths `[N]` and fields `{field1,field2}` enable validation +- ๐Ÿฑ **Minimal Syntax** - Removes redundant punctuation (braces, brackets, most quotes) +- ๐Ÿ“ **Indentation-Based** - Like YAML, uses whitespace instead of braces +- ๐Ÿงบ **Tabular Arrays** - Declare keys once, stream data as rows (CSV-like efficiency) + +### Why TOON? + +#### The Problem + +When sending data to LLMs (OpenAI, Claude, etc.) for analysis, every byte counts toward your token usage and API costs. Traditional JSON is verbose and includes significant overhead: + +```json +{ + "contacts": [ + { + "id": "contact_123", + "name": "John Doe", + "email": "john@example.com", + "score": 85, + "tags": ["hot-lead", "enterprise"] + }, + { + "id": "contact_456", + "name": "Jane Smith", + "email": "jane@example.com", + "score": 72, + "tags": ["warm-lead"] + } + ] +} +``` +**Size:** ~250 bytes (~62 tokens @ $0.00186/request with GPT-4) + +#### The Solution + +TOON encodes the same data in a tabular format: + +```toon +contacts[2]{id,name,email,score,tags}: + contact_123,John Doe,john@example.com,85,hot-lead|enterprise + contact_456,Jane Smith,jane@example.com,72,warm-lead +``` +**Size:** ~140 bytes (~35 tokens @ $0.00105/request with GPT-4) + +**Savings: 44% fewer bytes = 44% lower costs! ๐Ÿ’ฐ** + +**Note:** TOON's sweet spot is **uniform arrays of objects** (multiple fields per row, same structure across items). For deeply nested or non-uniform data, JSON may be more efficient. + +#### Real-World Impact + +**Scenario:** AI-powered lead scoring service processing 10,000 contacts/month + +| Metric | Without TOON | With TOON | Savings | +|--------|--------------|-----------|---------| +| Avg Request Size | 25 KB | 15 KB | 40% | +| Tokens per Request | ~6,250 | ~3,750 | 40% | +| Monthly Tokens | 62.5M | 37.5M | 25M | +| Monthly Cost (GPT-4) | $1,875 | $1,125 | **$750/mo** | +| **Annual Savings** | - | - | **$9,000/yr** ๐ŸŽ‰ | + +*Based on GPT-4 pricing: $0.03 per 1K input tokens* + +**Official Benchmarks (from TOON repository):** +- **Mixed-Structure Track:** 21.8% fewer tokens vs formatted JSON (289,901 โ†’ 226,613 tokens) +- **Flat-Only Track:** 58.8% fewer tokens vs formatted JSON (164,255 โ†’ 67,696 tokens) +- **Retrieval Accuracy:** 73.9% accuracy vs JSON's 69.7% while using 39.6% fewer tokens +- Token counts measured using GPT-5 `o200k_base` tokenizer + +### Where TOON is Used + +#### Current Implementation + +TOON is currently implemented in the following locations: + +##### 1. **Lead Intelligence Service** (`lib/code/lead-intelligence/lead-intelligence.ts`) + +The Lead Intelligence service uses TOON in 4 key methods: + +```typescript +// โœ… Used in scoreContacts() - Lines 56-75 +// Converts enriched contact data to TOON before sending to LLM +const { toonData, savings } = encodeToTOON(enrichedContacts, { + delimiter: '\t', + lengthMarker: true +}); + +// โœ… Used in analyzeConversionPatterns() - Lines 137-142 +// Encodes historical conversion data for pattern analysis +const toonData = toTOON(conversions, { + delimiter: '\t', + lengthMarker: true +}); + +// โœ… Used in predictDealClose() - Lines 165-169 +// Converts opportunity data for deal probability prediction +const toonData = toTOON(opportunity, { + delimiter: '\t', + lengthMarker: true +}); + +// โœ… Used in exportToTOON() - Lines 268-274 +// Public method for manual TOON export by developers +exportToTOON(scores, options) { + return encodeToTOON(scores, options); +} +``` + +##### 2. **Shared TOON Utilities** (`lib/utils/toon-utils.ts`) + +Universal utilities available to **ALL services**: + +```typescript +// Core encoding functions +export function encodeToTOON(data, options): { toonData, savings } +export function toTOON(data, options): string + +// Pre-built helpers for common use cases +export function prepareContactsForLLM(contacts, fields) +export function prepareOpportunitiesForLLM(opportunities, fields) +export function prepareConversationsForLLM(conversations, fields) + +// Utility functions +export function formatSavingsReport(savings): string +export function calculateMonthlySavings(...): ROI_Metrics +``` + +##### 3. **Main SDK Exports** (`index.ts`) + +TOON utilities are exported from the main SDK for easy access: + +```typescript +export { + encodeToTOON, + toTOON, + prepareContactsForLLM, + prepareOpportunitiesForLLM, + prepareConversationsForLLM, + formatSavingsReport, + calculateMonthlySavings +} from './lib/utils/toon-utils'; +``` + +#### Services Ready for TOON Integration + +The following services have AI/LLM use cases and can benefit from TOON: + +| Service | File Location | AI Use Case | Potential Savings | +|---------|---------------|-------------|-------------------| +| **Voice AI** | `lib/code/voice-ai/voice-ai.ts` | Call transcription analysis, sentiment detection | 40-60% | +| **Conversations** | `lib/code/conversations/conversations.ts` | Chat history analysis, intent classification | 50-60% | +| **Campaigns** | `lib/code/campaigns/campaigns.ts` | Performance analysis, optimization recommendations | 30-50% | +| **Emails** | `lib/code/emails/emails.ts` | Content analysis, subject line optimization | 40-55% | +| **Workflows** | `lib/code/workflows/workflows.ts` | AI-powered trigger optimization | 35-50% | +| **Forms** | `lib/code/forms/forms.ts` | Response analysis, pattern detection | 45-55% | +| **Opportunities** | `lib/code/opportunities/opportunities.ts` | Deal analysis, win probability prediction | 50-60% | + +### How TOON Works + +#### Technical Architecture + +``` +โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” +โ”‚ Developer's Code โ”‚ +โ”‚ โ”‚ +โ”‚ const contacts = await ghl.contacts.searchContacts({...}); โ”‚ +โ”‚ const opportunities = await ghl.opportunities.get({...}); โ”‚ +โ”‚ // Data received in JSON format from HighLevel API โ”‚ +โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜ + โ”‚ + โ”‚ JavaScript Objects/Arrays (from JSON) + โ”‚ + โ–ผ +โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” +โ”‚ TOON Utilities (lib/utils/toon-utils.ts) โ”‚ +โ”‚ โ”‚ +โ”‚ Core Functions: โ”‚ +โ”‚ โœ“ encodeToTOON(data, options) โ†’ Full conversion + metrics โ”‚ +โ”‚ โœ“ toTOON(data, options) โ†’ Simple conversion โ”‚ +โ”‚ โ”‚ +โ”‚ Pre-Built Helpers: โ”‚ +โ”‚ โœ“ prepareContactsForLLM(contacts, fields) โ”‚ +โ”‚ โœ“ prepareOpportunitiesForLLM(opportunities, fields) โ”‚ +โ”‚ โœ“ prepareConversationsForLLM(conversations, fields) โ”‚ +โ”‚ โ”‚ +โ”‚ Utilities: โ”‚ +โ”‚ โœ“ formatSavingsReport(savings) โ†’ Display metrics โ”‚ +โ”‚ โœ“ calculateMonthlySavings(...) โ†’ ROI calculator โ”‚ +โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜ + โ”‚ + โ”‚ Calls encode() function + โ”‚ + โ–ผ +โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” +โ”‚ @toon-format/toon Package (npm v0.8.0) โ”‚ +โ”‚ โ”‚ +โ”‚ Core Encoding Algorithm: โ”‚ +โ”‚ 1๏ธโƒฃ Analyze data structure โ”‚ +โ”‚ 2๏ธโƒฃ Detect uniform arrays (tabular format applies) โ”‚ +โ”‚ 3๏ธโƒฃ Generate header: array[N]{field1,field2}: โ”‚ +โ”‚ 4๏ธโƒฃ Serialize rows with delimiter (comma/tab/pipe) โ”‚ +โ”‚ 5๏ธโƒฃ Apply intelligent quoting (only when necessary) โ”‚ +โ”‚ 6๏ธโƒฃ Handle nested structures recursively โ”‚ +โ”‚ โ”‚ +โ”‚ Supported Formats: โ”‚ +โ”‚ โœ“ Tabular arrays โ†’ items[2]{id,name}: 1,Alice 2,Bob โ”‚ +โ”‚ โœ“ Primitive arrays โ†’ tags[3]: admin,ops,dev โ”‚ +โ”‚ โœ“ Nested objects โ†’ Indentation-based (YAML-like) โ”‚ +โ”‚ โœ“ Mixed structures โ†’ List format with hyphens โ”‚ +โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜ + โ”‚ + โ”‚ Returns TOON-formatted string + โ”‚ + โ–ผ +โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” +โ”‚ TOON-Encoded Output โ”‚ +โ”‚ โ”‚ +โ”‚ Comma-delimited (default): โ”‚ +โ”‚ contacts[2]{id,name,email,score}: โ”‚ +โ”‚ c123,John Doe,john@ex.com,85 โ”‚ +โ”‚ c456,Jane Smith,jane@ex.com,72 โ”‚ +โ”‚ โ”‚ +โ”‚ Tab-delimited (more efficient): โ”‚ +โ”‚ contacts[2 ]{id name email score}: โ”‚ +โ”‚ c123 John Doe john@ex.com 85 โ”‚ +โ”‚ c456 Jane Smith jane@ex.com 72 โ”‚ +โ”‚ โ”‚ +โ”‚ With length markers: โ”‚ +โ”‚ contacts[#2]{id,name,email,score}: โ”‚ +โ”‚ c123,John Doe,john@ex.com,85 โ”‚ +โ”‚ c456,Jane Smith,jane@ex.com,72 โ”‚ +โ”‚ โ”‚ +โ”‚ Size Comparison: โ”‚ +โ”‚ JSON: 250 bytes โ†’ TOON: 140 bytes = 44% savings! ๐Ÿ’ฐ โ”‚ +โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜ + โ”‚ + โ”‚ Send to LLM API + โ”‚ + โ–ผ +โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” +โ”‚ LLM Provider (OpenAI, Claude, Gemini, etc.) โ”‚ +โ”‚ โ”‚ +โ”‚ Token Efficiency (o200k_base tokenizer used by GPT-4): โ”‚ +โ”‚ โ€ข JSON input: ~62 tokens โ”‚ +โ”‚ โ€ข TOON input: ~35 tokens (44% fewer tokens!) โ”‚ +โ”‚ โ”‚ +โ”‚ Cost Savings (GPT-4 pricing: $0.03/1K tokens): โ”‚ +โ”‚ โ€ข JSON cost: $0.00186/request โ”‚ +โ”‚ โ€ข TOON cost: $0.00105/request ($0.00081 saved per request) โ”‚ +โ”‚ โ”‚ +โ”‚ Accuracy (from official benchmarks): โ”‚ +โ”‚ โ€ข TOON accuracy: 73.9% (GPT-4, Claude, Gemini average) โ”‚ +โ”‚ โ€ข JSON accuracy: 69.7% (TOON = +4.2% better!) โ”‚ +โ”‚ โ”‚ +โ”‚ Efficiency Score (accuracy per 1K tokens): โ”‚ +โ”‚ โ€ข TOON: 26.9 (best performance) โ”‚ +โ”‚ โ€ข JSON: 15.3 (76% less efficient) โ”‚ +โ”‚ โ”‚ +โ”‚ Result: Better accuracy + 44% lower costs = ๐ŸŽฏ Win-Win! โ”‚ +โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜ +``` + +#### Encoding Process + +1. **Input Data** (JavaScript objects/arrays) +2. **Field Extraction** - Identifies all unique keys in array objects +3. **Header Creation** - Generates header with length marker and field list: `items[N]{field1,field2}:` +4. **Tabular Detection** - Checks if array objects have identical primitive fields +5. **Data Serialization** - Converts values to delimited rows (comma/tab/pipe) +6. **Intelligent Quoting** - Quotes strings only when necessary (delimiter-aware) +7. **Metrics Calculation** - Compares TOON size vs JSON size +8. **Output** - Returns TOON string + savings metrics + +**TOON Syntax Rules:** +- **Objects:** Key-value pairs with `: ` separator, indentation-based nesting +- **Primitive Arrays:** Inline format with length: `tags[3]: admin,ops,dev` +- **Tabular Arrays:** Header + rows: `users[2]{id,name}: 1,Alice 2,Bob` +- **Quoting:** Minimal - only for empty strings, leading/trailing spaces, values containing delimiter/colon/quotes, or values that look like booleans/numbers +- **Delimiters:** Comma (default), tab (`\t`), or pipe (`|`) - explicitly shown in header for non-comma + +#### Example Transformation + +**Input (JavaScript):** +```javascript +const data = { + contacts: [ + { id: '123', name: 'John', score: 85 }, + { id: '456', name: 'Jane', score: 72 } + ] +}; +``` + +**Step 1: Detect Tabular Structure** +- All objects in `contacts` array have identical primitive fields: `id`, `name`, `score` +- Tabular format applies + +**Step 2: Create Header** +```toon +contacts[2]{id,name,score}: +``` +- `[2]` = array length (2 items) +- `{id,name,score}` = field names (order matches data) +- `:` = header terminator + +**Step 3: Serialize Data Rows** +```toon +contacts[2]{id,name,score}: + 123,John,85 + 456,Jane,72 +``` +- Each row is indented 2 spaces +- Values separated by delimiter (comma is default) +- No quotes needed (simple strings/numbers) + +**Step 4: Calculate Savings** +```javascript +// JSON size: ~110 bytes (formatted with 2-space indent) +// TOON size: ~60 bytes +{ + originalSize: 110, + toonSize: 60, + bytesSaved: 50, + percentageSaved: 45.5, + estimatedTokensSaved: 12, // ~4 chars per token + estimatedCostSavings: 0.00036 // @ $0.03/1K tokens +} +``` + +**Alternative Delimiters for More Savings:** + +Using tab delimiter (`\t`) - often more token-efficient: +```toon +contacts[2 ]{id name score}: + 123 John 85 + 456 Jane 72 +``` + +Using pipe delimiter (`|`) with length marker: +```toon +contacts[#2|]{id|name|score}: + 123|John|85 + 456|Jane|72 +``` + +### Adding TOON to New Services + +#### Step-by-Step Guide + +##### 1. **Import TOON Utilities** + +```typescript +import { encodeToTOON, toTOON, prepareContactsForLLM } from '../../utils/toon-utils'; +``` + +##### 2. **Use in Your Service Method** + +**Option A: Full Metrics (Recommended for user-facing features)** +```typescript +async analyzeWithAI(data: any[]): Promise { + // Convert to TOON with automatic savings tracking + const { toonData, savings } = encodeToTOON(data, { + delimiter: '\t', + lengthMarker: true + }); + + // Log savings (optional but helpful for debugging) + console.log(`๐Ÿ’ฐ Saved ${savings.estimatedTokensSaved} tokens ($${savings.estimatedCostSavings})`); + + // Send to LLM provider + const analysis = await this.llmProvider.analyze(toonData); + + return { + ...analysis, + tokensSaved: savings.estimatedTokensSaved + }; +} +``` + +**Option B: Simple Conversion (For internal use)** +```typescript +async quickAnalysis(data: any[]): Promise { + // Simple conversion without metrics + const toonData = toTOON(data, { + delimiter: '\t', + lengthMarker: true + }); + + await this.llmProvider.process(toonData); +} +``` + +**Option C: Use Pre-built Helpers** +```typescript +async analyzeContacts(contacts: Contact[]): Promise { + // Use specialized helper for contacts + const { toonData, savings } = prepareContactsForLLM( + contacts, + ['id', 'name', 'email', 'score', 'tags'] // Only needed fields + ); + + return await this.llmProvider.analyze(toonData); +} +``` + +##### 3. **Document Token Savings** + +Add savings information to your return types: + +```typescript +export interface AIAnalysisResult { + analysis: string; + confidence: number; + tokensSaved?: number; // โœ… Add this + costSavings?: number; // โœ… Add this +} +``` + +##### 4. **Update README Examples** + +Add usage examples showing TOON benefits: + +```markdown +### Using AI Analysis with Token Savings + +\`\`\`typescript +import { formatSavingsReport } from '@gohighlevel/api-client'; + +const result = await ghl.yourService.analyzeWithAI(data); + +console.log(formatSavingsReport({ + estimatedTokensSaved: result.tokensSaved, + // ... other metrics +})); +\`\`\` +``` + +##### 5. **Test Your Implementation** + +```typescript +// Test TOON encoding +const testData = [{ id: '1', value: 'test' }]; +const { toonData, savings } = encodeToTOON(testData); + +console.assert(savings.percentageSaved > 0, 'Should have token savings'); +console.assert(toonData.includes('\t'), 'Should use tab delimiter'); +``` + +#### Best Practices + +โœ… **DO:** +- Use TOON for any data sent to LLMs (OpenAI, Claude, etc.) +- Include only necessary fields to maximize savings +- Track and report token savings to users +- Use `prepareXForLLM()` helpers when available +- Set `delimiter: '\t'` for maximum efficiency +- Enable `lengthMarker: true` for arrays + +โŒ **DON'T:** +- Use TOON for data not going to LLMs (unnecessary overhead) +- Include sensitive fields that LLM doesn't need +- Forget to handle TOON encoding errors +- Assume all LLMs understand TOON (add context in prompts) + +#### Example: Adding TOON to Voice AI Service + +```typescript +// File: lib/code/voice-ai/voice-ai.ts + +import { encodeToTOON } from '../../utils/toon-utils'; + +export class VoiceAi { + // ... existing code ... + + /** + * Analyze call transcriptions with AI sentiment detection + * Uses TOON format for 40-60% token cost reduction + */ + async analyzeCallSentiment( + callIds: string[], + options?: { llmProvider?: LLMProvider } + ): Promise { + // Get call transcriptions + const calls = await this.getCalls(callIds); + + // Prepare data with only needed fields + const callData = calls.map(call => ({ + id: call.id, + transcript: call.transcript, + duration: call.duration, + speaker: call.speaker + })); + + // Convert to TOON format (40-60% token savings!) + const { toonData, savings } = encodeToTOON(callData, { + delimiter: '\t', + lengthMarker: true + }); + + // Send to LLM for analysis + const provider = options?.llmProvider || this.defaultLLMProvider; + const analysis = await provider.analyze(toonData, { + prompt: 'Analyze sentiment of these call transcriptions (TOON format)' + }); + + return { + ...analysis, + tokensSaved: savings.estimatedTokensSaved, + costSavings: savings.estimatedCostSavings + }; + } +} +``` + +--- + +## Development Workflow + +### Branch Naming Convention + +- `feature/` - New features +- `fix/` - Bug fixes +- `docs/` - Documentation updates +- `refactor/` - Code refactoring +- `test/` - Test additions/updates + +Example: `feature/toon-integration-campaigns` + +### Commit Message Format + +Follow conventional commits: + +``` +(): + + + +