Skip to content

SteveSimonson/architect-sync

BranchesTags

Repository files navigation

Architect Sync — Fluid Architect Sync Engine

Real-time sync engine between Dropbox cloud storage and Go High Level (GHL) marketing workflows. Automatically detects new media assets in Dropbox and syncs them to GHL social posts and media library with intelligent type-based routing and campaign mapping.

Intended for use with https://automate.parsimony.com

Community

Questions and answers live in GitHub Discussions (Q&A category).

Features

  • Dropbox OAuth Integration - Secure OAuth2 authentication with automatic token refresh
  • Go High Level API Integration - Direct integration with GHL social posting and media library APIs
  • Real-time File Sync - Configurable polling intervals for continuous sync operations
  • Type-Based Routing - Intelligent routing based on file type (images → social posts, videos → media library)
  • Campaign Mapping - Link specific Dropbox folders to specific social channels for organized campaigns
  • Activity Logging - Comprehensive logging with search, filtering, and status tracking
  • Community Q&A - Setup help via GitHub Discussions
  • Responsive Dashboard UI - Modern dashboard with Material Design 3 inspired styling
  • SQLite Database - Lightweight persistent storage with Prisma ORM for scalability

Tech Stack

  • Frontend - Next.js 14 (App Router), React 18, TypeScript, Tailwind CSS
  • Backend - Next.js 14 API Routes, Node.js
  • Database - SQLite with Prisma ORM
  • APIs - Dropbox SDK v10.34+, Go High Level API v2
  • Styling - Tailwind CSS with custom Material Design 3 theme
  • State Management - Zustand for client-side state
  • Task Scheduling - node-cron for background sync operations

Getting Started

Prerequisites

  • Node.js 18 or higher
  • npm or yarn package manager
  • Dropbox developer account and App created at https://www.dropbox.com/developers/apps
  • Go High Level API key and active account
  • A text editor or IDE (VS Code recommended)

Installation

# Clone the repository
git clone <your-repo-url>
cd architect-sync

# Install dependencies
npm install

Environment Setup

# Copy the example environment file
cp .env.example .env

# Edit .env with your credentials
# Required variables:
# - DATABASE_URL: Path to SQLite database (default: file:./dev.db)
# - DROPBOX_APP_KEY: From Dropbox App Console
# - DROPBOX_APP_SECRET: From Dropbox App Console
# - DROPBOX_REDIRECT_URI: OAuth callback URL (http://localhost:3000/api/auth/dropbox/callback for dev)
# - GHL_API_KEY: From Go High Level account settings
# - GHL_BASE_URL: GHL API endpoint (default: https://rest.gohighlevel.com/v1)
# - NEXT_PUBLIC_APP_URL: Application URL (http://localhost:3000 for dev)
# - SYNC_INTERVAL_MINUTES: Polling interval in minutes (default: 15)

Database Setup

# Generate Prisma client
npm run db:generate

# Push schema to database
npm run db:push

# (Optional) Open Prisma Studio to view data
npm run db:studio

Run Development Server

npm run dev

Open http://localhost:3000 in your browser to access the dashboard.

Production Build

# Build the application
npm run build

# Start the production server
npm start

Project Structure

architect-sync/
├── src/
│   ├── app/                    # Next.js App Router
│   │   ├── (dashboard)/        # Protected dashboard routes
│   │   │   ├── page.tsx        # Dashboard home
│   │   │   ├── activity/       # Activity log page
│   │   │   ├── settings/       # Configuration page
│   │   │   └── layout.tsx      # Dashboard layout with sidebar
│   │   ├── api/                # API routes
│   │   │   ├── auth/           # Authentication endpoints
│   │   │   │   ├── dropbox/    # Dropbox OAuth flow
│   │   │   │   └── callback/   # OAuth callback handler
│   │   │   ├── sync/           # Sync trigger and status
│   │   │   ├── activity/       # Activity log queries
│   │   │   ├── settings/       # Integration settings
│   │   │   ├── dropbox/        # Dropbox file operations
│   │   │   └── ghl/            # GHL account operations
│   │   ├── layout.tsx          # Root layout
│   │   └── globals.css         # Global styles
│   ├── components/
│   │   └── layout/             # Layout components
│   │       ├── Sidebar.tsx     # Navigation sidebar
│   │       ├── TopBar.tsx      # Header/top bar
│   │       └── MobileNav.tsx   # Mobile navigation
│   ├── lib/                    # Utility functions
│   │   ├── dropbox.ts          # Dropbox SDK wrapper
│   │   ├── ghl.ts              # GHL API wrapper
│   │   ├── sync-engine.ts      # Core sync logic
│   │   └── prisma.ts           # Prisma client instance
│   └── types/                  # TypeScript type definitions
│       └── index.ts
├── prisma/
│   └── schema.prisma           # Database schema
├── public/                     # Static assets
├── .env.example                # Example environment variables
├── .gitignore                  # Git ignore rules
├── tailwind.config.ts          # Tailwind configuration
├── tsconfig.json               # TypeScript configuration
├── next.config.mjs             # Next.js configuration
├── postcss.config.mjs          # PostCSS configuration
├── package.json                # Dependencies and scripts
└── README.md                   # This file

API Routes

Authentication

GET /api/auth/dropbox

  • Generates Dropbox OAuth authorization URL
  • Returns auth URL for user redirect
  • Response: { authUrl: string }

GET /api/auth/dropbox/callback

  • OAuth callback handler
  • Exchanges authorization code for access token
  • Stores tokens in database
  • Redirects to settings page on success

Sync Operations

GET /api/sync

  • Get current sync status and dashboard statistics
  • Returns sync state, last sync time, and activity metrics
  • Response: { isRunning: boolean, lastSyncAt: Date, cursor: string, systemStatus: string, filesSynced: number, ... }

POST /api/sync

  • Trigger a manual sync operation
  • Optional body: { integrationId: string } (uses first active integration if not provided)
  • Returns: { success: true, results: { synced: number, failed: number, skipped: number } }

Activity Logs

GET /api/activity

  • Retrieve sync activity logs with pagination and filtering
  • Query parameters:
    • page (default: 1) - Page number for pagination
    • limit (default: 20) - Items per page
    • status (optional) - Filter by status: "synced", "failed"
    • fileType (optional) - Filter by type: "image", "video", "document", "other"
    • search (optional) - Search in fileName, filePath, or destination
  • Response: { logs: SyncLogEntry[], pagination: { page, limit, total, totalPages } }

Settings & Configuration

GET /api/settings

  • Retrieve current integration configuration
  • Returns primary integration with all mappings
  • API keys are masked for security
  • Response: { integration: IntegrationConfig }

POST /api/settings

  • Create a new integration configuration
  • Body: { name, dropboxFolder, dropboxAccount, ghlApiKey, ghlSubAccountId, ghlSubAccountName, syncFrequency, typeMappings, campaignMappings }
  • Response: { integration: IntegrationConfig } with status 201

PUT /api/settings

  • Update existing integration configuration
  • Body: { id: string, ...updateFields }
  • Partial updates supported
  • Response: { integration: IntegrationConfig }

Type Mappings

GET /api/settings/mappings

  • Retrieve file type to destination mappings
  • Shows how different file types are routed in GHL

POST /api/settings/mappings

  • Add new type mapping rule
  • Body: { sourceType: string, destination: string }

Dropbox Operations

GET /api/dropbox/files

  • List files from configured Dropbox folder
  • Query parameters:
    • path (optional) - Folder path to list
    • recursive (optional) - Include subfolders
  • Response: { files: DropboxFile[] }

GHL Operations

GET /api/ghl/accounts

  • Retrieve GHL social media accounts for the location
  • Used for social post routing
  • Response: { accounts: { id: string, name: string }[] }

Configuration Guide

Dropbox Setup

  1. Create a Dropbox App

  2. Generate Credentials

    • From App Console, copy:
      • App Key → DROPBOX_APP_KEY
      • App Secret → DROPBOX_APP_SECRET
    • Set OAuth Redirect URI to: http://localhost:3000/api/auth/dropbox/callback (dev) or your production URL

  3. Set Permissions

    • In App Console, go to Permissions
    • Enable these scopes:
      • files.metadata.read
      • files.content.read
      • sharing.read

  4. OAuth Flow

    • User clicks "Connect Dropbox" in settings
    • Redirected to Dropbox authorization page
    • After approval, tokens are stored in database with automatic refresh

Go High Level Setup

  1. Get API Key

    • Log in to Go High Level account
    • Navigate to Settings → API Keys
    • Create new API key or use existing
    • Copy key to GHL_API_KEY in .env

  2. Identify Location ID

    • In GHL Settings, find your Location ID
    • Set as GHL_SUB_ACCOUNT_ID in your integration

  3. Set API Endpoint

    • Use https://rest.gohighlevel.com/v1 for public API
    • Set as GHL_BASE_URL in .env

  4. Create Type Mappings

    • Configure which file types go where:
      • Images → Social Posts (automatic posting)
      • Videos → Media Library (storage only)
      • Documents → Archive/Media Library

  5. Create Campaign Mappings

    • Link Dropbox folders to specific social channels
    • Example: /Images/Instagram → Instagram account

Sync Configuration

  • Polling Interval - Set SYNC_INTERVAL_MINUTES for how often to check Dropbox (default: 15)
  • Sync Frequency - Can be overridden per integration in settings (in minutes)
  • File Type Detection - Automatic based on file extension
  • Deduplication - Files already synced are skipped on subsequent runs

Database Schema

Integration

Main integration configuration linking Dropbox and GHL

  • id - Unique identifier
  • name - Integration name
  • dropboxFolder - Source folder path
  • dropboxToken - OAuth access token
  • dropboxRefresh - Refresh token for auto-renewal
  • ghlApiKey - Go High Level API key
  • ghlSubAccountId - GHL location ID
  • syncFrequency - Polling interval in minutes
  • isActive - Whether sync is enabled

TypeMapping

Rules for routing file types to destinations

  • sourceType - File type pattern (e.g., "image", "video")
  • destination - Target in GHL (e.g., "social post", "media library")

CampaignMapping

Campaign-specific routing for organized workflows

  • dropboxFolder - Source folder for this campaign
  • socialChannel - Target social platform
  • itemCount - Number of items synced for this campaign

SyncLog

Complete activity history of all sync operations

  • fileName - Original file name
  • filePath - Dropbox path
  • fileType - Detected file type
  • destination - Where it was sent
  • status - "synced", "failed", or "pending"
  • ghlPostId - Reference to created GHL asset
  • syncedAt - Timestamp of operation

SyncState

Tracks sync progress and state across runs

  • cursor - Dropbox long polling cursor for incremental syncs
  • lastSyncAt - Last completed sync timestamp
  • filesTotal - Cumulative count of files synced
  • isRunning - Whether sync is currently in progress

Development

Scripts

npm run dev        # Start dev server with hot reload
npm run build      # Build for production
npm start          # Run production build
npm run lint       # Run ESLint
npm run db:push    # Sync Prisma schema to database
npm run db:generate # Generate Prisma client
npm run db:studio  # Open Prisma Studio GUI

Key Libraries

  • Prisma - ORM for SQLite with automatic migrations
  • Dropbox SDK - Official SDK for Dropbox API
  • Zustand - Lightweight state management (if used in components)
  • Tailwind CSS - Utility-first CSS framework
  • node-cron - Background job scheduling (future implementation)

Adding New Features

  1. New API Endpoint - Add route in src/app/api/
  2. New Database Model - Update prisma/schema.prisma then run npm run db:push
  3. New Component - Add to src/components/
  4. New Type - Add to src/types/index.ts

Troubleshooting

Dropbox Token Expired

  • Sync engine automatically refreshes tokens using refresh token
  • If manual refresh needed, disconnect and reconnect in settings

Files Not Syncing

  • Verify integration is active in settings
  • Check type mappings are configured
  • Review activity log for error details
  • Ensure Dropbox and GHL credentials are valid

Database Issues

  • Check DATABASE_URL in .env points to writable location
  • Run npm run db:push to sync schema
  • Use npm run db:studio to inspect data

API Authentication Errors

  • Verify DROPBOX_APP_KEY and DROPBOX_APP_SECRET are correct
  • Confirm GHL_API_KEY has required permissions
  • Check OAuth redirect URI matches in Dropbox app settings

Performance Considerations

  • Long Polling - Uses Dropbox long polling for incremental syncs (only changes since last sync)
  • Batch Processing - Processes files in configurable batches
  • Database Indexes - Key fields indexed for fast queries
  • File Deduplication - Prevents re-syncing already processed files
  • Concurrent Limits - Single sync operation per integration to prevent conflicts

Security Notes

  • API keys are masked in responses (only last 4 characters visible)
  • Sensitive credentials stored encrypted in database (Prisma handles at rest)
  • OAuth tokens use PKCE flow where supported
  • All API responses validated and sanitized
  • No sensitive data logged to console in production

License

MIT

Support

For issues, feature requests, or questions:

  1. Check the troubleshooting section above
  2. Review API route documentation for endpoint specifics
  3. Check activity logs for detailed error messages
  4. Consult Dropbox and Go High Level official API documentation

Contributing

Contributions welcome! Please ensure:

  • Code follows existing patterns
  • Types are properly defined
  • Database schema changes are backwards compatible
  • Tests pass (if test suite exists)
  • README is updated with new features

About

Real-time sync engine between Dropbox media assets nd Go High Level social scheduler

Resources

Readme
Activity

Stars

0 stars

Watchers

0 watching

Forks

0 forks
Report repository

Releases

Packages

Contributors

Languages