- for: https://github.com/TheExGenesis/community-archive
- ref: https://github.com/DefenderOfBasic/twitter-semantic-search
# install Bun
curl -fsSL https://bun.sh/install | bash
# install dependencies, qdrant binary is ~70mb
bun install
# add .env for community archive supabase
# VITE_PUBLIC_SUPABASE_URL=
# VITE_PUBLIC_SUPABASE_ANON_KEY=
# start all services in one terminal
bun start
The script will:
- Start the Qdrant vector database
- Start the API server that talks to Qdrant
- Start the UI server
Once everything is running, open http://localhost:5173 in your browser
You can import your own Twitter archive by following these steps:
-
Download your Twitter archive:
- Go to https://x.com/settings/download_your_data
- Request your archive and wait for the email (can take a few days)
- Download the
zip
file from the email link
-
Import the archive using the UI:
- Click the import button in the top-right corner of the UI (or press
⌘ + I
) - Select "Upload File" tab
- Drag and drop your Twitter archive ZIP file or click to select it
- Optionally provide a custom username for your archive
- Click "Import"
- Click the import button in the top-right corner of the UI (or press
You can import tweets directly from the UI:
- Click the import button in the top-right corner of the UI (or press
⌘ + I
) - Choose between importing by username or uploading a Twitter/X archive JSON file
The system now maintains an import history file (packages/server/import-history.json
) that tracks:
- Last import date for each username
- Latest tweet date for each username
- Total tweet count imported for each username
This makes subsequent imports much faster by skipping over processed tweets
If you prefer to run services separately, you can use the following commands:
# qdrant binary (~70mb) + data folder is in packages/qdrant-local/bin
bun install
# terminal #1: run qdrant
bun run qdrant
# terminal #2: start server that talks between qdrant and ui
bun run dev:qdrant
# terminal #3: run the UI
bun run ui
# Test qdrant
bun run test:qdrant
# run supabase (need Docker)
# dashboard: http://localhost:54323
bunx supabase login
bunx supabase start
# add to ui/.env
VITE_LOCAL_SUPABASE_URL=http://localhost:54321
VITE_SUPABASE_ANON_KEY=<anon key>