atproto self-host in docker

[syui]

atproto: atproto.com

A fully self-hosted AT Protocol environment. While typical self-hosting only covers PDS, this setup includes the appview, relay, moderation, feed generator, and more.

Architecture

+-----------+    +-----------+    +-----------+
|    PDS    |--->|    BGS    |--->|   bsky    |
|   Data    |    |   Relay   |    |  appview  |
+-----------+    +-----+-----+    +-----+-----+
                       |                |
                 +-----v-----+   +-----v-----+
                 | Jetstream |   |   Ozone   |
                 |  WS feed  |   | moderation|
                 +-----+-----+   +-----------+
                       |
                 +-----v-----+
                 |   Feed    |
                 |  custom   |
                 +-----------+

Services

Service	Role	Description
pds	User data storage, authentication, OAuth	Personal Data Server
bgs	Firehose aggregation and relay	Big Graph Service (Relay)
bsky	AppView -- timeline, search, notification API	Application View
plc	DID-to-handle resolution (like DNS)	DID Registry
ozone	Moderation, verification badges	Moderation Service
jetstream	Lightweight WebSocket distribution from BGS	WebSocket Relay
feed-generator	Custom feed generator	Feed Generator
social-app	Web frontend (bskyweb)	Social App
bsky.chat	DM service	Chat Server

Data Flow

How a post appears on the timeline

User -> PDS (save post) -> BGS (firehose) -> bsky (index) -> Timeline

Custom feed flow

BGS -> Jetstream (WS) -> Feed Generator (filter & store) -> Client calls feed API

Federation with the official network

To make posts from your PDS visible on the official Bluesky network, request a crawl from the official relay.

bgs.bsky.social is the old relay (deprecated, now requires auth). Use bsky.network instead.

curl -X POST "https://bsky.network/xrpc/com.atproto.sync.requestCrawl" \
  -H "Content-Type: application/json" \
  -d '{"hostname": "example.com"}'

Also set PDS_CRAWLERS=https://bsky.network in your PDS configuration.

Repositories

Service	Repository	Dockerfile
pds	bluesky-social/atproto	services/pds/Dockerfile
bsky	bluesky-social/atproto	services/bsky/Dockerfile
ozone (daemon)	bluesky-social/atproto	services/ozone/Dockerfile
bgs	bluesky-social/indigo	cmd/relay/Dockerfile
plc	did-method-plc/did-method-plc	packages/server/Dockerfile
social-app	bluesky-social/social-app	Dockerfile
ozone-web	bluesky-social/ozone	Dockerfile
jetstream	bluesky-social/jetstream	Dockerfile
feed-generator	bluesky-social/feed-generator	Dockerfile

pds, bsky, and ozone (daemon) are built from the same repository (atproto) using different Dockerfiles.

Build

git clone https://github.com/bluesky-social/atproto
git clone https://github.com/bluesky-social/indigo
git clone https://github.com/did-method-plc/did-method-plc
git clone https://github.com/bluesky-social/social-app
git clone https://github.com/bluesky-social/ozone
git clone https://github.com/bluesky-social/jetstream
git clone https://github.com/bluesky-social/feed-generator

Environment Variables

postgres

POSTGRES_USER=postgres
POSTGRES_PASSWORD=xxx
POSTGRES_DB=postgres

pds

PDS_HOSTNAME=example.com
PDS_SERVICE_DID=did:web:example.com
PDS_DB_POSTGRES_URL=postgres://postgres:xxx@database/pds
PDS_DATA_DIRECTORY=/data
PDS_BLOBSTORE_DISK_LOCATION=/data/blobs
PDS_BSKY_APP_VIEW_DID=did:web:bsky.example.com
PDS_BSKY_APP_VIEW_URL=https://bsky.example.com
PDS_CRAWLERS=https://bgs.example.com
PDS_DID_PLC_URL=https://plc.example.com
PDS_SEQUENCER_ENABLED=true
PDS_SEQUENCER_DB_LOCATION=/data/sequencer
PDS_DEV_MODE=true
PDS_ENABLE_DID_DOC_WITH_SESSION=true
PDS_INVITE_REQUIRED=true
PDS_INVITE_INTERVAL=604800
PDS_EMAIL_FROM_ADDRESS=admin@example.com
# secrets
PDS_ADMIN_PASSWORD=xxx
PDS_JWT_SECRET=xxx
PDS_PLC_ROTATION_KEY_K256_PRIVATE_KEY_HEX=xxx
PDS_REPO_SIGNING_KEY_K256_PRIVATE_KEY_HEX=xxx
PDS_EMAIL_SMTP_URL=xxx

bsky

BSKY_PORT=2584
BSKY_PUBLIC_URL=https://bsky.example.com
BSKY_SERVER_DID=did:web:bsky.example.com
BSKY_DB_POSTGRES_URL=postgres://postgres:xxx@database/bsky
BSKY_DID_PLC_URL=https://plc.example.com
BSKY_REPO_PROVIDER=wss://bgs.example.com
BSKY_BLOB_CACHE_LOC=/data
BSKY_DATAPLANE_URLS=http://localhost:6100
BSKY_DATAPLANE_PORT=6100
BSKY_DATAPLANE_HTTP_VERSION=1.1
BSKY_BSYNC_URL=http://localhost:6200
BSKY_BSYNC_PORT=6200
BSKY_BSYNC_HTTP_VERSION=1.1
BSKY_COURIER_URL=
MOD_SERVICE_DID=did:web:ozone.example.com
# secrets
BSKY_ADMIN_PASSWORDS=xxx
BSKY_SERVICE_SIGNING_KEY=xxx

bgs

DATABASE_URL=postgres://postgres:xxx@database/bgs
CARSTORE_DATABASE_URL=postgres://postgres:xxx@database/carstore
DATA_DIR=/data
ATP_PLC_HOST=https://plc.example.com
BGS_NEW_PDS_PER_DAY_LIMIT=1000
# secrets
BGS_ADMIN_KEY=xxx

plc

DATABASE_URL=postgres://postgres:xxx@database/plc
DB_CREDS_JSON={"username":"postgres","password":"xxx","host":"database","port":"5432","database":"plc"}
ENABLE_MIGRATIONS=true
DB_MIGRATE_CREDS_JSON={"username":"postgres","password":"xxx","host":"database","port":"5432","database":"plc"}

ozone

OZONE_SERVER_DID=did:web:ozone.example.com
OZONE_PUBLIC_URL=https://ozone.example.com
OZONE_ADMIN_HANDLE=admin.example.com
OZONE_ADMIN_DIDS=did:plc:xxx
OZONE_MODERATOR_DIDS=did:plc:xxx
OZONE_DB_POSTGRES_URL=postgres://postgres:xxx@database/ozone
OZONE_DID_PLC_URL=https://plc.example.com
OZONE_APPVIEW_DID=did:web:bsky.example.com
OZONE_APPVIEW_URL=https://bsky.example.com
OZONE_APPVIEW_PUSH_EVENTS=true
OZONE_PDS_DID=did:web:example.com
OZONE_PDS_URL=https://example.com
OZONE_DEV_MODE=true
OZONE_DB_MIGRATE=true
NEXT_PUBLIC_PLC_DIRECTORY_URL=https://plc.example.com
NEXT_PUBLIC_OZONE_SERVICE_DID=did:web:ozone.example.com
NEXT_PUBLIC_SOCIAL_APP_DOMAIN=example.com
NEXT_PUBLIC_SOCIAL_APP_URL=https://example.com
# verifier (for bluecheck)
OZONE_VERIFIER_DID=did:plc:xxx
OZONE_VERIFIER_URL=https://example.com
OZONE_VERIFIER_PASSWORD=xxx
OZONE_VERIFIER_ISSUERS_TO_INDEX=did:plc:xxx
OZONE_VERIFIER_JETSTREAM_URL=wss://jetstream.example.com
# secrets
OZONE_ADMIN_PASSWORD=xxx
OZONE_SIGNING_KEY_HEX=xxx

jetstream

JETSTREAM_WS_URL=ws://bgs:2470/xrpc/com.atproto.sync.subscribeRepos
JETSTREAM_DATA_DIR=/data
JETSTREAM_LISTEN_ADDR=:6008
JETSTREAM_METRICS_LISTEN_ADDR=:6009
JETSTREAM_LIVENESS_TTL=300

feed-generator

FEEDGEN_PORT=3000
FEEDGEN_LISTENHOST=0.0.0.0
FEEDGEN_SQLITE_LOCATION=/data/db.sqlite
FEEDGEN_HOSTNAME=feed.example.com
FEEDGEN_PUBLISHER_DID=did:plc:xxx
FEEDGEN_SERVICE_DID=did:web:feed.example.com
FEEDGEN_JETSTREAM_URL=ws://jetstream:6008/subscribe

social-app

ATP_APPVIEW_HOST=bsky.example.com
EXPO_PUBLIC_BLUESKY_PROXY_DID=did:web:bsky.example.com
EXPO_PUBLIC_ENV=production

chat

bsky.chat

PORT=3100
HOST=0.0.0.0
DB_PATH=/data/chat.db
DID_HOST=bsky.example.com
SERVICE_URL=https://bsky.example.com
CHAT_DOMAIN=bsky.chat
APPVIEW_URL=https://bsky.example.com

Docker Compose Example

services:
  database:
    image: postgres
    env_file: ./envs/postgres
    volumes:
      - database:/var/lib/postgresql/data
  redis:
    image: redis
  plc:
    build:
      context: ./did-method-plc/
      dockerfile: packages/server/Dockerfile
    env_file: ./envs/plc
  pds:
    build:
      context: ./atproto/
      dockerfile: services/pds/Dockerfile
    env_file: ./envs/pds
    volumes:
      - pds:/data
  bsky:
    build:
      context: ./atproto/
      dockerfile: services/bsky/Dockerfile
    env_file: ./envs/bsky
    volumes:
      - bsky:/data
  bgs:
    build:
      context: ./indigo/
      dockerfile: cmd/relay/Dockerfile
    env_file: ./envs/bgs
    volumes:
      - bgs:/data
  ozone:
    build:
      context: ./atproto/
      dockerfile: services/ozone/Dockerfile
    env_file: ./envs/ozone
  ozone-web:
    build:
      context: ./ozone/
    env_file: ./envs/ozone
  jetstream:
    build:
      context: ./jetstream/
    env_file: ./envs/jetstream
    volumes:
      - jetstream:/data
  feed-generator:
    build:
      context: ./feed-generator/
    env_file: ./envs/feed-generator
    volumes:
      - feed-generator:/data
  social-app:
    build:
      context: ./social-app/
    env_file: ./envs/social-app

volumes:
  database:
  pds:
  bsky:
  bgs:
  jetstream:
  feed-generator:

docker compose build
docker compose up -d

Lexicons

The schema definition system for AT Protocol. In addition to standard collections (app.bsky.*), you can define custom collections.

Creating Custom Lexicons

1. Add a TXT record to the domain's DNS
2. Register the schema under com.atproto.lexicon.schema

Pattern	Use Case	Example
self	One record per user	Profile extensions, settings
TID	Time-series data	Posts, chat messages, logs

ref

• https://github.com/itaru2622/bluesky-selfhost-env
• https://github.com/ikuradon/atproto-starter-kit

[bnewbold]

There is an out-of-date docker-compose file and README in the indigo repo that might be helpful: https://github.com/bluesky-social/indigo/tree/main/cmd/fakermaker

Things are not in a great state for this right now. We plan to iterate on all the docs and dev support around self-hosting at the same time we open up federation, which means we aren't prioritizing updating docs on this until then.

In particular, I think the pds service is probably pretty out of date compared to the sandbox PDS "distribution", and we need to get those synced back up, and that may involve breaking changes to the PDS config env vars. Also the version of the PDS that we (bluesky) runs is now behind a "gateway service", which is unnecessary complexity for most folks (but needs documentation). The relay (BGS) and PLC are pretty stable. The appview is in the middle of several large refactors, with big changes in feature branches which have not been merged yet.

[itaru2622]

@syui did you succeed on self hosting with current codes? please update if you got progress.

[syui]

yes, my account https://syu.is

[itaru2622]

could you share how you configure those? I also want to get self-hosting environment but failed configuring palomar (search service in indigo/cmd)

[syui]

I have compiled information for self-hosting here.
We recommend sandbox first.
https://github.com/bluesky-social/pds

[itaru2622]

hmm, finally I got my self-hosting environment for latest bluesky including bgs, plc, appview, pds, and others.
my codes are located https://github.com/itaru2622/bluesky-selfhost-env and self-hosting domain name is easy tune-able by environment variable. enjoy ;-)

[syui]

if bsky(appview) doesn't work

• anyone maintaining code for docker containers? #2334
• https://github.com/itaru2622/bluesky-selfhost-env/blob/master/patching/105-atproto-services-for-docker.diff