Reversed Obsidian Sync (Unofficial)

This is an unofficial Obsidian Sync library which allows you to host your own server for syncing and publishing obsidian notes. Behind the scenes, it re-routes Sync and Publish tasks from the official Obsidian servers to your own self-hosted server.

This system consists of two parts:

Self-hosted Server | Repo
Obsidian Plugin | Repo

You will first need to configure the self-hosted server which will be responsible for storing all of your Obsidian.md vault plugins and files. Then you will need to install the associated plugin in your Obsidian.md software which will be responsible for telling Obsidian to use your servers instead of the official Obsidian servers.

Warning

The main branch is the development branch. For stable usage, use the latest release.

Note

If you have the time and energy, feel free to help out with PRs or suggestions.

Index

Features
Server Installation
Plugin Installation
Build & Run
FAQ

Features

End to end encryption
Live sync (across devices)
File history/recovery/snapshots
Works natively on IOS/Android/Linux/MacOS/Windows... (via the plugin)
Vault sharing
Publish (markdown only. no rendering yet)

_{🔝 Top}

Server Installation

Follow the instructions below to set up obsidian sync with your Obsidian.md program.

Create DNS Records / SSL

If you are wanting to host your sync server outside of localhost, you can use Cloudflare to take care of your SSL certificate needs. If you do not want to use Cloudflare but still require an SSL certificate, you can skip the instructions below and utilize Certbot.

Access your Domain or Cloudflare control panel and create new records for two new subdomains.

The Content value should be the IP Address your self-hosted sync server will be hosted on.

Each subdomain plays the following roles:

Subdomain	Purpose
`api.domain.com`	Used for `Obsidian Sync`. This URL needs to be plugged into your `docker-compose.yml`, and set up in the settings of the Unofficial obi-sync plugin as the `Obsidian Sync URL`
`publish.domain.com`	Used for `Obsidian Publish`. Configure your self-hosted server, enable the `Obsidian Plugin`; then upload your vault files using `Obsidian Publish`. View your uploaded files at: `https://publish.domain.com/vaultname/Path/To/File.md`

If you want Cloudflare to handle the SSL certificate, set each record's Proxy Status to Proxied

Docker-Compose (Option 1)

If using this option, you must decide which setup you would like to use.

One File: docker-compose.yml
Two Fles: docker-compose.yml and .env

Using the two file option is slightly more secure in regards to your SIGNUP_KEY being exposed to logs.

If you are not concerned about security, use DOCKER-COMPOSE.YML ONLY option.
If you want the extra security, use DOCKER-COMPOSE.YML + .ENV option.

🟢 DOCKER-COMPOSE.YML ONLY

Create a new file named docker-compose.yml, and then paste the corresponding code provided below:

version: '3.8'

services:
  obsidian_sync:
    image: ghcr.io/acheong08/obi-sync:latest
    container_name: obsidian_sync
    restart: always
    ports:
      - 3000:3000
    environment:
      - DOMAIN_NAME=api.domain.com
      - ADDR_HTTP=0.0.0.0:3000
      - SIGNUP_KEY=YOUR_PRIVATE_STRING_HERE
      - DATA_DIR=/obi-sync/
      - MAX_STORAGE_GB=10
      - MAX_SITES_PER_USER=5
    volumes:
      - ./obi-sync:/obi-sync

volumes:
  obi-sync:

_{🔹Continue With Installation}

🟢 DOCKER-COMPOSE.YML + .ENV

Create the following two files in the same folder, and then paste the corresponding code provided below:

docker-compose.yml
.env

📄 docker-compose.yml

version: '3.8'
services:
  obsidian_sync:
    image: ghcr.io/acheong08/obi-sync:latest
    container_name: obsidian_sync
    restart: always
    ports:
      - 3000:3000
    environment:
      DOMAIN_NAME: ${DOMAIN}
      ADDR_HTTP: ${ADDR_HTTP}
      SIGNUP_KEY: ${USER_SIGNUP_KEY}
      DATA_DIR: ${DIR_DATA}
      MAX_STORAGE_GB: ${USER_MAX_STORAGE}
      MAX_SITES_PER_USER: ${USER_MAX_SITES}
    volumes:
      - ./obi-sync:/obi-sync

volumes:
  obi-sync:

📄 .env

DOMAIN='api.domain.com'
ADDR_HTTP='0.0.0.0:3000'
DIR_DATA='/obi-sync/'
USER_SIGNUP_KEY='YOUR_PRIVATE_STRING_HERE'
USER_MAX_STORAGE=10
USER_MAX_SITES=5

_{🔹Continue With Installation}

Variable List

After creating docker-compose.yml / .env file(s), edit the variables to match your domain DOMAIN_NAME, registration signup key SIGNUP_KEY, etc.

A description of each variable is provided below:

Variable	Description	Required	Default
`DOMAIN_NAME`	This is the URL to your API subdomain	Yes	`localhost:3000`
`ADDR_HTTP`	The address to run Obi-Sync on	No	`127.0.0.1:3000`
`SIGNUP_KEY`	Required later when creating users who will be able to access your self-hosted server	No	None
`DATA_DIR`	Where encrypted `vault.db` and other files will be stored	No	`./`
`MAX_STORAGE_GB`	The maximum storage per user in GB.	No	`10`
`MAX_SITES_PER_USER`	The maximum number of sites per user.	No	`5`

Start/Stop Docker

After you finish configuring your docker's variables, you need to cd to the folder where you created the file(s) and execute:

docker compose up -d

To shut down the container:

docker compose down

To confirm container is running:

docker ps

Docker (Option 2)

To install obi-sync using docker pull, you can run the default command below:

docker pull ghcr.io/acheong08/obi-sync:latest

For a full list of pull commands based on your operating system, view Package Release.

Nginx Configuration

You must configure nginx to handle your new subdomains by adding a few entries to your nginx config file. If you wish to add this to your current running nginx webserver, you can paste it somewhere in \etc\nginx\sites-enabled\default.

Note

Do not copy/paste the server blocks below without looking them over. If you have changed port 3000 in your docker container, you must change it below to that same port. Also change domain.com to your correct domain.

map $http_upgrade $connection_upgrade {
        default upgrade;
        '' close;
}

#
#   Obsidian Sync
#
#   defined within the obi-sync plugin for Obsidian so that Obsidian and
#   the self-hosted server can sync files.
#

    server
    {
        listen 443 ssl http2;
        listen [::]:443 ssl http2;
        listen 80;
        listen [::]:80;

        server_name         	www.api.domain.com api.domain.com;

        location /
        {
            proxy_http_version  1.1;
            proxy_set_header    Upgrade $http_upgrade;
            proxy_set_header    Connection $connection_upgrade;
            proxy_set_header    Host $host;
            proxy_pass          http://127.0.0.1:3000/;
        }
        
    }

#
#   Obsidian Publish
#
#   used for viewing published Obsidian .md files on a webserver.
#

    server
    {
        listen 443 ssl http2;
        listen [::]:443 ssl http2;
        listen 80;
        listen [::]:80;

        server_name         	www.publish.domain.com publish.domain.com;
        
        location /
        {
            proxy_pass     	http://127.0.0.1:3000/published/;
        }
    }

After modifying your nginx service, restart it to load up the new configs.

sudo systemctl restart nginx

Or restart nginx via docker if you run it through a container.

Creating New User

You must create a user that will be allowed to access your self-hosted server from within Obsidian.md. This can be done by opening Powershell in Windows, or Terminal in Linux and executing the following:

Note

email: This is the email address you will use in Obsidian's About tab to sign into your self-hosted server.

password: Pick any password you wish to use. This is the password you will use in the Obsidian.md program once you configure the plugin to connect to your self-hosted server.

name: Can be anything, not super important.

signup_key: Variable you provided in docker-compose.yml or .env file.
If you removed signup_key from your docker container's variables and don't want to require a signup key for registration, remove that line from the command below.

⚙️ Windows `Powershell`

curl --request POST `
  --url https://api.domain.com/user/signup `
  --header 'Content-Type: application/json' `
  --data '{
	"email": "example@example.com",
	"password": "example_password",
	"name": "Example User",
	"signup_key": "<SIGNUP_KEY>"
}'

⚙️ Linux `Terminal`

curl --request POST \
  --url https://api.domain.com/user/signup \
  --header 'Content-Type: application/json' \
  --data '{
	"email": "example@example.com",
	"password": "example_password",
	"name": "Example User",
	"signup_key": "<SIGNUP_KEY>"
}'

A successful registration will return the following response:

{"email":"example@example.com","name":"Example User"}

A failed registration will return the following response:

Warning

If you receive a failure from registration, make sure you're not trying to sign up the same user multiple times.

{"error":"not sign up!"}

_{🔝 Top}

Plugin Installation

In order for your new self-hosted Publish and Sync server to function properly, you must install a plugin to your copy of Obsidian.md

Install with WGET

Navigate to your vault's .obsidian folder:

cd /path/to/vault/.obsidian

Create a new folder for the plugin and enter the folder:

mkdir -p plugins/custom-sync-plugin && cd plugins/custom-sync-plugin

Use wget to download the plugin files:

wget https://github.com/acheong08/rev-obsidian-sync-plugin/raw/master/main.js https://github.com/acheong08/rev-obsidian-sync-plugin/raw/master/manifest.json

Install Manually

To manually get the latest copy of the unofficial Obi-Sync Plugin, download here.

Navigate to the folder where your vault is on your local machine, and enter the .obsidian\plugins\ folder.
Create a new folder: custom-sync-plugin
Inside the custom-sync-plugin folder, install the plugin files:
- 📄 main.js
- 📄 manifest.json

Note

Alternatively, you can use https://github.com/TfTHacker/obsidian42-brat which can be found in the official community plugins list.

Enable Plugin

Once the plugin is installed, activate it by launching Obsidian.md.

Open Obsidian Settings
On left menu, click Community Plugins
Scroll down to Custom Native Sync and enable
To the left of the enable button for Custom Native Sync, press the options button.
Configure the Obsidian Sync URL in the Plugin settings after setting up your sync server
- By default, it is https://api.domain.com
Go to the Core Plugins section of Obsidian
Locate the core plugin Sync and enable

Configure Sync

Before doing these steps, ensure that your obi-sync self-hosted server is running.

Open your Obsidian.md Settings and select About.

You need to sign in and connect Obsidian.md to your self-hosted server by clicking Log In:

The login dialog will appear:

Fill in your email address and password that you used to register your account with the cURL command in the section Creating New User above.

curl --request POST `
  --url https://api.domain.com/user/signup `
  --header 'Content-Type: application/json' `
  --data '{
	"email": "example@example.com",      <----- The email you will use
	"password": "example_password",	     <----- The password you will use
	"name": "Example User",
	"signup_key": "<SIGNUP_KEY>"
}'

Note

After signing in successfully, the Log In button will change to Manage settings. Keep in mind though that clicking the Manage Settings button will take you to Obsidian's official website. It has nothing to do with your own self-hosted server.

Next, on the left side under Core Plugins, select Sync.

On the right side, click the Choose button:

A new dialog will then appear. Select Create New Vault

Then fill in the information for your new vault. The Encryption Password can be anything you want. Do not lose this password. You cannot unlock / decrypt your vault if you can't remember it.

Your new vault will be created, along with a few options you can select, and a Connect button.

Click the Connect button to start Syncing between your local vault and your self-hosted sync server. If you get a warning message labeled Confirm Merge Vault, click Continue.

If you provided an Encryption Password a few steps back, you will now be asked to enter that password, and then click Unlock Vault.

You will then get one more confirmation that your vault is now synced.

From this point on, you can adjust any of the Sync settings you wish to modify, which includes syncing things like images, videos, plugins, settings, and more. Whatever options you enable, will be included in the Sync job.

Ensure Sync Status is set to running. You can enable / disable the sync's state by clicking the button to the right.

You can confirm that your files are successfully syncing by clicking the View button to the right of Sync Activity.

Once pressing the View button, a new dialog will appear and show you the status of your Sync job.

2023-09-09 01:38 - Uploading file Test Folder/Subfolder/My File 1.md
2023-09-09 01:38 - Upload complete Test Folder/Subfolder/My File 1.md
2023-09-09 01:38 - Uploading file Test Folder/Subfolder/My File 2.md
2023-09-09 01:38 - Upload complete Test Folder/Subfolder/My File 2.md
2023-09-09 01:40 - Fully synced

Configure Publish

These instructions help guide you through setting up Obsidian Publish. This feature will allow you to view your Vault .md notes on your webserver.

Open your Obsidian.md settings, and under Options on the left, select Core Plugins. In the middle of the screen, enable Publish.

Back on the main Obsidian interface, select the Publish Changes button in your Obsidian side / toolbar.

In the new dialog, enter a Site ID in the textfield provided. This will become the slug name for your new vault.

Once you have a name, click Create.

You will then see a list of all your vault's associated files. Select the checkbox to the left of each folder you wish to upload to your self-hosted Publish server.

Once you've selected the desired folders, click Publish in the bottom right.

Yet another dialog will appear which confirms your uploaded vault documents.

Note

Because we are hosting our own server, the link provided at the bottom of the dialog will not work since it goes to Obsidian's official publish server.

Open your web browser and go to the URL for your self-hosted publish server. In our example, we would view our Testing Folder/Tags.md page by entering the following URL:

https://publish.domain.com/myvault/Testing Folder/Tags.md

myvault: Name given to example vault
Testing Folder: Folder name the note resides in
Tags.md: Note filename

To see an overview of files you have uploaded to your publish server, go to your publish subdomain, and add your vault name at the end. Nothing else needs added.

https://publish.domain.com/myvault

In our example vault, visiting the URL above displays JSON and includes a list of all files uploaded from the test vault:

From this point, you can upload whatever files you wish to have published and play around with the settings.

_{🔝 Top}

Build & Run

Note

This requires you to install the Go interrupter on your machine. You may download it here.

To build and run Obi-sync directly from the git repo, execute the following:

git clone https://github.com/acheong08/obi-sync
cd obsidian-sync
go run cmd/obsidian-sync/main.go

_{🔝 Top}

FAQ

Where Are Files Stored?

This depends on how you've configured obsidian sync server to run. If you installed this project using Install with Docker-compose, and specified the environment variable DATA_DIR, then the files should exist in the same folder you specified the variable to use. By default, vault files are placed in the folder obi-sync.

If you installed this project using Install with Docker or did not specify a variable for DATA_DIR, then the files are typically stored in /var/lib/docker/*

Some users may be running Portainer, which allows you to view your docker containers and volumes within a graphical user interface. Login to the portainer web admin panel and under Volumes, find out which volume is assigned to obsidian_sync, and copy the value Mount path. Open your File Explorer and go to that location to view your vault files.

What Files Are Created On Server?

This project will store your vault data in the following files:

📁 obi-sync
   📄 publish.db
   📄 secret.gob
   📄 vault.db
   📄 vaultfiles.db

Error: User Not Signed Up

This error typically occurs when you're creating a user with cURL and shows the following:

{"error":"not sign up!"}

If you receive this error when creating your first API user, ensure the user did not previously exist.

Network Error Occured. Network Unavailable

If you are attempting to use the Obsidian Publish feature and receive the error:

Ensure you have provided the correct environment variable DOMAIN_NAME=api.domain.com. If you provided the wrong domain name and need to change it, you must do the following:

Edit docker-compose.yml
Change DOMAIN_NAME variable to correct domain.
Locate the project file publish.db and DELETE it completely.
Restart the docker container and Obsidian.md program

Once back in Obsidian.md program, click Publish Changes button again.

You can also attempt to locate the root cause of the issue by pressing CTRL + SHIFT + I inside of Obsidian.md. Then on the right side, click Network and then re-open the Publish Changes interface again.

Anything listed in red is an error and needs to be corrected. Ensure that it is trying to fetch the correct domain from the Request URL. Anything listed in white and with a Status Code of 200 is properly connecting.

Error: No Matching Manifest linux/amd64

Make sure you are pulling the correct docker image.

You can also visit the Package Release page, select OS / Arch tab, and copy the pull command listed below linux/amd64

Docker-compose.yml vs .Env File

In the section above titled Install with Docker-compose, there are two ways to install obi-sync using docker-compose.

Single docker-compose.yml file
- See DOCKER-COMPOSE.YML ONLY
docker-compose.yml and .env file
- See DOCKER-COMPOSE.YML + .ENV

🟢 Single `docker-compose.yml` File

This method involves creating a single docker-compose.yml file which will hold all of your settings for this project.

🟢 `docker-compose.yml` and `.env` File

This method requires you to create two files which both exist in the same folder. The benefit of this method is that your SIGNUP_KEY environment variable will not be leaked in your docker logs and is slightly better if you are worried about security.

You can decide to use either one of the two options.

If using Portainer web manager for Docker, you can access the environment variables clicking Containers on left side menu, then click obsidian-sync in your list of containers. Then scroll down to the ENV section.

Can't Find `.obsidian` Folder or `.Env` File

The .obsidian plugin folder and docker .env file may be hidden. You can configure your operating system's File Explorer show hidden files or use a terminal.

Linux: By default, this OS hides all folders and files that start with a period. To view these files in your File Explorer, press the key combination CTRL + H. You may be asked for your account or sudo password.

iOS: You may need to plug the device into a computer, then to access your .obsidian folder, go to Documents on iPhone, then go to Obsidian/<your vault name>/.obsidian.

Privacy & Security

The plugin and self-hosted server are dead simple. No data is collected. You can even run your sync server on the local network without access to the internet. All vault data is stored behind an encrypted password and inside a database file.

_{🔝 Top}

Name		Name	Last commit message	Last commit date
Latest commit History 158 Commits
README.md		README.md

Aetherinox/obi-sync-docs

Folders and files

Latest commit

History

README.md

README.md

Repository files navigation

Reversed Obsidian Sync (Unofficial)

Index

Features

Server Installation

Create DNS Records / SSL

Docker-Compose (Option 1)

🟢 DOCKER-COMPOSE.YML ONLY

🟢 DOCKER-COMPOSE.YML + .ENV

📄 docker-compose.yml

📄 .env

Variable List

Start/Stop Docker

Docker (Option 2)

Nginx Configuration

Creating New User

⚙️ Windows Powershell

⚙️ Linux Terminal

Plugin Installation

Install with WGET

Install Manually

Enable Plugin

Configure Sync

Configure Publish

Build & Run

FAQ

Where Are Files Stored?

What Files Are Created On Server?

Error: User Not Signed Up

Network Error Occured. Network Unavailable

Error: No Matching Manifest linux/amd64

Docker-compose.yml vs .Env File

🟢 Single docker-compose.yml File

🟢 docker-compose.yml and .env File

Can't Find .obsidian Folder or .Env File

Privacy & Security

About

Topics

Resources

Stars

Watchers

Forks

Releases

Packages 0

⚙️ Windows `Powershell`

⚙️ Linux `Terminal`

🟢 Single `docker-compose.yml` File

🟢 `docker-compose.yml` and `.env` File

Can't Find `.obsidian` Folder or `.Env` File

Packages