Skip to content

Latest commit

 

History

History
205 lines (134 loc) · 10.4 KB

SETUP.md

File metadata and controls

205 lines (134 loc) · 10.4 KB
Raw

Server Architecture

The Server is divided into a number of services. Each service is a Visual Studio project in the Server solution. These are:

  • Admin
  • Api
  • Icons
  • Identity
  • Notifications
  • SQL

Each service is built and run separately. The Bitwarden clients can use different servers for different services.

This means that you don't need to run all services locally for a development environment. You can run only those services that you intend to modify, and use Bitwarden.com or a self-hosted instance for all other services required.

By default some of the services depends on the Bitwarden licensed CommCore, however it can easily be disabled by including the /p:DefineConstants="OSS" as an argument to dotnet.

Local Development Environment Setup

This guide will show you how to set up the Api, Identity and SQL projects for development. These are the minimum projects for any development work. You may need to set up additional projects depending on the changes you want to make.

We recommend using Visual Studio, and PowerShell which is used for the helper scripts.

Docker containers

To simplify the setup process we provide a Docker Compose application model. This is split up into multiple service profiles to facilitate easily customization.

Some settings can be customized by modifying the dev/.env file, such as the MSSQL_PASSWORD which should be modified before starting the project.

# Copy the example environment file
cp ./.env.example ./.env

# We recommend running the following command when developing for self-hosted
docker compose --profile mssql --profile mail up

# We also provide a storage profile which uses Azurite to emulate some services used by the cloud instance
#  Usually only needed by internal Bitwarden developers
docker compose --profile cloud --profile mail up

SQL Server

We recommend changing the MSSQL_PASSWORD variable in dev/.env to avoid exposing the sqlserver with a default password. Note: changing this after first running docker compose may require a re-creation of the storage volume. To do this, stop the running containers and run docker volume rm bitwardenserver_mssql_dev_data. (Warning: this will delete your development database.)

We provide a helper script which will create the development database vault_dev and also run all migrations. This command should be run after starting docker the first time, as well as after syncing against upstream and after creating a new migration.

.\dev\migrate.ps1

# You can also re-run the last migration using
.\dev\migrate.ps1 -r

Note: If all or many migrations are skipped even though this is a new database, make sure that there is not a last_migration file located in dev/.data/mssql. If there is, remove it and run the helper script again. This can happen if you create the database, run migrations, then delete it.

Azurite

Azurite is a emulator for Azure Storage API and supports Blob, Queues and Table storage. We use it to avoid a hard dependency on online services for cloud development.

To bootstrap the local Azurite instance please run the following command:

# This script requires the Az module, which can be installed using
Install-Module -Name Az -Scope CurrentUser -Repository PSGallery -Force

.\dev\setup_azurite.ps1

Mailcatcher

Since the server uses emails for many user interactions a working SMTP server is a requirement, we provide a pre-setup instance of MailCatcher which exposes a web interface at http://localhost:1080.

Certificates

In order to run Bitwarden, we require two certificates which for local development can be resolved by using self signed certificates.

Windows

We provide a helper script which will generate and add the certificates to the users Certificate Store. After running the script it will output the thumbprints needed for the next step. The certificates can later be accessed using certml.msc under Personal/Certificates.

.\create_certificates_windows.ps1

   PSParentPath: Microsoft.PowerShell.Security\Certificate::CurrentUser\My
Thumbprint                                Subject
----------                                -------
0BE8A0072214AB37C6928968752F698EEC3A68B5  CN=Bitwarden Identity Server Dev
C3A6CECAD3DB580F91A52FC9C767FE780300D8AB  CN=Bitwarden Data Protection Dev

MacOS

We provide a helper script which will generate the certificates and add them to the keychain.

Note: You should update the Trust options for each certificate to always trust using Keychain Access.

./create_certificates_mac.sh

Certificate fingerprints:
Identity Server Dev: 0BE8A0072214AB37C6928968752F698EEC3A68B5
Data Protection Dev: C3A6CECAD3DB580F91A52FC9C767FE780300D8AB

User Secrets

User secrets are a method for managing application settings on a per-developer basis. They are stored outside of the local git repository so that they are not pushed to remote.

User secrets override the settings in appsettings.json of each project. Your user secrets file should match the structure of the appsettings.json file for the settings you intend to override.

For more information, see: Safe storage of app secrets in development in ASP.NET Core.

Automated Helper script

We provide a helper scripts which simplifies setting user secrets for all projects in the repository.

Start by copying the secret.json.example file to secrets.json and modify the existing settings and add any other required setting. Afterwards run the following command which will add the settings to each project in the bitwarden repository.

.\setup_secrets.ps1

# The script also supports an optional flag which removes all existing settings before re-applying them
.\setup_secrets.ps1 -clear 1

Manually creating and modifying

It is also possible to manually create and modify the user secrets using either the dotnet CLI or Visual Studio on Windows. For more details see Appendix A.

Required User Secrets

selfhosted: It is highly recommended that you use the selfHosted: true setting when running a local development environment. This tells the system not to use cloud services, assuming that you are running your own local SQL instance.

sqlServer__connectionString: this provides the information required for the Server to connect to the SQL instance. See the example connection string in secrets.json.example. You may need to change the default password in the connection string.

licenseDirectory: this must be set to avoid errors, but it can be set to an arbitrary empty folder.

installation__key and installation__id: request your own private Installation Id and Installation Key for self-hosting: https://bitwarden.com/host/.

Running and Debugging

After you have completed the above steps, you should be ready to launch your development environment for the Api and Identity projects.

Visual Studio

To debug:

  • On Windows, right-click on each project > click Debug > click Start New Instance
  • On MacOS, right-click each project > click Start Debugging Project

To run without debugging, open a terminal and navigate to the location of the .csproj file for that project (usually in src/ProjectName). Start the project with:

dotnet run

NOTE: check the output of the running project to find the port it is listening on. If this is different to the default in appsettings.json, you may need to update your user secrets to override this (typically the Api user secrets for the Identity URL).

Rider

From within Rider, launch both the Api project and the Identity project by clicking the "play" button for each project separately.

Testing your deployment

Troubleshooting

  • If you get a 404 error, the projects may be listening on a non-default port. Check the output of your running projects to check the port they are listening on.
  • If you get an error while restoring the nuget packages like error NU1403: Package content hash validation failed for ... then these following commands should fix the problem: 
    dotnet nuget locals all --clear
git clean -xfd
git rm **/packages.lock.json -f
dotnet restore
    For more details read NuGet/Home#7921 (comment)

Appendix A (User Secrets)

Editing user secrets - Visual Studio on Windows

Right-click on the project in the Solution Explorer and click Manage User Secrets.

Editing user secrets - Visual Studio on macOS

Open a terminal and navigate to the project directory. Once there, initiate and create the blank user secrets file by running:

dotnet user-secrets init

Add a user secret by running:

dotnet user-secrets set "<key>" "<value>"

View currently set secrets by running:

dotnet user-secrets list

By default, user secret files are located in ~/.microsoft/usersecrets/<project name>/secrets.json. After the file has been created, you can edit this directly with VSCode, which is much easier than using the command line tool.

Editing user secrets - Rider

  • Navigate to Preferences -> Plugins and Install .NET Core User Secrets
  • Right click on the a project and click Tools > Open project user secrets