Skip to content

Add MariaDB instructions and clarify EF testing #687

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 6 commits into from
Oct 8, 2025
+137 −61
Merged

Add MariaDB instructions and clarify EF testing #687

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
f0762a7
Add mariadb instructions and other tweaks
eliykat Oct 7, 2025
ff64d73
Tweak separate testing page
eliykat Oct 7, 2025
233180f
headings and cross references
eliykat Oct 7, 2025
da1c008
clarify
eliykat Oct 7, 2025
b1b8eda
remove refs to DatabaseTheory
eliykat Oct 7, 2025
abbc4e0
sentence case for headings
eliykat Oct 7, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
43 changes: 21 additions & 22 deletions docs/contributing/testing/database/index.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,20 +7,21 @@ pipeline.

## Creating a new test

To create a new database test, add the `[DatabaseTheory]` and `[DatabaseData]` attributes to test.
Then, use the parameters of the test to inject any repository layer services you need. The test will
run for every database that is [configured in the current environment](#configure-the-tests). Since
you inject the interface of the service, some runs of the test will use the Dapper-based repository
To create a new database test, add the `[Theory]` and `[DatabaseData]` attributes to test. Then, use
the parameters of the test to inject any repository layer services you need. The test will run for
every database that is [configured in the current environment](#configure-the-tests). Since you
inject the interface of the service, some runs of the test will use the Dapper-based repository
implementation targeting Microsoft SQL Server and others will use the Entity Framework Core based
implementations (which we use for MySql, Postgres, and SQLite).
implementations (which we use for all other supported databases).

The goal of database tests is to test the business logic that is encapsulated in a given method. For
example, if a stored procedure in SQL Server calls another procedure to update the
`User.AccountRevisionDate` then the corresponding EF implementation should do that as well. By
running the test against all variants, we are ensuring all the variants are feature-equal. to only
run the SQL Server tests along with one EF implementation; SQLite is often the easiest in that
regard. The other supported EF database providers will still run in the pipeline to catch any
differences between them.
example, if a stored procedure in SQL Server updates the `User.AccountRevisionDate` then the
corresponding EF implementation should do that as well. By running the test against all variants, we
are ensuring all the variants are feature-equal.

During development, you may choose to only run the SQL Server tests along with one EF
implementation; SQLite is often the easiest in that regard. The other supported EF database
providers will still run in the pipeline to catch any differences between them.

## Configure the tests

Expand Down Expand Up @@ -51,14 +52,11 @@ will be configured with the Entity Framework Core repositories. `Enabled` allows
disable one database but not delete the entry; it can be helpful if you are encountering a problem
with just a single database type and want to run the tests just for it instead of for all of them.

### Locally

To set the tests up locally you may want to add the configuration to your `server/dev/secrets.json`
file. You may have already done this during setup and can just run the tests with `dotnet test`. If
not, please refer to
[the getting started guide](/getting-started/server/database/ef/#testing-ef-changes).
## Locally

You can also configure the tests just like the pipeline.
To run the tests locally during development (highly recommended), see
[Getting Started - Entity Framework](../../../getting-started/server/database/ef/index.mdx). Once
configured, you can run the tests with `dotnet test` or using your IDE.

### Pipeline

Expand All @@ -82,9 +80,10 @@ across all configured database providers.

:::note

This is meant for testing data migrations only. It assumes your database schema is already fully
up-to-date. After setting up your test data, it re-runs the specified migration to verify how it
transforms the data. It will not work for schema-only migrations.
This is meant for testing data migrations only (i.e. migrations that transform data already present
in the database). It assumes your database schema is already fully up-to-date. After setting up your
test data, it re-runs the specified migration to verify how it transforms the data. It will not work
and is not required for schema-only migrations.

:::

Expand All @@ -94,7 +93,7 @@ To test a migration, set the `MigrationName` property on the `[DatabaseData]` at
`IMigrationTesterService`:

```csharp
[DatabaseTheory, DatabaseData(MigrationName = "ExampleDataMigration")]
[Theory, DatabaseData(MigrationName = "ExampleDataMigration")]
public async Task TestExampleDataMigration(
IMigrationTesterService migrationTester,
IOrganizationRepository organizationRepository)
Expand Down
155 changes: 116 additions & 39 deletions docs/getting-started/server/database/ef/index.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,10 @@ export const providers = [
label: "MySQL",
value: "mysql",
},
{
label: "MariaDB",
value: "mariadb",
},
{
label: "SQLite",
value: "sqlite",
Expand Down Expand Up @@ -80,7 +84,20 @@ sure you update the existing values instead of creating new ones
<TabItem value="mysql">

Be sure to change information like root password as needed. If you already have these secrets, make
sure you update the existing values instead of creating new ones
sure you update the existing values instead of creating new ones.

```json
"globalSettings:databaseProvider": "mysql",
"globalSettings:mySql:connectionString": "server=localhost;uid=root;pwd=example;database=vault_dev",
```

</TabItem>
<TabItem value="mariadb">

MariaDB is generally a drop-in alternative to MySQL, so it shares the `mysql` configuration keys.

Be sure to change information like root password as needed. If you already have these secrets, make
sure you update the existing values instead of creating new ones.

```json
"globalSettings:databaseProvider": "mysql",
Expand Down Expand Up @@ -109,7 +126,7 @@ that the changes take effect.

:::

### Updating the database
### Start the database server

<Tabs
groupId="provider"
Expand Down Expand Up @@ -137,7 +154,7 @@ Docker storage volume and initialize the database from scratch.

1. Confirm that `MYSQL_ROOT_PASSWORD` in `dev/.env` matches the password in `dev/secrets.json`.

2. In the `dev` folder of your server repository, run
2. In the `dev` folder of your server repository, start your database container:

```bash
docker compose --profile mysql up
Expand All @@ -151,57 +168,86 @@ Docker storage volume and initialize the database from scratch.

:::

</TabItem>
<TabItem value="mariadb">

1. Confirm that `MYSQL_ROOT_PASSWORD` in `dev/.env` matches the password in `dev/secrets.json`.

2. In the `dev` folder of your server repository, run

```bash
docker compose --profile mariadb up
```

:::tip[Confirm your database connection!]

If you run into connection errors, double check that your `.env` and `secrets.json` files have
matching passwords. If they do, you may have initialized your database incorrectly. Delete the
Docker storage volume and initialize the database from scratch.

:::

</TabItem>
<TabItem value="sqlite">

No additional step is required for SQLite. The migrator will create the database file if it doesn't
exist.

:::tip[Confirm your database path!]

The migrator creates the database file if it doesn't exist, but it does not create folders. If you
get an error that the path doesn't exist, double check that the path exists and that the folder
containing the sqlite database has write and/or create permissions.
The migrator does not create folders. If you get an error that the path doesn't exist, double check
that the path exists and that the folder containing the sqlite database has write and/or create
permissions.

:::

</TabItem>
</Tabs>

### Migrations
:::tip[Start all containers at once]

Use the `ef` profile to start all EntityFramework database containers at once:

```bash
docker compose --profile ef up
```

:::

### Run migrations

In the `dev` folder, run the following command to update the database to the latest migration:

<Tabs
groupId="provider"
values={providers}>
<TabItem value="postgres">

In the `dev` folder run the following to update the database to the latest migration

```bash
pwsh migrate.ps1 -postgres
```

The `-postgres` flag on `migrate.ps1` runs `dotnet ef` commands to perform the migrations.

</TabItem>
<TabItem value="mysql">

In the `dev` folder run the following to update the database to the latest migration

```bash
pwsh migrate.ps1 -mysql
```

The `-mysql` flag on `migrate.ps1` runs `dotnet ef` commands to perform the migrations.
</TabItem>
<TabItem value="mariadb">

```bash
pwsh migrate.ps1 -mariadb
```

</TabItem>
<TabItem value="sqlite">

In the `dev` folder run the following to update the database to the latest migration

```bash
pwsh migrate.ps1 -sqlite
```

The `-sqlite` flag on `migrate.ps1` runs `dotnet ef` commands to perform the migrations.

:::note

The migrator creates the database file if it doesn't exist, but it does not create folders. If you
Expand All @@ -212,13 +258,7 @@ get an error that the path doesn't exist, it's referring to missing folders.
</TabItem>
</Tabs>

You can also run migrations for all database providers at once using

```bash
pwsh migrate.ps1 -all
```

### Verifying changes
### Verify changes

If you would like to verify that everything worked correctly:

Expand All @@ -227,26 +267,46 @@ If you would like to verify that everything worked correctly:
- Note: this requires a configured MSSQL database. You may also need to set up other EF providers
for tests to pass.

## Testing changes
## Database integration tests

Database integration tests run for each database provider for both Dapper (MSSQL) and Entity
Framework. Developers are not expected to manually test each database provider. Instead, use
integration tests to ensure correctness across all supported databases.

Integration tests have their own connection strings, so that you can use separate databases to the
one used by your local development server. This is recommended because integration tests produce a
lot of test data over time. In the example below, this is done by using `vault_test` as the database
name.

### Configuring test databases

In your `server/dev/secrets.json` file find or add this block of secrets in the root of the json
structure:
1. In your user secrets, find or add this block of secrets in the root of the json structure
(**not** in `GlobalSettings`):

```
"databases:0:type": "Postgres",
"databases:0:connectionString": "Host=localhost;Username=postgres;Password=_________;Database=ef_test",
"databases:0:connectionString": "Host=localhost;Username=postgres;Password=_________;Database=vault_test",
"databases:0:enabled": "true",
"databases:1:type": "Sqlite",
"databases:1:enabled": "true",
"databases:1:connectionString": "Data Source=_________",
"databases:2:type": "MySql",
"databases:2:connectionString": "server=localhost;uid=root;pwd=_________;database=ef_test",
"databases:2:connectionString": "server=localhost;uid=root;pwd=_________;database=vault_test",
"databases:2:enabled": "true",
"databases:3:type": "SqlServer",
"databases:3:connectionString": "Server=localhost;Database=ef_test;User Id=SA;Password=_________;Encrypt=True;TrustServerCertificate=True;",
"databases:3:connectionString": "Server=localhost;Database=vault_test;User Id=SA;Password=_________;Encrypt=True;TrustServerCertificate=True;",
"databases:3:enabled": "true"
"databases:4:type": "MySql",
"databases:4:connectionString": "server=localhost;port=4306;uid=maria;pwd=_________;database=vault_test;AllowUserVariables=true",
"databases:4:enabled": "true",
```

:::note

The second MySql entry refers to MariaDB.

:::

:::info

The example database index + type combinations are required for the tooling to work, and to support
Expand All @@ -255,16 +315,33 @@ multiple versions of the same database running tests at the same time.
:::

This block is used for test databases for each supported provider type. These are what integration
tests will connect to. You should update the password for these connection strings to match your
existing databases if you have not already. If these settings are not present at all in your
`server/dev/secrets.json` file just add them to the bottom. These settings _do not_ go in
`globalSettings`. Then run `pwsh setup_secrets.ps1 -clear` to apply them to your local projects.
tests will connect to. Make sure that you fill in the password for each connection string.

2. Run `pwsh setup_secrets.ps1 -clear` to apply the updated user secrets to your local projects.

3. Ensure your databases are all migrated (see instructions above). You can use the `-test` flag to
only migrate the integration test databases, or `-all` to migrate everything.

```
# Migrate a specific integration test database (used by tests)
pwsh migrate.ps1 -postgres -test

# Migrate a specific development database (used by your local server)
pwsh migrate.ps1 -postgres

# Migrate all local databases
pwsh migrate.ps1 -all
```

4. Run integration tests from the `test/Infrastructure.IntegrationTest` folder using `dotnet test`.

### Writing integration tests

With connection strings applied to your projects: ensure your databases are all migrated using
`pwsh server/dev/migrate.ps1 --all`. Then you can run EF tests from the
`test/Infrastructure.IntegrationTest` folder using `dotnet test`.
See
[Contributing - Database Integration Testing](../../../../contributing/testing/database/index.md)
for more information on testing utilities and patterns.

# Modifying the database
## Modifying the database

The process for modifying the database is described in
[Migrations](./../../../../contributing/database-migrations/ef.md).