diff --git a/docs/contributing/testing/database/index.md b/docs/contributing/testing/database/index.md index 1f62c5d13..b016cc5a8 100644 --- a/docs/contributing/testing/database/index.md +++ b/docs/contributing/testing/database/index.md @@ -7,20 +7,21 @@ pipeline. ## Creating a new test -To create a new database test, add the `[DatabaseTheory]` and `[DatabaseData]` attributes to test. -Then, use the parameters of the test to inject any repository layer services you need. The test will -run for every database that is [configured in the current environment](#configure-the-tests). Since -you inject the interface of the service, some runs of the test will use the Dapper-based repository +To create a new database test, add the `[Theory]` and `[DatabaseData]` attributes to test. Then, use +the parameters of the test to inject any repository layer services you need. The test will run for +every database that is [configured in the current environment](#configure-the-tests). Since you +inject the interface of the service, some runs of the test will use the Dapper-based repository implementation targeting Microsoft SQL Server and others will use the Entity Framework Core based -implementations (which we use for MySql, Postgres, and SQLite). +implementations (which we use for all other supported databases). The goal of database tests is to test the business logic that is encapsulated in a given method. For -example, if a stored procedure in SQL Server calls another procedure to update the -`User.AccountRevisionDate` then the corresponding EF implementation should do that as well. By -running the test against all variants, we are ensuring all the variants are feature-equal. to only -run the SQL Server tests along with one EF implementation; SQLite is often the easiest in that -regard. The other supported EF database providers will still run in the pipeline to catch any -differences between them. +example, if a stored procedure in SQL Server updates the `User.AccountRevisionDate` then the +corresponding EF implementation should do that as well. By running the test against all variants, we +are ensuring all the variants are feature-equal. + +During development, you may choose to only run the SQL Server tests along with one EF +implementation; SQLite is often the easiest in that regard. The other supported EF database +providers will still run in the pipeline to catch any differences between them. ## Configure the tests @@ -51,14 +52,11 @@ will be configured with the Entity Framework Core repositories. `Enabled` allows disable one database but not delete the entry; it can be helpful if you are encountering a problem with just a single database type and want to run the tests just for it instead of for all of them. -### Locally - -To set the tests up locally you may want to add the configuration to your `server/dev/secrets.json` -file. You may have already done this during setup and can just run the tests with `dotnet test`. If -not, please refer to -[the getting started guide](/getting-started/server/database/ef/#testing-ef-changes). +## Locally -You can also configure the tests just like the pipeline. +To run the tests locally during development (highly recommended), see +[Getting Started - Entity Framework](../../../getting-started/server/database/ef/index.mdx). Once +configured, you can run the tests with `dotnet test` or using your IDE. ### Pipeline @@ -82,9 +80,10 @@ across all configured database providers. :::note -This is meant for testing data migrations only. It assumes your database schema is already fully -up-to-date. After setting up your test data, it re-runs the specified migration to verify how it -transforms the data. It will not work for schema-only migrations. +This is meant for testing data migrations only (i.e. migrations that transform data already present +in the database). It assumes your database schema is already fully up-to-date. After setting up your +test data, it re-runs the specified migration to verify how it transforms the data. It will not work +and is not required for schema-only migrations. ::: @@ -94,7 +93,7 @@ To test a migration, set the `MigrationName` property on the `[DatabaseData]` at `IMigrationTesterService`: ```csharp -[DatabaseTheory, DatabaseData(MigrationName = "ExampleDataMigration")] +[Theory, DatabaseData(MigrationName = "ExampleDataMigration")] public async Task TestExampleDataMigration( IMigrationTesterService migrationTester, IOrganizationRepository organizationRepository) diff --git a/docs/getting-started/server/database/ef/index.mdx b/docs/getting-started/server/database/ef/index.mdx index 1267411e2..c337595f1 100644 --- a/docs/getting-started/server/database/ef/index.mdx +++ b/docs/getting-started/server/database/ef/index.mdx @@ -12,6 +12,10 @@ export const providers = [ label: "MySQL", value: "mysql", }, + { + label: "MariaDB", + value: "mariadb", + }, { label: "SQLite", value: "sqlite", @@ -80,7 +84,20 @@ sure you update the existing values instead of creating new ones Be sure to change information like root password as needed. If you already have these secrets, make -sure you update the existing values instead of creating new ones +sure you update the existing values instead of creating new ones. + +```json +"globalSettings:databaseProvider": "mysql", +"globalSettings:mySql:connectionString": "server=localhost;uid=root;pwd=example;database=vault_dev", +``` + + + + +MariaDB is generally a drop-in alternative to MySQL, so it shares the `mysql` configuration keys. + +Be sure to change information like root password as needed. If you already have these secrets, make +sure you update the existing values instead of creating new ones. ```json "globalSettings:databaseProvider": "mysql", @@ -109,7 +126,7 @@ that the changes take effect. ::: -### Updating the database +### Start the database server + + +1. Confirm that `MYSQL_ROOT_PASSWORD` in `dev/.env` matches the password in `dev/secrets.json`. + +2. In the `dev` folder of your server repository, run + +```bash +docker compose --profile mariadb up +``` + +:::tip[Confirm your database connection!] + +If you run into connection errors, double check that your `.env` and `secrets.json` files have +matching passwords. If they do, you may have initialized your database incorrectly. Delete the +Docker storage volume and initialize the database from scratch. + +::: + +No additional step is required for SQLite. The migrator will create the database file if it doesn't +exist. + :::tip[Confirm your database path!] -The migrator creates the database file if it doesn't exist, but it does not create folders. If you -get an error that the path doesn't exist, double check that the path exists and that the folder -containing the sqlite database has write and/or create permissions. +The migrator does not create folders. If you get an error that the path doesn't exist, double check +that the path exists and that the folder containing the sqlite database has write and/or create +permissions. ::: -### Migrations +:::tip[Start all containers at once] + +Use the `ef` profile to start all EntityFramework database containers at once: + +```bash +docker compose --profile ef up +``` + +::: + +### Run migrations + +In the `dev` folder, run the following command to update the database to the latest migration: -In the `dev` folder run the following to update the database to the latest migration - ```bash pwsh migrate.ps1 -postgres ``` -The `-postgres` flag on `migrate.ps1` runs `dotnet ef` commands to perform the migrations. - -In the `dev` folder run the following to update the database to the latest migration - ```bash pwsh migrate.ps1 -mysql ``` -The `-mysql` flag on `migrate.ps1` runs `dotnet ef` commands to perform the migrations. + + + +```bash +pwsh migrate.ps1 -mariadb +``` -In the `dev` folder run the following to update the database to the latest migration - ```bash pwsh migrate.ps1 -sqlite ``` -The `-sqlite` flag on `migrate.ps1` runs `dotnet ef` commands to perform the migrations. - :::note The migrator creates the database file if it doesn't exist, but it does not create folders. If you @@ -212,13 +258,7 @@ get an error that the path doesn't exist, it's referring to missing folders. -You can also run migrations for all database providers at once using - -```bash -pwsh migrate.ps1 -all -``` - -### Verifying changes +### Verify changes If you would like to verify that everything worked correctly: @@ -227,26 +267,46 @@ If you would like to verify that everything worked correctly: - Note: this requires a configured MSSQL database. You may also need to set up other EF providers for tests to pass. -## Testing changes +## Database integration tests + +Database integration tests run for each database provider for both Dapper (MSSQL) and Entity +Framework. Developers are not expected to manually test each database provider. Instead, use +integration tests to ensure correctness across all supported databases. + +Integration tests have their own connection strings, so that you can use separate databases to the +one used by your local development server. This is recommended because integration tests produce a +lot of test data over time. In the example below, this is done by using `vault_test` as the database +name. + +### Configuring test databases -In your `server/dev/secrets.json` file find or add this block of secrets in the root of the json -structure: +1. In your user secrets, find or add this block of secrets in the root of the json structure + (**not** in `GlobalSettings`): ``` "databases:0:type": "Postgres", -"databases:0:connectionString": "Host=localhost;Username=postgres;Password=_________;Database=ef_test", +"databases:0:connectionString": "Host=localhost;Username=postgres;Password=_________;Database=vault_test", "databases:0:enabled": "true", "databases:1:type": "Sqlite", "databases:1:enabled": "true", "databases:1:connectionString": "Data Source=_________", "databases:2:type": "MySql", -"databases:2:connectionString": "server=localhost;uid=root;pwd=_________;database=ef_test", +"databases:2:connectionString": "server=localhost;uid=root;pwd=_________;database=vault_test", "databases:2:enabled": "true", "databases:3:type": "SqlServer", -"databases:3:connectionString": "Server=localhost;Database=ef_test;User Id=SA;Password=_________;Encrypt=True;TrustServerCertificate=True;", +"databases:3:connectionString": "Server=localhost;Database=vault_test;User Id=SA;Password=_________;Encrypt=True;TrustServerCertificate=True;", "databases:3:enabled": "true" +"databases:4:type": "MySql", +"databases:4:connectionString": "server=localhost;port=4306;uid=maria;pwd=_________;database=vault_test;AllowUserVariables=true", +"databases:4:enabled": "true", ``` +:::note + +The second MySql entry refers to MariaDB. + +::: + :::info The example database index + type combinations are required for the tooling to work, and to support @@ -255,16 +315,33 @@ multiple versions of the same database running tests at the same time. ::: This block is used for test databases for each supported provider type. These are what integration -tests will connect to. You should update the password for these connection strings to match your -existing databases if you have not already. If these settings are not present at all in your -`server/dev/secrets.json` file just add them to the bottom. These settings _do not_ go in -`globalSettings`. Then run `pwsh setup_secrets.ps1 -clear` to apply them to your local projects. +tests will connect to. Make sure that you fill in the password for each connection string. + +2. Run `pwsh setup_secrets.ps1 -clear` to apply the updated user secrets to your local projects. + +3. Ensure your databases are all migrated (see instructions above). You can use the `-test` flag to + only migrate the integration test databases, or `-all` to migrate everything. + + ``` + # Migrate a specific integration test database (used by tests) + pwsh migrate.ps1 -postgres -test + + # Migrate a specific development database (used by your local server) + pwsh migrate.ps1 -postgres + + # Migrate all local databases + pwsh migrate.ps1 -all + ``` + +4. Run integration tests from the `test/Infrastructure.IntegrationTest` folder using `dotnet test`. + +### Writing integration tests -With connection strings applied to your projects: ensure your databases are all migrated using -`pwsh server/dev/migrate.ps1 --all`. Then you can run EF tests from the -`test/Infrastructure.IntegrationTest` folder using `dotnet test`. +See +[Contributing - Database Integration Testing](../../../../contributing/testing/database/index.md) +for more information on testing utilities and patterns. -# Modifying the database +## Modifying the database The process for modifying the database is described in [Migrations](./../../../../contributing/database-migrations/ef.md).