From 2002c24a3b591d2732b7e94ea24168c12c90b321 Mon Sep 17 00:00:00 2001 From: Timm Preetz Date: Mon, 23 Jun 2025 09:42:02 +0200 Subject: [PATCH 1/3] docs: Document `serverpod_auth_email` setup and migrating from `serverpod_auth` to the new provider using `serverpod_auth_migration` Closes https://github.com/serverpod/serverpod/issues/3663 --- .../11-authentication/01-setup_new.md | 197 ++++++++++++++++++ .../06-upgrading-from-serverpod_auth.md | 118 +++++++++++ ...rading-from-serverpod_auth_successively.md | 139 ++++++++++++ 3 files changed, 454 insertions(+) create mode 100644 docs/06-concepts/11-authentication/01-setup_new.md create mode 100644 docs/08-upgrading/06-upgrading-from-serverpod_auth.md create mode 100644 docs/08-upgrading/08-upgrading-from-serverpod_auth_successively.md diff --git a/docs/06-concepts/11-authentication/01-setup_new.md b/docs/06-concepts/11-authentication/01-setup_new.md new file mode 100644 index 00000000..0bb5dca0 --- /dev/null +++ b/docs/06-concepts/11-authentication/01-setup_new.md @@ -0,0 +1,197 @@ +# Setup (`serverpod_auth_*`) + +Serverpod comes with built-in support for authentication. It is possible to build a [custom authentication implementation](custom-overrides), but the recommended way to authenticate users is to use one of the provided `serverpod_auth_*` modules. The modules make it easy to authenticate with email or social sign-ins and currently supports signing in with email, Google, Apple, and Firebase. + +Future versions of the authentication module will include more options. If you write another authentication module, please consider [contributing](/contribute) your code. + +We provide the following packages of ready-to use authentication providers. They all include a basic user profile courtesy of `serverpod_auth_profile`, and session management through `serverpod_auth_session`. + +|Package|Functionality| +|-|-| +|`serverpod_auth_email`|Ready-to-use email authentication.| +|`serverpod_auth_apple`|Ready-to-use "Sign in with Apple" authentication.| +|`serverpod_auth_google`|Ready-to-use "Sign in with Google" authentication.| + +If you would like the basic authentication offered by these packages, but combine them with a different approach to session management or another kind of user profile have [a look at the underlying packages below](#low-level-building-blocks). + +## Sessions + +When using any of the "ready-to-use" authentication providers listed above, session management based on `serverpod_auth_session` is already included. + +Just follow any of the individual authentication provider guides to set the `AuthSessions`' `authenticationHandler` on your `Serverpod` instance. + +## Email + +To get started with email based authentication, add `serverpod_auth_email` to your project. This will add a sign-up flow with email verification, and support logins and session management (through `serverpod_auth_session`). By default, this adds user profiles for each registration through `serverpod_auth_profile`. + +The only requirement for using this module is having a way to send out emails, so users can receive the initial verification email and also request a password reset later on. + + +### Server setup + +Add the module as a dependency to the server project's `pubspec.yaml`. + +```sh +$ dart pub add serverpod_auth_email_server +``` + + +As the email auth module does not expose any endpoint by default, but rather just an [`abstract` endpoint](../working-with-endpoints#endpoint-method-inheritance), a subclass of the default implementation has to be added to the current project in order to expose its APIs to outside client. + +For this add an `email_account_endpoint.dart` file to the project: + +```dart +import 'package:serverpod_auth_email_server/serverpod_auth_email_server.dart' + as email_account; + +/// Endpoint for email-based authentication. +class EmailAccountEndpoint extends email_account.EmailAccountEndpoint {} +``` + +In this `class` `@override`s could be used to augment the default endpoint implementation. + +Next, add the authentication handler to the Serverpod instance. + +```dart +import 'package:serverpod_auth_email_server/serverpod_auth_email_server.dart'; + +void run(List args) async { + var pod = Serverpod( + args, + Protocol(), + Endpoints(), + authenticationHandler: AuthSessions.authenticationHandler, // Add this line + ); + + ... +} +``` + +In order to generate server and client the code for the newly added endpoint, run: + +```bash +$ serverpod generate +``` + +Additionally, the database schema will need to be extended to add new tables for the email accounts. Create a new migration using the `create-migration` command. + +```bash +$ serverpod create-migration +``` + +As the last step, the email authentication package needs to be configured. +For this set the `EmailAccounts.config` (from package `serverpod_auth_email_account_server`), which contains the business logic used by the endpoint. This configuration should be added before the `await pod.start();` call. Callbacks need to be provided for at least `sendRegistrationVerificationCode` and `sendPasswordResetVerificationCode`, while the rest can be left to their default values. + +```dart +import 'package:serverpod_auth_email_server/serverpod_auth_email_server.dart'; + + EmailAccounts.config = EmailAccountConfig( + sendRegistrationVerificationCode: ( + final session, { + required final email, + required final accountRequestId, + required final verificationCode, + required final transaction, + }) { + // Send out actual email with the verification code + }, + sendPasswordResetVerificationCode: ( + final session, { + required final email, + required final passwordResetRequestId, + required final transaction, + required final verificationCode, + }) { + // Send out actual email with the verification code + }, +); +``` + +If you're not hosting on Serverpod Cloud, you might consider an email sendout provider like [Mailjet](https://www.mailjet.com/) or [SendGrid](https://sendgrid.com/en-us). + +It is up to the application to decide how to use the callbacks. Basically there are 2 primary approaches possible: +- Send out the `verificationCode` and require that the client initiating the request completes the operation (account creation or password reset). In that case the user could copy/paste or retype the (short) verification into a form on the client. +- Alternatively emails could contain a deep link with both the respective request ID and the verification code, which would then even support them being opened on any device (e.g. on a desktop, even if the original request was made on mobile). + +Additionally you need to update the `passwords.yaml` file to include secrets for both `serverpod_auth_session_sessionKeyHashPepper` and `serverpod_auth_email_account_passwordHashPepper`. +These should be random and at least 10 characters long. These pepper values must not be changed after the initial deployment of the server, as they are baked into every session key and stored password, and thus a rotation would invalidate previously created credentials. + +After a restart of the Serverpod the new endpoints will be usable from the client. + +### Client setup + +First, add a dependency on `serverpod_auth_session_flutter` to your app's `pubspec.yaml`, to be able to make use of the sessions generated by the newly created email endpoint. + +```yaml +dependencies: + flutter: + sdk: flutter + serverpod_flutter: ^3.0.0 + auth_example_client: # You generated client, name may differ + path: ../auth_example_client + + serverpod_auth_session_flutter: ^3.0.0 # Add this line +``` + +Next, add the `SessionManager` to your app, passing it to the generated Serverpod `Client`. + +```dart +import 'package:serverpod_auth_session_flutter/serverpod_auth_session_flutter.dart' show SessionManager; + +late SessionManager sessionManager; +late Client client; + +void main() async { + // Need to call this as we are using Flutter bindings before runApp is called. + WidgetsFlutterBinding.ensureInitialized(); + + // The session manager keeps track of the signed-in state of the user. You + // can query it to see if the user is currently signed in and get information + // about the user. + sessionManager = SessionManager(); + await sessionManager.init(); + + // The android emulator does not have access to the localhost of the machine. + // const ipAddress = '10.0.2.2'; // Android emulator ip for the host + + // On a real device replace the ipAddress with the IP address of your computer. + const ipAddress = 'localhost'; + + // Sets up a singleton client object that can be used to talk to the server from + // anywhere in our app. The client is generated from your server code. + // The client is set up to connect to a Serverpod running on a local server on + // the default port. You will need to modify this to connect to staging or + // production servers. + client = Client( + 'http://$ipAddress:8080/', + authenticationKeyManager: sessionManager, + )..connectivityMonitor = FlutterConnectivityMonitor(); + + runApp(MyApp()); +} +``` + +### User consolidation + + + + +## Low-level building blocks + +### Session Management + +|Package|Functionality| +|-|-| +|`serverpod_auth_session`|Database-backed session handling, with flexible configuration per session.| +|`serverpod_auth_jwt`|JWT-based session implemented, which can also generate access token with public/private key cryptography to use with 3rd party services.| + +### Authentication + +The following package provide the core authentication functionality, but without providing a default `Endpoint` base implementation. Thus they can be combined with another session package and include further modification, like for example a custom user profile. + +|Package|Functionality| +|-|-| +|`serverpod_auth_email_account`|Basic email authentication.| +|`serverpod_auth_apple_account`|Basic "Sign in with Apple" authentication.| +|`serverpod_auth_google_account`|Basic "Sign in with Google" authentication.| + diff --git a/docs/08-upgrading/06-upgrading-from-serverpod_auth.md b/docs/08-upgrading/06-upgrading-from-serverpod_auth.md new file mode 100644 index 00000000..d75f87ee --- /dev/null +++ b/docs/08-upgrading/06-upgrading-from-serverpod_auth.md @@ -0,0 +1,118 @@ +# Upgrading from `serverpod_auth` (in one go) + +With the release of Serverpod 3.0, the single legacy `serverpod_auth` package was replaced with a set of modular packages providing flexible modules for users, authentication providers, profiles, and session management. + +For an existing Serverpod application which makes use of `serverpod_auth`[^1] to upgrade to the new packages, there exists the packages `serverpod_auth_migration` to facilitate moving over all authentication- and user-related data into the new modules. +The package `serverpod_auth_backwards_compatibility` was created to support existing clients with legacy sessions and the migration of social logins and email passwords[^2]. + +Once the one-time migration is complete, the legacy `serverpod_auth` module and the `serverpod_auth_migration` dependencies can be removed. This will also remove the then obsolete tables from the database. +The backwards compatibility needs to kept until all (or all relevant) data has been fully migrated. + +Due to the modular approach, each used authentication provider (email, Apple, Google, etc.) needs to be configured individually for the migration. +No matter through which authentication provider(s) a user is migrated, their profile will always be migrated as well by default. + +## General timeline + +The overall timeline to migrate from `serverpod_auth` to the new modules generally looks like this: + +1. Add and configure the new auth modules for the desired providers, e.g. [email](../concepts/authentication/setup_new#email). +2. Add the migration module `serverpod_auth_migration` and configure the migration your `run` method +3. Update the `authenticationHandler` to support both legacy (migrated) and new sessions +4. Set up the `serverpod_auth_backwards_compatibility` in your email and social account endpoints + +4. Release client working against the new endpoints, make sure the legacy `serverpod_auth` ones will not be used anymore + If deploying to an app store with long lead times, prepare this well in advance, so that new updates / downloads will be able to login and register against the new APIs. +5. Start the server and run the migration to finish +6. Remove the `serverpod_auth_migration` and `serverpod_auth` module + ⚠️ This will remove the ability to login or register through the old endpoints. If this is a problem for your application, see the [guide for a continuous migration]. + +7. Deploy the server with the legacy and migration dependencies removed + For any migrated entities, drop the `int` user ID column and make the new `UUID` auth user ID required + +8. Once the backwards compatibility is not needed anymore (because all passwords have been imported and the legacy session are not in use anymore), drop the dependency on `serverpod_auth_backwards_compatibility` + +Since the actual data migration in step 5 should only take a few seconds to complete in most cases, it is recommended to already build out the "next" release beforehand, so that you can directly switch to the post-migration version of your backend. This then also ensures the final version works properly and that no further entities are created against in `serverpod_auth`, which would then not be migrated anymore. + +## Sessions + +In order to support both sessions created by the legacy `serverpod_auth` and through the new `serverpod_auth_session` package, the `authenticationHandler` has to be updated to try both of them. +Since an invalid/unknown session key just yields a `null` result, they can be chained like this: + +```dart +import 'package:serverpod_auth_migration_server/serverpod_auth_migration_server.dart' show AuthMigrations; + + +final pod = Serverpod( + args, + Protocol(), + Endpoints(), + authenticationHandler: AuthMigrations.authenticationHandler +); + + +// … + +AuthMigrations.config = AuthMigrationConfig( + userMigrationHook: ( + final session, { + required final newAuthUserId, + required final oldUserId, + final transaction, + }) async { + // Run any custom migration updating the mapping from old to new IDs here. + // ignore: avoid_print + print('Migrated account $newAuthUserId'); + }, +); + +// Start the server. +await pod.start(); + +// This is how a "one stop" migration could look like + +await AuthMigrations.migrateUsers( + await pod.createSession(), + customUserMigration: ( + final session, { + required final newAuthUserId, + required final oldUserId, + required final transaction, + }) async { + // Run any custom migration updating the mapping from old to new IDs here. + // Be sure to run the migration in the `transaction`, so a failure can be fully reverted. + print('Migrated account $newAuthUserId'); + }, +); +``` + +All high-level authentication provider packages use `serverpod_auth_session` under the hood, and only a single handler needs to be registered to support all of them. + +## User ID Migration + +The `userMigrationHook` shown above is also the place where you should migraten all references to a legacy `UserInfo` id to the new `AuthUser` `UUID`. + +A possible way to upgrade with this hook is to add an optional relation to the `AuthUser` (`authUser: module:auth_user:AuthUser?, relation(optional)`) to every entity which currently references `UserInfo` or the user ID. + +Inside the hook then find all entities for the currently migration user and set the `authUserId` for them. + +Once the migration has completed, drop the field relating to the legacy `UserInfo` and make the `AuthUser` one required (non-optional) if possible. + +Then complete the migration by deploying the updated schema, dropping the legacy columns. + +## Email passwords and social logins + +Unfortunately the migration can not migrate email passwords or social logins automatically. + +The storage of passwords changed between the modules, and thus they need to be written anew in the database. Since the plain text password is only available upon login, we have to migrate them at that point if the credentials are valid in the old system and the user does not yet have a password in the new one. + + + +Similar the legacy "user identifiers" used for social logins could not be migrated as they did not store any information which provider they were from. + +This is why you must keep a dependency on `serverpod_auth_migration` and keep the migration in the endpoints until all (relevant) accounts have been migrated. + +[^1]: Which also still works with Serverpod 3.0, if you want to continue using that. + +[^2]: As passwords are stored in a secure fashion (hashed, salted, and peppered), they can not be directly moved from the old system to the new one. Thus the migration can only happen once a client logs in again, providing the backend with the plain-text password, at which point the migrated account can be updated. \ No newline at end of file diff --git a/docs/08-upgrading/08-upgrading-from-serverpod_auth_successively.md b/docs/08-upgrading/08-upgrading-from-serverpod_auth_successively.md new file mode 100644 index 00000000..7852e145 --- /dev/null +++ b/docs/08-upgrading/08-upgrading-from-serverpod_auth_successively.md @@ -0,0 +1,139 @@ +# Upgrading from `serverpod_auth` + +With the release of Serverpod 3.0, the single legacy `serverpod_auth` package was replaced with a set of modular packages providing flexible modules for users, authentication providers, profiles, and session management. + +For an existing Serverpod application which makes use of `serverpod_auth` (which also still works with Serverpod 3.0) to upgrade to the new package, there exists `serverpod_auth_migration` to facilitate moving over all authentication- and user-related data into the new modules. +Once the migration is complete, the legacy `serverpod_auth` module and the `serverpod_auth_migration` dependencies can be removed, which will also remove the then obsolete tables from the database. + +Due to the modular approach, each used authentication provider (email, Apple, Google, etc.) needs to be configured individually for the migration. +No matter through which authentication provider(s) a user is migrated, their profile will always be migrated as well by default. + +## General timeline + +The suggested migration timeline applies to all auth providers. + +1. Add and configure the new auth modules for the desired providers, e.g. [email](../concepts/authentication/setup_new#email) +2. Add the migration module and configure each auth provider and set up a background migration job +3. Switch over all clients to use the new authentication endpoints +4. After a sufficient percentage of the userbase has migrated to the new endpoints, disable sign-ups throught the legacy package +5. Later on also disable logins and password resets on the `serverpod_auth` APIs +6. Once all users have been migrated (some of them via the background processes, albeit without passwords) remove the dependency on `serverpod_auth` and `serverpod_auth_migration` and delete all obsolete code +7. The next migration will then also remove obsolete tables + +If the Serverpod application stores data with a relation to the `int` `UserInfo` ID, this needs to be migrated to the new `UUID` id of the `AuthUser`. +To support this the `serverpod_auth_migration` packages provides a single hooks which is run for each user migration in which any related entities from your app can be migrated as well. + +## Sessions + +In order to support both sessions created by the legacy `serverpod_auth` and through the new `serverpod_auth_session` package, the `authenticationHandler` has to be updated to try both of them. +Since an invalid/unknown session key just yields a `null` result, they can be chained like this: + +```dart +import 'package:serverpod_auth_server/serverpod_auth_server.dart' as auth; +import 'package:serverpod_auth_email/serverpod_auth_email.dart' show AuthSessions; + + +final pod = Serverpod( + args, + Protocol(), + Endpoints(), + authenticationHandler: (session, key) async { + return await AuthSessions.authenticationHandler(session, key) ?? + await auth.authenticationHandler(session, key); + }, +); +``` + +All high-level authentication provider packages use `serverpod_auth_session` under the hood, and only a single handler needs to be registered to support all of them. + +## Migrating Email Authentications + +Before starting with the email account migration, the email authentication from `serverpod_auth_email` must be [set up as described](../concepts/authentication/setup_new#email). + +Since the password storage format changed between the legacy and new modules, and because we only have access to the plain text password during `login` (when it's sent from the client), there are 2 scenarios of migrating user accounts and the email authentication. + +During a login, we can verify the incoming credentials, and then migrate the entire account including the password. +In all other cases (e.g. a password reset or a background migration job) we can migrate the user profile and account, but not set its password. The password could be set on a subsequent login (if it matches the one from the legacy module), or in case the user did not log in during the migration phase, they will have to resort to a password reset. + +In order to avoid creating duplicate accounts for the "same" user in both the legacy and new system, one needs to ensure that the migration is always called before the new module would attempt any user lookups or creations. +To support this in the new endpoint, which now exists as a subclass in the Serverpod application, the migration methods need to be added to each affected endpoint. + + + +```dart +class EmailAccountEndpointWithMigration + extends email_account.EmailAccountEndpoint { + @override + Future login( + final Session session, { + required final String email, + required final String password, + }) async { + // Add this before the call to `super` in the endpoint subclass + await AuthMigrationEmail.migrateOnLogin( + session, + email: email, + password: password, + ); + + return super.login(session, email: email, password: password); + } + + @override + Future startRegistration( + final Session session, { + required final String email, + required final String password, + }) async { + // Add this before the call to `super` in the endpoint subclass + await AuthMigrationEmail.migrateOnLogin( + session, + email: email, + password: password, + ); + + return super.startRegistration(session, email: email, password: password); + } + + @override + Future startPasswordReset( + final Session session, { + required final String email, + }) async { + // Add this before the call to `super` in the endpoint subclass + await AuthMigrationEmail.migrateWithoutPassword(session, email: email); + + return super.startPasswordReset(session, email: email); + } +} +``` + +After this modification, the endpoint will always attempt a migration first, before then proceeding with the desired request (e.g. registering a new account if none exists yet). + +The migration works fully out of the box. But in case the existing `UserInfo` should not be moved to a new `serverpod_auth_profile` `UserProfile`, this can be disabled through the `AuthMigrationEmailConfig`: + +```dart +AuthMigrationEmail.config = AuthMigrationConfig( + importProfile: false, + userMigrationHook: (session, {required newAuthUserId, required oldUserId, transaction}) => …, +); +``` + +## User ID Migration + +The `userMigrationHook` shown above is also the place where you should migraten all references to a legacy `UserInfo` id to the new `AuthUser` `UUID`. + +Commonly there are 2 approach how this could be handled: + +One way would be to extend any database table that currently points to a `UserInfo` to also gain an optional `AuthUser` id column. +Then during the migration identify all rows belonging to the user and set their `AuthUser` id value. Later on the `UserInfo`-relation column will just be dropped when the migration is done. +Care needs to be taken that one does not keep writing new rows with just a `UserInfo` relation, as the migration for the previous user will not be run again, and thus this would become unreachable with the user's new ID. +A benefit of this approach is that existing queries for the user's entities (e.g. team memberships) could easily be accommodated by looking for the `int` or `UUID` value depending on the ID type of the `AuthenticationInfo`. + +Alternatively, especially if one wants to make further changes to one schema in conjunction with the user migration, one could migrate to altogether new tables in the migration, which only point to `AuthUser`s and where the relation would not need to be optional. +All future reads and writes (for migrated users) should then go to the new tables – which might not be feasible if the data is shared between both legacy and new users. But potentially one could just run a migration for all users in a team/company so all users in that group could start using the new tables. + + +## Future calls (background migration) + + \ No newline at end of file From 858012d9307b2ddf9e1c22cba1c665c1bfcc2044 Mon Sep 17 00:00:00 2001 From: Timm Preetz Date: Tue, 8 Jul 2025 17:11:49 +0200 Subject: [PATCH 2/3] Rewrite for new one-shot migration with client upgrade requirement --- .../11-authentication/01-setup_new.md | 12 +- .../06-upgrading-from-serverpod_auth.md | 263 +++++++++++++----- ...rading-from-serverpod_auth_successively.md | 139 --------- 3 files changed, 197 insertions(+), 217 deletions(-) delete mode 100644 docs/08-upgrading/08-upgrading-from-serverpod_auth_successively.md diff --git a/docs/06-concepts/11-authentication/01-setup_new.md b/docs/06-concepts/11-authentication/01-setup_new.md index 0bb5dca0..7b630ccc 100644 --- a/docs/06-concepts/11-authentication/01-setup_new.md +++ b/docs/06-concepts/11-authentication/01-setup_new.md @@ -4,7 +4,7 @@ Serverpod comes with built-in support for authentication. It is possible to buil Future versions of the authentication module will include more options. If you write another authentication module, please consider [contributing](/contribute) your code. -We provide the following packages of ready-to use authentication providers. They all include a basic user profile courtesy of `serverpod_auth_profile`, and session management through `serverpod_auth_session`. +We provide the following packages of ready-to-use authentication providers. They all include a basic user profile courtesy of `serverpod_auth_profile`, and session management through `serverpod_auth_session`. |Package|Functionality| |-|-| @@ -22,7 +22,7 @@ Just follow any of the individual authentication provider guides to set the `Aut ## Email -To get started with email based authentication, add `serverpod_auth_email` to your project. This will add a sign-up flow with email verification, and support logins and session management (through `serverpod_auth_session`). By default, this adds user profiles for each registration through `serverpod_auth_profile`. +To get started with email-based authentication, add `serverpod_auth_email` to your project. This will add a sign-up flow with email verification, and support logins and session management (through `serverpod_auth_session`). By default, this adds user profiles for each registration through `serverpod_auth_profile`. The only requirement for using this module is having a way to send out emails, so users can receive the initial verification email and also request a password reset later on. @@ -36,9 +36,9 @@ $ dart pub add serverpod_auth_email_server ``` -As the email auth module does not expose any endpoint by default, but rather just an [`abstract` endpoint](../working-with-endpoints#endpoint-method-inheritance), a subclass of the default implementation has to be added to the current project in order to expose its APIs to outside client. +As the email auth module does not expose any endpoint by default, but rather just an [`abstract` endpoint](../working-with-endpoints#endpoint-method-inheritance), a subclass of the default implementation has to be added to the current project in order to expose its APIs to clients. -For this add an `email_account_endpoint.dart` file to the project: +For this, add an `email_account_endpoint.dart` file to the project: ```dart import 'package:serverpod_auth_email_server/serverpod_auth_email_server.dart' @@ -111,9 +111,9 @@ If you're not hosting on Serverpod Cloud, you might consider an email sendout pr It is up to the application to decide how to use the callbacks. Basically there are 2 primary approaches possible: - Send out the `verificationCode` and require that the client initiating the request completes the operation (account creation or password reset). In that case the user could copy/paste or retype the (short) verification into a form on the client. -- Alternatively emails could contain a deep link with both the respective request ID and the verification code, which would then even support them being opened on any device (e.g. on a desktop, even if the original request was made on mobile). +- Alternatively, emails could contain a deep link with both the respective request ID and the verification code, which would then even support them being opened on any device (e.g. on a desktop, even if the original request was made on mobile). -Additionally you need to update the `passwords.yaml` file to include secrets for both `serverpod_auth_session_sessionKeyHashPepper` and `serverpod_auth_email_account_passwordHashPepper`. +Additionally, update the `passwords.yaml` file to include secrets for both `serverpod_auth_session_sessionKeyHashPepper` and `serverpod_auth_email_account_passwordHashPepper`. These should be random and at least 10 characters long. These pepper values must not be changed after the initial deployment of the server, as they are baked into every session key and stored password, and thus a rotation would invalidate previously created credentials. After a restart of the Serverpod the new endpoints will be usable from the client. diff --git a/docs/08-upgrading/06-upgrading-from-serverpod_auth.md b/docs/08-upgrading/06-upgrading-from-serverpod_auth.md index d75f87ee..c7f9b4ff 100644 --- a/docs/08-upgrading/06-upgrading-from-serverpod_auth.md +++ b/docs/08-upgrading/06-upgrading-from-serverpod_auth.md @@ -1,118 +1,237 @@ -# Upgrading from `serverpod_auth` (in one go) +# Upgrading from `serverpod_auth` -With the release of Serverpod 3.0, the single legacy `serverpod_auth` package was replaced with a set of modular packages providing flexible modules for users, authentication providers, profiles, and session management. +With the release of Serverpod 3.0, the “monolith” `serverpod_auth` package was deprecated and replaced with a set of modular packages providing flexible modules for user management, authentication providers, profiles, and sessions. Switching to the new authentication package enables you to make use of the updated and extended authentication methods (and upcoming ones like Passkeys and magic lines). +The new package also makes used of the recently introduced support for `UUID` primary key on all its entities. Thus in addition to migrating from the legacy package to the new ones, one also has to update all their own entities which previously referenced the `UserInfo`’s `id`. For an existing Serverpod application which makes use of `serverpod_auth`[^1] to upgrade to the new packages, there exists the packages `serverpod_auth_migration` to facilitate moving over all authentication- and user-related data into the new modules. The package `serverpod_auth_backwards_compatibility` was created to support existing clients with legacy sessions and the migration of social logins and email passwords[^2]. -Once the one-time migration is complete, the legacy `serverpod_auth` module and the `serverpod_auth_migration` dependencies can be removed. This will also remove the then obsolete tables from the database. -The backwards compatibility needs to kept until all (or all relevant) data has been fully migrated. +Once the one-time migration is complete, the legacy `serverpod_auth` module and the `serverpod_auth_migration` dependencies can be removed. This will also remove the obsolete tables from the database. +The backwards-compatibility package needs to be kept until all relevant data has been fully migrated.[^3] -Due to the modular approach, each used authentication provider (email, Apple, Google, etc.) needs to be configured individually for the migration. -No matter through which authentication provider(s) a user is migrated, their profile will always be migrated as well by default. +ℹ️ The currently provided migration helpers make the following assumptions: + +1. That the size of your database is small enough to be migrated in whatever maintenance window you can allot. +2. That clients either get updated immediately (e.g. Flutter Web) or that an update can be forced (for installed apps), in order to align with breaking changes on the API when switching from the legacy to the new endpoints. \ +Clients which update will be able to keep running on their existing session, but clients that do not update won’t work anymore. + +As there is no urgency to migrate to the new packages for existing applications, the transition should be carefully planned and tested. ## General timeline -The overall timeline to migrate from `serverpod_auth` to the new modules generally looks like this: +The overall timeline to migrate from `serverpod_auth` to the new modules is given below. 1. Add and configure the new auth modules for the desired providers, e.g. [email](../concepts/authentication/setup_new#email). -2. Add the migration module `serverpod_auth_migration` and configure the migration your `run` method -3. Update the `authenticationHandler` to support both legacy (migrated) and new sessions -4. Set up the `serverpod_auth_backwards_compatibility` in your email and social account endpoints - -4. Release client working against the new endpoints, make sure the legacy `serverpod_auth` ones will not be used anymore +2. Add the migration module `serverpod_auth_migration` and configure the migration the server’s `run` method. +3. Update the `authenticationHandler` to use the new `serverpod_auth_session` package. +4. Add the `serverpod_auth_backwards_compatibility` module and connect its helpers in the account login methods. +5. Disable the `serverpod_auth` endpoint. +6. Update the client to only use the new endpoints and the `SessionManager` from `serverpod_auth_session_flutter`. Ensure that `serverpod_auth_client` and `serverpod_auth_shared_flutter` are not used anymore. If deploying to an app store with long lead times, prepare this well in advance, so that new updates / downloads will be able to login and register against the new APIs. -5. Start the server and run the migration to finish -6. Remove the `serverpod_auth_migration` and `serverpod_auth` module - ⚠️ This will remove the ability to login or register through the old endpoints. If this is a problem for your application, see the [guide for a continuous migration]. - -7. Deploy the server with the legacy and migration dependencies removed +7. Once the updated app is available for users, deploy the backed and force the client to upgrade. +8. Remove the `serverpod_auth_migration` and `serverpod_auth` module from the server. +9. Deploy the server with the legacy and migration dependencies removed. For any migrated entities, drop the `int` user ID column and make the new `UUID` auth user ID required - -8. Once the backwards compatibility is not needed anymore (because all passwords have been imported and the legacy session are not in use anymore), drop the dependency on `serverpod_auth_backwards_compatibility` +10. Once the backwards compatibility is not needed anymore (because all passwords and sessions have been fully imported into the new module), drop the dependency on `serverpod_auth_backwards_compatibility` + +Since the actual data migration in step 5 should only take a few seconds to complete in most cases, it is recommended to already build out the "next" release beforehand, so that you can directly switch to the post-migration version of your backend. + +### Detailed Steps + +#### Add new authentication modules -Since the actual data migration in step 5 should only take a few seconds to complete in most cases, it is recommended to already build out the "next" release beforehand, so that you can directly switch to the post-migration version of your backend. This then also ensures the final version works properly and that no further entities are created against in `serverpod_auth`, which would then not be migrated anymore. +Add all desired authentication packages as described [here](../concepts/authentication/setup_new#email). The general flow is always the same: -## Sessions +- Add the dependency +- Configure the package +- Subclass the endpoint in the application’s code to get it exposed +- Make use of the new endpoint from the client -In order to support both sessions created by the legacy `serverpod_auth` and through the new `serverpod_auth_session` package, the `authenticationHandler` has to be updated to try both of them. -Since an invalid/unknown session key just yields a `null` result, they can be chained like this: +#### Set up the one-time migration + +In order to run the migration once with the next deployment of the sever, add a dependency on `serverpod_auth_migration_server` and modify the `run` method as follows: ```dart import 'package:serverpod_auth_migration_server/serverpod_auth_migration_server.dart' show AuthMigrations; +… + +void run(final List args) async { + … + // Start the server. + await pod.start(); + + // This is how a "one stop" migration could look like + await AuthMigrations.migrateUsers( + await pod.createSession(), + userMigration: ( + final session, { + required final newAuthUserId, + required final oldUserId, + final transaction, + }) async { + // Run any custom migration updating the mapping from old to new IDs here. + // Be sure to run the migration in the `transaction`, so a failure can be fully reverted. + print('Migrated account $newAuthUserId'); + }, + ); +} +``` -final pod = Serverpod( - args, - Protocol(), - Endpoints(), - authenticationHandler: AuthMigrations.authenticationHandler -); +The `userMigration` parameter shown above is also the place where you should migrate all references to a legacy `UserInfo` `int` id to the new `AuthUser` `UUID` id. +A possible way to upgrade is to add an optional relation to the `AuthUser` (`authUser: module:auth_user:AuthUser?, relation(optional)`) to every entity which currently references `UserInfo` or the user ID. (Beware that you have to configure the `serverpod_auth_user` module in your `config/generator.yaml` for the code generator to find it.) -// … +Inside the callback then find all entities for the currently migrated user and set the `authUserId` for them. -AuthMigrations.config = AuthMigrationConfig( - userMigrationHook: ( - final session, { - required final newAuthUserId, - required final oldUserId, - final transaction, - }) async { - // Run any custom migration updating the mapping from old to new IDs here. - // ignore: avoid_print - print('Migrated account $newAuthUserId'); - }, -); +Later in step 9, once the migration has completed, drop the field relating to the legacy `UserInfo` and make the `AuthUser` one required (non-optional) where possible. -// Start the server. -await pod.start(); +#### Switch to the new authentication handler -// This is how a "one stop" migration could look like +By default all new auth packages use database-backed sessions from `serverpod_auth_session`. -await AuthMigrations.migrateUsers( - await pod.createSession(), - customUserMigration: ( - final session, { - required final newAuthUserId, - required final oldUserId, - required final transaction, +Replace the `authenticationHandler` in the `Serverpod` instance with the new one like this: + +```dart +import 'package:serverpod_auth_email_server/serverpod_auth_email_server.dart' + show AuthSessions; + +void run(final List args) async { + final pod = Serverpod( + args, + Protocol(), + Endpoints(), + authenticationHandler: AuthSessions.authenticationHandler, + ); + + … +} +``` + +#### Add backwards compatibility to be able to import legacy sessions and passwords + +The migration package stores all legacy sessions and password mapped to the new user IDs in a transitional table. But since we can only fully migrate the passwords once the clients send the clear-text one upon login and migrate sessions on a “per use” basis on demand from the client, we need to add the `serverpod_auth_backwards_compatibility_server` module to the server. + +This will automatically expose a new endpoint where updated clients can exchange their legacy session for a new one backed by the `serverpod_auth_session` module. + +In order to support importing passwords set in the legacy module into the new `serverpod_auth_email_account` one you have to update your email account endpoint subclass to see whether the password can be imported like this: + +```dart +import 'package:serverpod/serverpod.dart'; +import 'package:serverpod_auth_backwards_compatibility_server/serverpod_auth_backwards_compatibility_server.dart'; +import 'package:serverpod_auth_email_server/serverpod_auth_email_server.dart' + as email_account; + +/// Endpoint for email-based authentication which imports the legacy passwords. +class PasswordImportingEmailAccountEndpoint extends email_account.EmailAccountEndpoint { + /// Logs in the user and returns a new session. + /// + /// In case an expected error occurs, this throws a `EmailAccountLoginException`. + @override + Future login( + final Session session, { + required final String email, + required final String password, + }) async { + await AuthBackwardsCompatibility.importLegacyPasswordIfNeeded( + session, + email: email, + password: password, + ); + + return super.login(session, email: email, password: password); + } + + /// Starts the registration for a new user account with an email-based login associated to it. + /// + /// Upon successful completion of this method, an email will have been + /// sent to [email] with a verification link, which the user must open to complete the registration. + @override + Future startRegistration( + final Session session, { + required final String email, + required final String password, }) async { - // Run any custom migration updating the mapping from old to new IDs here. - // Be sure to run the migration in the `transaction`, so a failure can be fully reverted. - print('Migrated account $newAuthUserId'); - }, + await AuthBackwardsCompatibility.importLegacyPasswordIfNeeded( + session, + email: email, + password: password, + ); + + return super.startRegistration(session, email: email, password: password); + } +} +``` + +This checks on every login and registration request whether the email and password existed in the legacy system, and if the account does not yet have a password set in the new module migrates the previous one over. + +#### Disable `serverpod_auth` + +The `serverpod_auth` module can not yet be removed from the server’s source code, but nonetheless we should disable its endpoint. This will make sure that for example no new registration take place once the migration is underway. + +❗️ TODO: Support config disabling the endpoint + +#### Update the client’s `SessionManager` + +The client should drop all dependencies on `serverpod_auth_client` and `serverpod_auth_shared_flutter` and instead make use of the new `SessionManager` from `serverpod_auth_session_flutter` like this: + +```dart +import 'package:serverpod_auth_session_flutter/serverpod_auth_session_flutter.dart'; +import 'package:serverpod_auth_backwards_compatibility_flutter/serverpod_auth_backwards_compatibility_flutter.dart'; + +// Ensure the one from `serverpod_auth_session_flutter` is used +final sessionManager = SessionManager(); + +final client = Client( + 'http://localhost:8080/', // leave this as it's in your app + authenticationKeyManager: sessionManager, +); + +await sessionManager.initAndImportLegacySessionIfNeeded( + client.modules.serverpod_auth_backwards_compatibility, + legacyStringGetter: null, ); ``` -All high-level authentication provider packages use `serverpod_auth_session` under the hood, and only a single handler needs to be registered to support all of them. +In case the app was using a custom `Storage` for its session manager, the `legacyStringGetter` would need to be set to point to the correct source. By default it will use `SharedPreferences`, where the legacy package stored the session key. + +The `initAndImportLegacySessionIfNeeded` checks whether the session manager does not already have a session attached. If not, then it tries to obtain the previous session key from the legacy module’s storage location. In case it receives one, it’ll exchange the legacy session key for a new session on the server and set that session on the session manager. Returning a new session from the server automatically deletes it from the database, so this can only be done once. +On subsequent launches, it will detect that a key is present and not attempt any further imports. + +#### Release the app update and deploy the server -## User ID Migration +Once (or in conjunction with, when talking about a Flutter web app) the client application is made available to consumers, deploy the backend and force the clients to update by your preferred means. -The `userMigrationHook` shown above is also the place where you should migraten all references to a legacy `UserInfo` id to the new `AuthUser` `UUID`. +Upon first start the server will now run the migration for all entities. -A possible way to upgrade with this hook is to add an optional relation to the `AuthUser` (`authUser: module:auth_user:AuthUser?, relation(optional)`) to every entity which currently references `UserInfo` or the user ID. +#### Remove the `serverpod_auth_migration` and `serverpod_auth` module from the server -Inside the hook then find all entities for the currently migration user and set the `authUserId` for them. +Now the legacy and migration module can be fully removed from the server’s codebase. -Once the migration has completed, drop the field relating to the legacy `UserInfo` and make the `AuthUser` one required (non-optional) if possible. +Furthermore the database schemas can be updated to drop the old `int` user ID columns, and instead make the new `UUID` columns required wherever the previous `int` ID was mandatory. -Then complete the migration by deploying the updated schema, dropping the legacy columns. +#### Final deployment without the legacy tables -## Email passwords and social logins +Generate the code & migrations, and deploy the server without the legacy modules. This will remove the no-longer-needed tables. -Unfortunately the migration can not migrate email passwords or social logins automatically. +The previous created client is already compatible with this backend, as no further usages of legacy code path should be included. -The storage of passwords changed between the modules, and thus they need to be written anew in the database. Since the plain text password is only available upon login, we have to migrate them at that point if the credentials are valid in the old system and the user does not yet have a password in the new one. +#### Eventual removal of `serverpod_auth_backwards_compatibility` - +As mentioned above, the legacy sessions and email authentication password get migrated upon use. Each session when the user’s client application is started on the new version, and the passwords whenever the user logs in again. -Similar the legacy "user identifiers" used for social logins could not be migrated as they did not store any information which provider they were from. +Whenever the migration of such an entity is thus completed, the respective row gets deleted from the compatibility module’s database table. This way the progress can be monitored. -This is why you must keep a dependency on `serverpod_auth_migration` and keep the migration in the endpoints until all (relevant) accounts have been migrated. +For the sessions is might be appropriate the drop all unused ones after for example 30 days, at which points clients are probably unlikely to update and the session can be deemed abandoned. + +Passwords and social login “external user identifiers” should probably be kept around longer, as the whole migration was build in a way that there was no need for the user to log in again. + +--- + +🥳 Congratulations, your server is now up to date with the latest authentication modules and best practices! [^1]: Which also still works with Serverpod 3.0, if you want to continue using that. -[^2]: As passwords are stored in a secure fashion (hashed, salted, and peppered), they can not be directly moved from the old system to the new one. Thus the migration can only happen once a client logs in again, providing the backend with the plain-text password, at which point the migrated account can be updated. \ No newline at end of file +[^2]: As passwords are stored in a secure fashion (hashed, salted, and peppered), they can not be directly moved from the legacy system to the new one. Thus the migration can only happen once a client logs in again, providing the backend with the plain-text password, at which point the migrated authentication method can be updated. + +[^3]: The remaining un-migrated data in the backwards compatibility package can be monitored by inspecting the size of the package’s tables. diff --git a/docs/08-upgrading/08-upgrading-from-serverpod_auth_successively.md b/docs/08-upgrading/08-upgrading-from-serverpod_auth_successively.md deleted file mode 100644 index 7852e145..00000000 --- a/docs/08-upgrading/08-upgrading-from-serverpod_auth_successively.md +++ /dev/null @@ -1,139 +0,0 @@ -# Upgrading from `serverpod_auth` - -With the release of Serverpod 3.0, the single legacy `serverpod_auth` package was replaced with a set of modular packages providing flexible modules for users, authentication providers, profiles, and session management. - -For an existing Serverpod application which makes use of `serverpod_auth` (which also still works with Serverpod 3.0) to upgrade to the new package, there exists `serverpod_auth_migration` to facilitate moving over all authentication- and user-related data into the new modules. -Once the migration is complete, the legacy `serverpod_auth` module and the `serverpod_auth_migration` dependencies can be removed, which will also remove the then obsolete tables from the database. - -Due to the modular approach, each used authentication provider (email, Apple, Google, etc.) needs to be configured individually for the migration. -No matter through which authentication provider(s) a user is migrated, their profile will always be migrated as well by default. - -## General timeline - -The suggested migration timeline applies to all auth providers. - -1. Add and configure the new auth modules for the desired providers, e.g. [email](../concepts/authentication/setup_new#email) -2. Add the migration module and configure each auth provider and set up a background migration job -3. Switch over all clients to use the new authentication endpoints -4. After a sufficient percentage of the userbase has migrated to the new endpoints, disable sign-ups throught the legacy package -5. Later on also disable logins and password resets on the `serverpod_auth` APIs -6. Once all users have been migrated (some of them via the background processes, albeit without passwords) remove the dependency on `serverpod_auth` and `serverpod_auth_migration` and delete all obsolete code -7. The next migration will then also remove obsolete tables - -If the Serverpod application stores data with a relation to the `int` `UserInfo` ID, this needs to be migrated to the new `UUID` id of the `AuthUser`. -To support this the `serverpod_auth_migration` packages provides a single hooks which is run for each user migration in which any related entities from your app can be migrated as well. - -## Sessions - -In order to support both sessions created by the legacy `serverpod_auth` and through the new `serverpod_auth_session` package, the `authenticationHandler` has to be updated to try both of them. -Since an invalid/unknown session key just yields a `null` result, they can be chained like this: - -```dart -import 'package:serverpod_auth_server/serverpod_auth_server.dart' as auth; -import 'package:serverpod_auth_email/serverpod_auth_email.dart' show AuthSessions; - - -final pod = Serverpod( - args, - Protocol(), - Endpoints(), - authenticationHandler: (session, key) async { - return await AuthSessions.authenticationHandler(session, key) ?? - await auth.authenticationHandler(session, key); - }, -); -``` - -All high-level authentication provider packages use `serverpod_auth_session` under the hood, and only a single handler needs to be registered to support all of them. - -## Migrating Email Authentications - -Before starting with the email account migration, the email authentication from `serverpod_auth_email` must be [set up as described](../concepts/authentication/setup_new#email). - -Since the password storage format changed between the legacy and new modules, and because we only have access to the plain text password during `login` (when it's sent from the client), there are 2 scenarios of migrating user accounts and the email authentication. - -During a login, we can verify the incoming credentials, and then migrate the entire account including the password. -In all other cases (e.g. a password reset or a background migration job) we can migrate the user profile and account, but not set its password. The password could be set on a subsequent login (if it matches the one from the legacy module), or in case the user did not log in during the migration phase, they will have to resort to a password reset. - -In order to avoid creating duplicate accounts for the "same" user in both the legacy and new system, one needs to ensure that the migration is always called before the new module would attempt any user lookups or creations. -To support this in the new endpoint, which now exists as a subclass in the Serverpod application, the migration methods need to be added to each affected endpoint. - - - -```dart -class EmailAccountEndpointWithMigration - extends email_account.EmailAccountEndpoint { - @override - Future login( - final Session session, { - required final String email, - required final String password, - }) async { - // Add this before the call to `super` in the endpoint subclass - await AuthMigrationEmail.migrateOnLogin( - session, - email: email, - password: password, - ); - - return super.login(session, email: email, password: password); - } - - @override - Future startRegistration( - final Session session, { - required final String email, - required final String password, - }) async { - // Add this before the call to `super` in the endpoint subclass - await AuthMigrationEmail.migrateOnLogin( - session, - email: email, - password: password, - ); - - return super.startRegistration(session, email: email, password: password); - } - - @override - Future startPasswordReset( - final Session session, { - required final String email, - }) async { - // Add this before the call to `super` in the endpoint subclass - await AuthMigrationEmail.migrateWithoutPassword(session, email: email); - - return super.startPasswordReset(session, email: email); - } -} -``` - -After this modification, the endpoint will always attempt a migration first, before then proceeding with the desired request (e.g. registering a new account if none exists yet). - -The migration works fully out of the box. But in case the existing `UserInfo` should not be moved to a new `serverpod_auth_profile` `UserProfile`, this can be disabled through the `AuthMigrationEmailConfig`: - -```dart -AuthMigrationEmail.config = AuthMigrationConfig( - importProfile: false, - userMigrationHook: (session, {required newAuthUserId, required oldUserId, transaction}) => …, -); -``` - -## User ID Migration - -The `userMigrationHook` shown above is also the place where you should migraten all references to a legacy `UserInfo` id to the new `AuthUser` `UUID`. - -Commonly there are 2 approach how this could be handled: - -One way would be to extend any database table that currently points to a `UserInfo` to also gain an optional `AuthUser` id column. -Then during the migration identify all rows belonging to the user and set their `AuthUser` id value. Later on the `UserInfo`-relation column will just be dropped when the migration is done. -Care needs to be taken that one does not keep writing new rows with just a `UserInfo` relation, as the migration for the previous user will not be run again, and thus this would become unreachable with the user's new ID. -A benefit of this approach is that existing queries for the user's entities (e.g. team memberships) could easily be accommodated by looking for the `int` or `UUID` value depending on the ID type of the `AuthenticationInfo`. - -Alternatively, especially if one wants to make further changes to one schema in conjunction with the user migration, one could migrate to altogether new tables in the migration, which only point to `AuthUser`s and where the relation would not need to be optional. -All future reads and writes (for migrated users) should then go to the new tables – which might not be feasible if the data is shared between both legacy and new users. But potentially one could just run a migration for all users in a team/company so all users in that group could start using the new tables. - - -## Future calls (background migration) - - \ No newline at end of file From b30458d65826247be02c3be2fa383150fedc29cd Mon Sep 17 00:00:00 2001 From: Timm Preetz Date: Wed, 9 Jul 2025 11:04:23 +0200 Subject: [PATCH 3/3] Show how to disable `serverpod_auth` endpoints In conjunction with https://github.com/serverpod/serverpod/pull/3768 --- .../06-upgrading-from-serverpod_auth.md | 18 +++++++++++++++++- 1 file changed, 17 insertions(+), 1 deletion(-) diff --git a/docs/08-upgrading/06-upgrading-from-serverpod_auth.md b/docs/08-upgrading/06-upgrading-from-serverpod_auth.md index c7f9b4ff..aafebcab 100644 --- a/docs/08-upgrading/06-upgrading-from-serverpod_auth.md +++ b/docs/08-upgrading/06-upgrading-from-serverpod_auth.md @@ -169,7 +169,23 @@ This checks on every login and registration request whether the email and passwo The `serverpod_auth` module can not yet be removed from the server’s source code, but nonetheless we should disable its endpoint. This will make sure that for example no new registration take place once the migration is underway. -❗️ TODO: Support config disabling the endpoint +Update the `AuthConfig` as follows (this likely is done in the `run` method as well: + +```dart +import 'package:serverpod_auth_server/serverpod_auth_server.dart' as auth; + +auth.AuthConfig.set( + auth.AuthConfig( + … // retain all previous configurations + disableAccountEndpoints: true + ), +); +``` + +This prevents any further logins or registrations on the legacy endpoints, so that no new user data is create while the migration is underway. + + #### Update the client’s `SessionManager`