API: Add support for OAuth2 Client Credentials and Access Tokens

[lastzero]

External applications must be able to authenticate with OAuth2 Client Credentials in order to obtain valid Access Tokens for communication with our REST API.

Further OAuth2 use cases and authentication options are beyond the scope of this issue. They may be added over time after this has been implemented.

Acceptance Criteria:

• In addition to (a) implementing a POST /api/v1/oauth/token endpoint for creating access tokens, this includes (b) adding support for standard Bearer Token authentication headers and (c) a minimum of scope-based authorization checks.
• As a first step, Prometheus should be able to query the GET /api/v1/metrics endpoint with authentication so that it won't need to be publicly accessible: API: Expose Prometheus-style metrics endpoint #3730
• Helpful implementation details and usage examples should be added to the docs, so developers understand the authentication options and know how to use the API: https://docs.photoprism.app/developer-guide/

Related Issues:

• API: Expose Prometheus-style metrics endpoint #3730
• Monitoring: Add a Prometheus-compatible API endpoint #213
• Account: Add Support for 2-Factor Authentication #808
• Account: Add Support for OpenID Connect (OIDC) #782
• Multi-User Photo Gallery with private and shared photos/albums #98

Protocol References:

• https://prometheus.io/docs/prometheus/latest/configuration/configuration/#oauth2
• https://www.scottbrady91.com/oauth/client-authentication#:~:text=OAuth%20client%20secrets
• https://www.scottbrady91.com/oauth/oauth-is-not-user-authorization
• https://www.oauth.com/oauth2-servers/access-tokens/refreshing-access-tokens/
• https://www.oauth.com/oauth2-servers/access-tokens/access-token-lifetime/
• https://www.oauth.com/oauth2-servers/access-tokens/access-token-response/
• https://www.oauth.com/oauth2-servers/client-registration/client-id-secret/
• https://learn.microsoft.com/en-us/linkedin/shared/authentication/programmatic-refresh-tokens
• https://oauth.net/2/grant-types/client-credentials/
• https://oauth.net/2/scope/
• https://datatracker.ietf.org/doc/html/rfc6749#section-4.4
• https://auth0.com/docs/get-started/authentication-and-authorization-flow/call-your-api-using-the-client-credentials-flow#example-post-to-token-url
• https://auth0.com/intro-to-iam/what-is-oauth-2
• https://auth0.com/docs/authenticate/protocols/oauth
• https://auth0.com/docs/secure/tokens/id-tokens/id-token-structure
• https://auth0.com/docs/authenticate/protocols/openid-connect-protocol
• https://www.digitalocean.com/community/tutorials/an-introduction-to-oauth-2#grant-type-client-credentials
• https://aaronparecki.com/oauth-2-simplified/
• https://rclone.org/webdav/
• https://owncloud.dev/clients/rclone/webdav-sync-oidc/
• https://www.webdavsystem.com/server/documentation/choosing_authentication/
• http://www.webdav.org/specs/rfc2617.html#rfc.section.4.1
• https://frontegg.com/blog/oauth-grant-types

Authentication Libraries:

• https://github.com/zitadel/oidc
• https://pkg.go.dev/golang.org/x/oauth2
• https://pkg.go.dev/golang.org/x/oauth2/jwt
• https://github.com/go-oauth2/oauth2
• https://github.com/pquerna/otp

Documentation Examples:

• https://docs.semui.co/administration-guide/openid
• https://api.stackexchange.com/docs/authentication
• https://dev.fitbit.com/build/reference/web-api/developer-guide/authorization/
• https://cloudentity.com/developers/basics/oauth-client-authentication/client-secret-authentication/
• https://developer.okta.com/docs/reference/api/oidc/#get-started
• https://www.authelia.com/configuration/identity-providers/open-id-connect/
• https://goauthentik.io/integrations/sources/oauth/#openid-connect
• https://developers.google.com/identity/openid-connect/openid-connect
• https://connect2id.com/products/server/docs/api
• https://connect2id.com/products/server/docs/api/discovery
• https://connect2id.com/products/server/docs/api/authorization
• https://connect2id.com/products/server/docs/api/token
• https://www.zoho.com/accounts/protocol/oauth/token-limits.html
• https://docs.github.com/en/authentication/securing-your-account-with-two-factor-authentication-2fa/configuring-two-factor-authentication-recovery-methods
• https://help.akana.com/content/current/cm/api_oauth/oauth_oauth20/m_oauth20_getTokenPOST.htm
• https://help.akana.com/content/current/cm/api_oauth/aaref/Ref_Values.htm#values_oauthgranttypes

[lastzero]

I will push my changes and provide an updated preview build for testing as soon as the new CLI commands (almost done) and the scope-based authorization checks are implemented, so probably early next week :)

[lastzero]

My last commit adds standard OAuth2 client credentials and bearer token support as well as scope-based authorization checks for REST API clients:

• Note that this first implementation should not be used in production and that the optional access token limit has not been implemented yet.
• If you would like to test these changes without building from source, you can use the photoprism/photoprism:test image available on Docker Hub: https://hub.docker.com/r/photoprism/photoprism/tags?page=1&name=test
• Use the photoprism clients CLI subcommands to list, create, update and delete clients and then test the credentials e.g. with the new GET /api/v1/metrics endpoint. The required scope for this is "metrics", but the "*" setting should also work.

[lastzero]

Here's what I think is still needed to make this releasable:

• The ability to use client authentication with a bearer token header for WebDAV access too
• Enforcement of the Access Token limit, so that clients cannot create as many tokens as they like
• Add /.well-known/ endpoints for automatic client configuration
• Helpful documentation with implementation details and usage examples
• A little more time so it can be tested in a variety of scenarios, e.g. together with 2FA

[brandon1024]

I'll happily take on documentation tasks related to the setup of prometheus :-)

[lastzero]

If you would like to test the new POST /api/v1/oauth/token endpoint, you can use the photoprism/photoprism:test image available on Docker Hub and, for example, run this in a terminal to create a new access token:

curl -Ss -X POST http://localhost:2342/api/v1/oauth/token \
-H 'Content-Type: application/x-www-form-urlencoded' \
-d 'grant_type=client_credentials&scope=metrics&client_id=ID&client_secret=SECRET'

Simply replace ID and SECRET with the actual client ID and secret created using the new photoprism clients add command and make sure the base URL is correct, i.e. you may also need to change http://localhost:2342/.

[lastzero]

With the commits referenced above, you can now create personal access tokens for authentication with our API without having to first register an OAuth2 client application, e.g. by running the following command in a terminal:

photoprism auth add --name MyApp --user admin

To see all supported command flags:

photoprism auth add --help

Note that there is no updated development preview for testing yet. So if you want to try this, you will need to build from source until we feel it has been tested enough to make it available to everyone.

[lastzero]

@Radiokot I've started working on a /.well-known/oauth-authorization-server service discovery endpoint for OAuth2-compatible API clients:

To complete this, it would be good to know which fields you actually need?

We currently don't use JWT tokens, so the related fields don't seem to be necessary and could be left blank or omitted depending on what is most compatible/easiest for API clients to handle?

Please also let us know if there are any other service discovery endpoints we should add :)

[Radiokot]

@lastzero in order to initialize the mobile OpenID connector, only the following fields are required:

• authorization_endpoint
• token_endpoint

[lastzero]

@Radiokot To authenticate with the API, you can currently create an access_token (aka "auth token") with the photoprism auth add command in a terminal (later also through the user interface). In this case, the information under /.well-known/oauth-authorization-server does not seem to be needed at all?

Alternatively, you can use the photoprism client add command to create client credentials (client_id and client_secret) to get access tokens from the token_endpoint. For this, the information about the token endpoint URL and the available scopes seems to be somewhat useful and should help to reduce manual configuration?

An authorization_endpoint does not exist yet. However, if it is helpful to e.g. connect your mobile app in a (more convenient) way, I can look into that and provide an implementation for that specific use case. Could you provide me with more specifics on this, e.g. what flow and credentials are being used/expected?

[Radiokot]

@lastzero if, as we discussed earlier, if the user will be able to create a perpetual access token for a third-party app in the web interface, then it makes sense to add auth by token as an alternative to username+password, and in this case the app has nothing to do with /.well-known.

If PhotoPrism itself is going to be an OAuth/OpenID server, then the apps will be able to get access tokens through the Authorization Code Flow, allowing the user to sign in and grant the access on a web page. In this case, .well-known/openid-configuration should return both authorization_endpoint and token_endpoint so the app could 1) render the auth page and get the auth code; 2) exchange the auth code for the access token.

I don't think, however, that you should spend time implementing the authorization endpoint, at least for the first release. For me, having the ability to create an access token in the web app and then paste it to the 3rd party app is enough.

[lastzero]

@Radiokot A simple /.well-known/oauth-authorization-server endpoint is now implemented and can be tested with the updated photoprism/photoprism:test image available on Docker Hub:

{
  "issuer": "http://localhost:2342/",
  "authorization_endpoint": "",
  "token_endpoint": "http://localhost:2342/api/v1/oauth/token",
  "registration_endpoint": "",
  "response_types_supported": [
    "token"
  ],
  "response_modes_supported": [],
  "grant_types_supported": [
    "client_credentials"
  ],
  "subject_types_supported": [],
  "scopes_supported": [
    "shares",
    "videos",
    "places",
    "feedback",
    "folders",
    "favorites",
    "albums",
    "moments",
    "logs",
    "metrics",
    "photos",
    "calendar",
    "people",
    "settings",
    "services",
    "users",
    "files",
    "labels",
    "config",
    "password",
    "webdav"
  ],
  "token_endpoint_auth_methods_supported": [
    "client_secret_basic",
    "client_secret_post"
  ],
  "claims_supported": [],
  "code_challenge_methods_supported": [],
  "introspection_endpoint": "",
  "introspection_endpoint_auth_methods_supported": [],
  "revocation_endpoint": "http://localhost:2342/api/v1/oauth/revoke",
  "revocation_endpoint_auth_methods_supported": [
    "none"
  ],
  "end_session_endpoint": "",
  "request_parameter_supported": false,
  "request_object_signing_alg_values_supported": [],
  "device_authorization_endpoint": "",
  "dpop_signing_alg_values_supported": []
}

While implementing this, I also added a POST /api/v1/oauth/revoke endpoint so that API clients can delete their sessions (access tokens) again when they are not needed anymore.

[lastzero]

An updated preview build is now available on Docker Hub for final testing:

• https://docs.photoprism.app/getting-started/updates/#development-preview
• https://docs.photoprism.app/release-notes/#development-preview

Any help with that is much appreciated! 🎉

[lastzero]

I have added the CLI command documentation to the User Guide, so that it's easy to find:

• https://docs.photoprism.app/user-guide/users/client-credentials/

Any contributions that help improve the API documentation and make it easier to use for developers would be much appreciated:

• https://docs.photoprism.app/developer-guide/api/

We also plan to work on that, but we don't know when we will get to it due to the many other things on our plate.

[brandon1024]

@lastzero The documentation was very clear! Upgraded photoprism and hooked it into my prometheus instance without a hitch!

Thanks for your amazing efforts 🙌 I'll buy you a beer next time I'm in Berlin!