IAM authentication to access the Database

[adelinn]

Summary

It would be great if Directus would become capable of authenticating to the database using a Service Account / Service Principal and IAM permissions in a cloud environment instead of requiring the use of plain password. There is currently possible to achieve that but the solution is very hacky.

Basic Example

Say I want to deploy Directus to Cloud Run. I can configure there an integration with Cloud SQL. Instead of specifying the DB_PASSWORD I would set something like DB_IAM_AUTH=gcp and Directus would configure Knex to use a Google Cloud Access Token for the password and refresh it before expiring.

Motivation

Moving away form secrets in favor of temporary tokens has become the trend now as it reduces the risk of getting the database compromised if the service is compromised and makes secret management easier.
Additionally, it's a step forward in making security easier as handling a database secret can be done wrong, while configuring token based authentication is less likely to go wrong. And why to entrust a cloud environment a secret when that cloud environment is already trusted to run Directus? Why not leverage that trust as our database "password"?

Detailed Design

I suggest that a new environment variable would be implemented DB_IAM_AUTH whose value would specify what authentication libraries to use and how to configure Knex to authenticate with a token. Possible values could be azure, gcp, aws and maybe more.

Note: The following examples have been written having Postgres in mind but they may require little adjustments to work for other databases.

A value of gcp would configure Knex this way:

import { useEnv } from '@directus/env';
import { GoogleAuth } from 'google-auth-library';

const env = useEnv();

// ...

const auth = new GoogleAuth({
  scopes: 'https://www.googleapis.com/auth/sqlservice.login',
});
const client = await auth.getClient();

knexConfig.connection = async () => {
  const token = await auth.getAccessToken();

  return {
    host : env['DB_HOST'],
    port : env['DB_PORT'],
    database : env['DB_DATABASE'],
    user : env['DB_USER'],
    password : token,
    ssl: env['DB_SSL'],
    expirationChecker: () => {
      return client.credentials.expiry_date <= Date.now() - 1000 * 60 * 5;
    }
  };
}

Note: I'm using google-auth-library instead of directly using @google-cloud/cloud-sql-connector because the Cloud SQL connector establishes its own tunnel to the DB and can't use the already provided UNIX socket that the Cloud Run - Cloud SQL integration provides, and which is more secure than connecting to the DB through the public IP.

And a value of azure this way:

import { useEnv } from '@directus/env';
import { DefaultAzureCredential } from "@azure/identity";

const env = useEnv();

// ...

const credential = new DefaultAzureCredential();

knexConfig.connection = async () => {
  const {token, expiresOnTimestamp} = await credential.getToken('https://ossrdbms-aad.database.windows.net/.default');

  return {
    host : env['DB_HOST'],
    port : env['DB_PORT'],
    database : env['DB_DATABASE'],
    user : env['DB_USER'],
    password : token,
    ssl: env['DB_SSL'],
    expirationChecker: () => {
      return expiresOnTimestamp <= Date.now() - 1000 * 60 * 5;
    }
  };
}

Requirements List

Must Have:

• Directus is able to authenticate to a Cloud SQL instance in Google Cloud using the default credentials available in that context (e.g. Cloud Run)
• Directus is able to authenticate to an Azure Database Flexible Server using the default credentials available in that context (e.g. Azure Container Apps)
• Directus is able to authenticate to an AWS database using the default credentials available in that context

Drawbacks

• Using IAM authentication may make Directus start slower as a token has to be fetched first.
• It can happen that the Cloud Provider's metadata servers fail at some point and in that case Directus will fail to start, but this can either be accepted or mitigated by some retry mechanism
• This can be implemented in user space, but in a hacky way, using a javascript config file

Alternatives

Implement this using a javascript config file.
For GCP:

// .env.js
import { useEnv } from '@directus/env';
import { GoogleAuth } from 'google-auth-library';

const auth = new GoogleAuth({
  scopes: 'https://www.googleapis.com/auth/sqlservice.login',
});

const env = {
    // ... other Directus configs
    "DB_CLIENT": "pg",
    "DB_CONNECTION_STRING": async () => { // This is set dynamically
      const env = useEnv();
      const client = await auth.getClient();
      const token = await auth.getAccessToken();
  
      return {
        host: env['DB_HOST'], // This can be the unix socket path provided by the Cloud SQL integration with Cloud Run
        port: env['DB_PORT'],
        database: env['DB_DATABASE'],
        user: env['DB_USER'],
        password: token,
        ssl: env['DB_SSL'],
        expirationChecker: () => {
          return (client.credentials.expiry_date || 0) <= Date.now() - 1000 * 60 * 5;
        }
      };
    },
    // ... other Directus configs
};

export default function(_processEnv) {
  // Workaround to prevent the value from being treated as array if the function code contains commas
  env.DB_CONNECTION_STRING.toString = () => "function";
  return env;
}

For Azure:

// .env.js
import { useEnv } from '@directus/env';
import { DefaultAzureCredential } from "@azure/identity";

const credential = new DefaultAzureCredential();

const env = {
    // ... other Directus configs
    "DB_CLIENT": "pg",
    "DB_CONNECTION_STRING": async () => {
      const env = useEnv();
      const {token, expiresOnTimestamp} = await credential.getToken('https://ossrdbms-aad.database.windows.net/.default');
  
      return {
        host : env['DB_HOST'],
        port : env['DB_PORT'],
        database : env['DB_DATABASE'],
        user : env['DB_USER'],
        password : token,
        ssl: env['DB_SSL'],
        expirationChecker: () => {
          return expiresOnTimestamp <= Date.now() - 1000 * 60 * 5;
        }
      };
    },
    // ... other Directus configs
};

export default function(_processEnv) {
  // Workaround to prevent the value from being treated as array if the function code contains commas
  env.DB_CONNECTION_STRING.toString = () => "function";
  return env;
}

Adoption Strategy

It will not be a breaking change, nor require migration. It will just be a new feature.

Unresolved Questions

Whether implementing DB_IAM_AUTH config variable is the best way to solve this.