s3flags

Flag system for Node.js backed by AWS S3

(TODO: Update README)

Example

import { S3Flags } from "s3flags";

async function example() {
  const flags = new S3Flags({
    location: "my-bucket/flags",
    defaultEnv: "dev",
    refreshInterval: 300000,
    optimizeInterval: 300000,
    optimizeDelay: 300000
  });

  await flags.init();

  if (flags.on("enableFeatureA")) {
    console.log(flags.get("maxNumberOfX", { default: 10 }));
  }
}

How it works

This library stores two kinds of files on AWS S3. One is called the bundle files. A bundle file stores all flags for all environments at a certain time. The other is called the update files. Whenever a update (or delete) is made, a update file is created.

Periodically, update files are bundled and a new bundle file is created. After waiting for some time, the old bundle file and update files get deleted. We call this the optimize process.

At some point your files on AWS S3 might look like this.

bundle.1519469473054.json
bundle.1519470224755.json
update.1519469652006.4r6ud75o.json
update.1519470224755.lziok5td.json
update.1519470450911.r83tlq0i.json

Each file contains a timestamp in the filename. For update files, the timestamp stands for when the update was made. For bundle files, however, the timestamp is not when the bundle was created but is when the last update was made. With this setup, it is easy for us to know which file(s) are currently effective and which are obsolete.

For update files, the filename also contains a random string because multiple users might make changes at exactly the same time.

When this library initializes, it requests for the list of files on AWS S3. It then loads the contents of the latest bundle file and all update files that have timestamp larger than that of the bundle. Loaded files and merge result of the bundle and updates are cached.

For the example above, the contents of bundle.1519470224755.json and update.1519470450911.r83tlq0i.json are loaded. Other files are ignored.

Later on, when the client tries to get flags, it loads from the cached result.

Periodically, the cached result is refreshed. The refresh process is similar to the initialization process, except that cached files are not loaded again from AWS S3.

Why so complicated? Because this library is designed to be used by multi-server systems and tries to minimize the posibilities of getting problems due to the uncertain delay of the Eventual Consistency model of AWS S3.

API

class S3Flags

constructor(config: Config)

Instantiate S3Flags with configurations. See interface Config for more details.

const flags = new S3Flags({
  location: "my-bucket/flags",
  defaultEnv: "dev",
  refreshInterval: 300000,
  optimizeInterval: 300000,
  optimizeDelay: 300000
});

init(): Promise<void>

Initialize by loading flag files from S3 and setup refresh and optimize timers. This must be called and waited before you can start getting flags.

await flags.init();

get<T>(flag: string, options: { default: T, env?: string }): T

Get value of a flag, for the default or given environment. If the flag does not exist or the default / given environment of this flag is not configured, the default value provided in the options is returned.

const enableFeatureA = flags.get("enableFeatureA", { default: false });
const maxNumberOfX = flags.get("maxNumberOfX", { default: 10 });

get<T>(flag: string, options: { info: true, env?: string }): FlagInfo<T> | null

Get full information of a flag, for the default or given environment. See interface FlagInfo for more details on what is returned. If the flag does not exist or the default / given environment of this flag is not configured, null is returned.

const flagInfo = flags.get("enableFeatureA", { info: true });
if (flagInfo) {
  console.log(flagInfo.updatedBy);
}

on(name: string, options?: { env?: string }): boolean

An alias for getting the value of a boolean flag with default of false.

Note: If the flag exists but is not of type boolean, false is returned no matter what the actual value is.

if (flags.on("enableFeatureA")) {
  // feature A code here
}

all(): FlagCollection

Export all flags for all environments with full information. See interface FlagCollection for more details.

app.get("/flags", function (req, res) {
  res.send(flags.all());
});

update(flag: string, value: string | number | boolean, user: string): Promise<void>

Update value of the given flag for the default environment. If the flag does not exist or it is not configured for the default environment, it gets created / configured.