Skip to content

Commit

Permalink
ci: move to prettier and add CI workflows
Browse files Browse the repository at this point in the history
  • Loading branch information
Qix- committed Aug 22, 2022
1 parent 44d33f1 commit ec119c2
Show file tree
Hide file tree
Showing 10 changed files with 220 additions and 236 deletions.
4 changes: 1 addition & 3 deletions .github/FUNDING.yml
Original file line number Diff line number Diff line change
@@ -1,3 +1 @@
# These are supported funding model platforms

github: "qix-"
github: 'qix-'
21 changes: 21 additions & 0 deletions .github/workflows/test.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
name: Test
on: [push, pull_request]
jobs:
test:
name: Test
runs-on: ubuntu-latest
timeout-minutes: 10
steps:
- name: Check out repository
uses: actions/checkout@v2
with:
submodules: recursive
fetch-depth: 0
- name: Install dependencies
run: npm i
- name: Lint commit message
uses: wagoid/commitlint-github-action@v4
- name: Lint source
run: npm run lint
- name: Test
run: npm run test
1 change: 1 addition & 0 deletions .npmrc
Original file line number Diff line number Diff line change
@@ -1,2 +1,3 @@
save-exact = true
package-lock = false
update-notifier = false
2 changes: 1 addition & 1 deletion LICENSE
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
Copyright (c) 2020 Josh Junon
Copyright (c) 2020-2022 Josh Junon

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
Expand Down
97 changes: 49 additions & 48 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ functions. Yes, those really exist.
```console
$ npm install --save scaly
```

## Usage

Inspired by CPU caching, applications can build out data access layers for various
Expand All @@ -18,11 +19,11 @@ and facilitate the propagation of data back up the chain in the event of misses.

That's a lot of buzzwords - here's an example problem that Scaly solves:

- You have a database where an Employee ID number is associated with their First Name.
- In the most common case, that name isn't going to be updated frequently - if ever.
- Not only is it not updated frequently, but you have to look up the First Name often, hammering your database (MongoDB, for example).
- You set up a key/value store (e.g. Redis) to cache the entries - maybe with an hour TTL by default.
- You'd also like a per-service-instance memory cache with extremely low TTLs (in the order of seconds) for many subsequent requests, too.
- You have a database where an Employee ID number is associated with their First Name.
- In the most common case, that name isn't going to be updated frequently - if ever.
- Not only is it not updated frequently, but you have to look up the First Name often, hammering your database (MongoDB, for example).
- You set up a key/value store (e.g. Redis) to cache the entries - maybe with an hour TTL by default.
- You'd also like a per-service-instance memory cache with extremely low TTLs (in the order of seconds) for many subsequent requests, too.

Your code might look something like this:

Expand Down Expand Up @@ -74,27 +75,27 @@ export async function getFirstName(eid) {

That's a lot of code. Here are some problems:

- This is **one** API call. There are a lot of branches, very unclear code,
and the repeating of e.g. the `setUsernameInMemcache()` duplicate call
can lead to bugs (especially if you inlined the `set/getUsernameInXXX()`
functions).
- The control flow is hard to follow. This is a simple getter from multiple
data stores - anything more complicated will result in even more code.
- It's not extensible. Adding a new data source requires surgical changes
to existing APIs that cannot be individually tested (easily, at least
without _extensive_ fragmentation).
- Multiply this code by 100x. That's a conservative number of datastore
operations a medium-sized application might have.
- Error handling is ambiguous - do you return an error value, or throw
an exception? How do I differentiate between user (request) error (e.g.
requesting an invalid ID) and an internal error (e.g. connection was
reset, invalid database credentials, etc.)?
- RedisLabs just went down. There's a bug in the fault-tolerant Redis
implementation and now any attempts at getting cached values fail.
Our upstream outage just turned into a downstream outage. We need to
re-deploy without any Redis implementation and fall-back to just
memcache and mongo. How do you do this when you have 100x callsites
that need to be modified?
- This is **one** API call. There are a lot of branches, very unclear code,
and the repeating of e.g. the `setUsernameInMemcache()` duplicate call
can lead to bugs (especially if you inlined the `set/getUsernameInXXX()`
functions).
- The control flow is hard to follow. This is a simple getter from multiple
data stores - anything more complicated will result in even more code.
- It's not extensible. Adding a new data source requires surgical changes
to existing APIs that cannot be individually tested (easily, at least
without _extensive_ fragmentation).
- Multiply this code by 100x. That's a conservative number of datastore
operations a medium-sized application might have.
- Error handling is ambiguous - do you return an error value, or throw
an exception? How do I differentiate between user (request) error (e.g.
requesting an invalid ID) and an internal error (e.g. connection was
reset, invalid database credentials, etc.)?
- RedisLabs just went down. There's a bug in the fault-tolerant Redis
implementation and now any attempts at getting cached values fail.
Our upstream outage just turned into a downstream outage. We need to
re-deploy without any Redis implementation and fall-back to just
memcache and mongo. How do you do this when you have 100x callsites
that need to be modified?

Along with a host of other issues.

Expand Down Expand Up @@ -157,27 +158,27 @@ try {

So, what's happening here?

- Each layer is comprised of the same (sub)set of API methods (in the example case, every layer
has a `getFirstNameByEid(eid)` method).
- Each API method must be an async generator - i.e. `async *foo` (note the `*`). This allows both
the `await` and `yield` keywords - the former useful for application developers, and the latter
required for Scaly to work.
- The layer first checks if it can resolve the request. If it can, it `return`s the result.
- If it cannot resolve the request, but wants to be notified of the eventual result (e.g. for
inserting into the cache), it can choose to use the result of a `yield` expression, which
resolves to the result from a deeper layer.
- `yield` will not return if an error is `yield`ed or `throw`n by another layer.
- The API method does NOT need to `return` in this case (the return value is ignored anyway).
This is what makes the single-line implementation for Redis's layer above work.
- If it does _not_ care about the eventual result, it can simply `return;` or `return undefined;`.
Scaly will move on to the next layer without notifying this layer of a result later on.
In the event the very last (deepest) layer `return undefined`'s, an error is thrown - **all
API methods must resolve or error at some point**.
- If the layer method wishes to raise a **recoverable or user error**, it should `yield err;` (where
`err` is anything your application needs - a string, an `Error` object, or something else).
**The function will not resume after the yield expression**, effectively acting like a `throw` statement.
- If the layer method wishes to raise an **unrecoverable/exceptional error**, it should `throw`.
This should be reserved for unrecoverable (e.g. connection lost, bad DB credentials, etc.) errors.
- Each layer is comprised of the same (sub)set of API methods (in the example case, every layer
has a `getFirstNameByEid(eid)` method).
- Each API method must be an async generator - i.e. `async *foo` (note the `*`). This allows both
the `await` and `yield` keywords - the former useful for application developers, and the latter
required for Scaly to work.
- The layer first checks if it can resolve the request. If it can, it `return`s the result.
- If it cannot resolve the request, but wants to be notified of the eventual result (e.g. for
inserting into the cache), it can choose to use the result of a `yield` expression, which
resolves to the result from a deeper layer.
- `yield` will not return if an error is `yield`ed or `throw`n by another layer.
- The API method does NOT need to `return` in this case (the return value is ignored anyway).
This is what makes the single-line implementation for Redis's layer above work.
- If it does _not_ care about the eventual result, it can simply `return;` or `return undefined;`.
Scaly will move on to the next layer without notifying this layer of a result later on.
In the event the very last (deepest) layer `return undefined`'s, an error is thrown - **all
API methods must resolve or error at some point**.
- If the layer method wishes to raise a **recoverable or user error**, it should `yield err;` (where
`err` is anything your application needs - a string, an `Error` object, or something else).
**The function will not resume after the yield expression**, effectively acting like a `throw` statement.
- If the layer method wishes to raise an **unrecoverable/exceptional error**, it should `throw`.
This should be reserved for unrecoverable (e.g. connection lost, bad DB credentials, etc.) errors.

`scaly(layer1, layer2, layerN)` returns a new object with all of the API methods between all layers.
This means that if `layer1` has a `getFoo()` API method, and `layer2` has a `getBar()` method, the
Expand All @@ -199,4 +200,4 @@ converted to a `yield err`).

# License

Copyright © 2020, Josh Junon. Released under the [MIT License](LICENSE).
Copyright © 2020-2022, Josh Junon. Released under the [MIT License](LICENSE).
38 changes: 24 additions & 14 deletions index.d.ts
Original file line number Diff line number Diff line change
@@ -1,29 +1,39 @@
type ScalyResult<ReturnValue, Error_ = string> = AsyncGenerator<Error_ | undefined, ReturnValue | void, ReturnValue>;
type ScalyResult<ReturnValue, Error_ = string> = AsyncGenerator<
Error_ | undefined,
ReturnValue | void,
ReturnValue
>;

type BoxedTupleTypes<T extends any[]> =
{[P in keyof T]: [T[P]]}[Exclude<keyof T, keyof any[]>];
type BoxedTupleTypes<T extends any[]> = { [P in keyof T]: [T[P]] }[Exclude<
keyof T,
keyof any[]
>];

type UnionToIntersection<U> =
(U extends any ? (k: U) => void : never) extends ((k: infer I) => void) ? I : never;
type UnionToIntersection<U> = (U extends any ? (k: U) => void : never) extends (
k: infer I
) => void
? I
: never;

type UnboxIntersection<T> = T extends {0: infer U} ? U : never;
type UnboxIntersection<T> = T extends { 0: infer U } ? U : never;

declare function scaly<
T,
S extends any[],
U = T & UnboxIntersection<UnionToIntersection<BoxedTupleTypes<S>>>,
U = T & UnboxIntersection<UnionToIntersection<BoxedTupleTypes<S>>>
>(
target: T,
...sources: S
): {
[P in keyof U]: U[P] extends (...args: any[]) => ScalyResult<infer ReturnValue, infer Error_>
? (...args: Parameters<U[P]>) => Promise<[true, ReturnValue] | [false, Error_]>
: never
[P in keyof U]: U[P] extends (
...args: any[]
) => ScalyResult<infer ReturnValue, infer Error_>
? (
...args: Parameters<U[P]>
) => Promise<[true, ReturnValue] | [false, Error_]>
: never;
};

export {
scaly,
ScalyResult,
};
export { scaly, ScalyResult };

export default scaly;
18 changes: 10 additions & 8 deletions index.js
Original file line number Diff line number Diff line change
Expand Up @@ -4,9 +4,9 @@ module.exports = (...layers) => {
}

// eslint-disable-next-line unicorn/no-array-reduce
const allOperations = new Set(layers.reduce(
(acc, layer) => acc.concat(Object.keys(layer))
, []));
const allOperations = new Set(
layers.reduce((acc, layer) => acc.concat(Object.keys(layer)), [])
);

const DB = {};

Expand All @@ -27,16 +27,16 @@ module.exports = (...layers) => {
const gen = layer[op](...args);

// Run until first return/yield
const {value: result, done} = await gen.next();
const { value: result, done } = await gen.next();

// Did we return?
if (done) {
if (result !== undefined) {
// Layer returned a value; propagate it up
// to all other layers who asked for it.
await Promise.all(pendingGenerators.map(
gen => gen.next(result),
));
await Promise.all(
pendingGenerators.map(gen => gen.next(result))
);

return [true, result];
}
Expand Down Expand Up @@ -71,7 +71,9 @@ module.exports = (...layers) => {
// a user cannot fix this case by changing the
// inputs, for example).
throw new Error(
`operation was not handled by any configured layers: ${op} (attempted layers: ${attemptedLayers.join(', ')})`,
`operation was not handled by any configured layers: ${op} (attempted layers: ${attemptedLayers.join(
', '
)})`
);
};
}
Expand Down
26 changes: 13 additions & 13 deletions index.test-d.ts
Original file line number Diff line number Diff line change
@@ -1,45 +1,45 @@
import {expectType} from 'tsd';
import scaly, {ScalyResult} from '.';
import { expectType } from 'tsd';
import scaly, { ScalyResult } from '.';

const db = {
maxUsers: 10,
cursor: 0,
users: new Map<number, {username: string; messages: string[]}>(),
users: new Map<number, { username: string; messages: string[] }>(),

async * registerUser(username: string): ScalyResult<number> {
async *registerUser(username: string): ScalyResult<number> {
if (this.users.size >= this.maxUsers) {
yield 'tooManyUsers';
}

const id = this.cursor++;
this.users.set(id, {username, messages: []});
this.users.set(id, { username, messages: [] });
return id;
},

async * getUsername(id: number): ScalyResult<string> {
async *getUsername(id: number): ScalyResult<string> {
const user = this.users.get(id);
return user ? user.username : yield 'noSuchUser';
},

async * getMessages(id: number): ScalyResult<string[]> {
async *getMessages(id: number): ScalyResult<string[]> {
const user = this.users.get(id);
return user ? user.messages.slice() : yield 'noSuchUser';
},

async * sendMessage(id: number, message: string): ScalyResult<null> {
async *sendMessage(id: number, message: string): ScalyResult<null> {
const user = this.users.get(id);
return user ? (user.messages.push(message), null) : yield 'noSuchUser';
},

async * checkDBStatus(): ScalyResult<null> {
async *checkDBStatus(): ScalyResult<null> {
return this.users.size < this.maxUsers ? null : yield 'tooManyUsers';
},
}
};

const cache = {
usernames: new Map<number, string>(),

async * getUsername(id: number): ScalyResult<string> {
async *getUsername(id: number): ScalyResult<string> {
const username = this.usernames.get(id);
if (username) {
return username;
Expand All @@ -48,9 +48,9 @@ const cache = {
this.usernames.set(id, yield);
},

async * checkCacheStatus(): ScalyResult<null> {
async *checkCacheStatus(): ScalyResult<null> {
return null;
},
}
};

const api = scaly(db, cache);
Expand Down
Loading

0 comments on commit ec119c2

Please sign in to comment.