Skip to content
This repository has been archived by the owner on Feb 16, 2021. It is now read-only.
/ fetch-optimizer Public archive

[No Maintenance Intended] Optimises dependent data fetchers

License

Notifications You must be signed in to change notification settings

gcanti/fetch-optimizer

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

16 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Say you have these dependencies:

example

i.e. you must fetch the f2 and f3 data before fetching the f1 data. However f2 and f3 can be fetched in parallel.

fetch-optimizer takes care of your fetchers running them in parallel when possible.

// fetchers.js

import { Poset, optimize } from 'fetch-optimizer';

// define the fetchers. A fetcher must have the following signature:
// () => Promise
const fetchers = {
  f1: () => new Promise(resolve => setTimeout(resolve('f1'), 200)),
  f2: () => new Promise(resolve => setTimeout(resolve('f2'), 50)),
  f2: () => new Promise(resolve => setTimeout(resolve('f3'), 150))
};

// define the dependencies between the fetchers
const dependencies = new Poset()
  .addEdge('f1', 'f2')  // f1 requires f2
  .addEdge('f1', 'f3'); // f1 requires also f3

optimize(dependencies, fetchers).then(() => {
    // tasks completed...
});

Run:

DEBUG=fetch-optimizer node fetchers.js

Console output:

fetch-optimizer the following fetchers will run in parallel: ["f2","f3"] with input: null +0ms
fetch-optimizer fetcher `f2` returns: "f2" +53ms
fetch-optimizer fetcher `f3` returns: "f3" +100ms
fetch-optimizer the following fetchers will run in parallel: ["f1"] with input: ["f2","f3"] +0ms
fetch-optimizer fetcher `f1` returns: "f1" +202ms

Algorithm

Say you have these dependencies:

ante

The elements a and c have no dependencies so they can run in parallel. After removing them consider the resulting poset:

rest

Iterating gets you the following execution plan:

post

About

[No Maintenance Intended] Optimises dependent data fetchers

Resources

License

Stars

Watchers

Forks

Packages

No packages published