Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Create one way to handle network dependencies (validator, etc.) #378

Closed
ithinkihaveacat opened this issue Jul 8, 2019 · 9 comments · Fixed by #747
Closed

Create one way to handle network dependencies (validator, etc.) #378

ithinkihaveacat opened this issue Jul 8, 2019 · 9 comments · Fixed by #747
Labels
Milestone

Comments

@ithinkihaveacat
Copy link
Collaborator

ithinkihaveacat commented Jul 8, 2019

AMP has some dependencies on a few resources hosted on cdn.ampproject.org: a document is valid iff the validator hosted at cdn.ampproject.org says it is, the definitive source of runtime version comes from this host, etc.

For various reasons, it's not always a good idea to actually fetch these resources every time a module is invoked. (e.g. if attempting to validate multiple files it's not feasible/reasonable to fetch the validator for every file that needs to be validated; running tests should be possible without network access.)

Various AMP Toolbox modules (and friends) have created different approaches to dealing with these problems, see:

We should probably come up with a single mechanism for handling these use cases.

/cc @Gregable @sebastianbenz @fstanis

@ithinkihaveacat
Copy link
Collaborator Author

Proposal: every module that requires runtime data takes an optional constructor argument runtimeData that has the signature () => Promise<string>.

  • Having an async function in the constructor makes every method async as well. I think this will okay?
  • If it were mandatory, then there would be no need for any network code in the modules themselves, which is a nice property. However, I think for ease of use it should be optional. (In which case the module will need to fetch whatever it needs to fetch if the parameter not provided.)

@fstanis
Copy link
Collaborator

fstanis commented Jul 8, 2019

optional constructor argument runtimeData that has the signature () => Promise<string>

I don't like this. It has several bad implications:

  • The constructor can fail.
  • Testing becomes hard.
  • It's a leaky abstraction.

Instead, I think we should assume the data comes as a string. It's not the module's responsibility to fetch this data, it's the caller's, so we can move all the fetch logic to a util.

@fstanis
Copy link
Collaborator

fstanis commented Jul 12, 2019

My proposal / intent to implement after discussing with @ithinkihaveacat earlier:

  • Build a utility method in @ampproject/toolbox-core that takes care of network lookups (see below).
  • Modify all constructors to expect already fetched data, either string, Buffer or parsed JSON. In other words, classes should not do any network requests.
  • Expose a factory function that performs the network request and instantiates the class with the data, the way we did in validate.ts for example.
  • Allow these factory methods to cache the instance if possible like in this file.

The utility method in @ampproject/toolbox-core should:

  • Be isomorphic (seamlessly usable from both node and browser).
  • Support both loading from URL and local files (like loadUrlOrFile).
  • Support caching out of the box.

Let me know your thoughts.

@sebastianbenz
Copy link
Collaborator

@fstanis Sounds good to me!

@sebastianbenz sebastianbenz added this to the v1.0.0 milestone Jul 12, 2019
@sebastianbenz
Copy link
Collaborator

We should get this in soon for the 1.0 release.

@sebastianbenz
Copy link
Collaborator

The only thing we should make sure of is to maintain the oneBehind behavior is a default.

@fstanis
Copy link
Collaborator

fstanis commented Jul 12, 2019

I'm OK with ensuring the modules that use it keep using it (we can add it as an option to the new method), but I wouldn't make it the default. Fetching a potentially stale (older than max age) resource doesn't seem like an intuitive thing to do. I think the default should be "fetch if older than max age".

@sebastianbenz
Copy link
Collaborator

I think the default should be "fetch if older than max age".

sgtm

@ithinkihaveacat
Copy link
Collaborator Author

@fstanis Your proposal goes further than I was thinking, but I think I like it. I don't completely follow what the code would end up looking like though--will the classes be exposed at all? Is it like this:

// Option A (class exposed to callers)
import {Validator} from "@amproject/toolbox-validator"; // Validator is a class
const validator = await init("https://cdn.ampproject.org/validator.js", Validator); // init() is generic
validator.validate("<html>...</html>");

?

Or this:

// Option B (class hidden from callers)
import {validatorInit} from "@ampproject/toolbox-validator"; // validatorInit() is a function
const validate = await validatorInit("https://cdn.ampproject.org/validator.js");
validate("<html>...</html>");

?

Supporting functions:

// @ampproject/toolbox-validator
class Validator {
    data: string;
    constructor(data: string) { // no network access!
        this.data = data;
    }
    validate(s: string) {
        return true;
    }
}

async function validatorInit(urlOrString: string) {
  const data = await fetch(urlOrString);
  return new Validator(data);
}

// @ampproject/toolbox-core
async function init<T>(urlOrString: string, klass: new(...args: any[]) => T): Promise<T> {
    const data = await fetch(urlOrString);
    return new klass(data);
}

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants