Skip to content

Commit

Permalink
Removed the caching options. All files are now cached, and the entire…
Browse files Browse the repository at this point in the history
… cache is reset for each new parse operation.
  • Loading branch information
JamesMessinger committed Mar 29, 2016
1 parent 2d5b52c commit 1f42601
Show file tree
Hide file tree
Showing 7 changed files with 50 additions and 197 deletions.
1 change: 0 additions & 1 deletion README.md
Expand Up @@ -50,7 +50,6 @@ JSON Schema $Ref Parser is a full [JSON Reference](https://tools.ietf.org/html/d

- Use **JSON** or **YAML** schemas — or even a mix of both!
- Fully supports `$ref` pointers to external files and URLs
- Configurable caching of referenced files
- Can [bundle](docs/ref-parser.md#bundlepath-options-callback) multiple files into a single schema that only has _internal_ `$ref` pointers
- Can [dereference](docs/ref-parser.md#dereferencepath-options-callback) your schema, producing a plain-old JavaScript object that's easy to work with
- Supports [circular references](docs/README.md#circular-refs), nested references, back-references, and cross-references between files
Expand Down
4 changes: 1 addition & 3 deletions docs/README.md
Expand Up @@ -23,8 +23,6 @@ Classes & Methods
- [`circular`](refs.md#circular)
- [`paths()`](refs.md#pathstypes)
- [`values()`](refs.md#valuestypes)
- [`isExpired()`](refs.md#isexpiredref)
- [`expire()`](refs.md#expireref)
- [`exists()`](refs.md#existsref)
- [`get()`](refs.md#getref-options)
- [`set()`](refs.md#setref-value-options)
Expand All @@ -50,7 +48,7 @@ var parser = new $RefParser();
parser.bundle("my-schema.json");
```

The difference is that in the second example you now have a reference to `parser`, which means you can access the results ([`parser.schema`](ref-parser.md#schema) and [`parser.$refs`](ref-parser.md#refs)) anytime you want, rather than just in the callback function. Also, having a `$RefParser` instance allows you to benefit from **[caching](options.md#cache)**, so the next time you call [`parser.resolve()`](ref-parser.md#resolveschema-options-callback), it won't need to re-download those files again (as long as the cache hasn't expired).
The difference is that in the second example you now have a reference to `parser`, which means you can access the results ([`parser.schema`](ref-parser.md#schema) and [`parser.$refs`](ref-parser.md#refs)) anytime you want, rather than just in the callback function.


### Callbacks vs. Promises
Expand Down
41 changes: 11 additions & 30 deletions docs/options.md
Expand Up @@ -23,9 +23,8 @@ $RefParser.dereference("my-schema.yaml", {
resolve: {
file: false, // Don't resolve local file references
http: {
cache: 30000, // Cache downloaded files for 30 seconds
timeout: 2000 // 2 second timeout
}
}
},
dereference: {
circular: false // Don't allow circular $refs
Expand All @@ -49,7 +48,7 @@ $RefParser.dereference("my-schema.yaml", { parse: { json: false } });
#### `order`
Parsers run in a specific order, relative to other parsers. For example, a parser with `order: 5` will run _before_ a parser with `order: 10`. If a parser is unable to successfully parse a file, then the next parser is tried, until one succeeds or they all fail.

You can change the order in which parsers run, which is useful if you know that most of your referenced files will be a certain type, or if you add your own custom parser that you want to run _first_.
You can change the order in which parsers run, which is useful if you know that most of your referenced files will be a certain type, or if you add your own custom parser that you want to run _first_.

```javascript
// Run the plain-text parser first
Expand All @@ -64,11 +63,11 @@ Multiple parsers can contain the same file extensions. For example, both the JSO
You can set your own file extensions for any parser. Each extension can be a string or a regular expression to test against the _full file path_. Here's an example:

```javascript
$RefParser.dereference("my-schema.yaml", {
parse: {
text: {
$RefParser.dereference("my-schema.yaml", {
parse: {
text: {
// parse text, html, and readme files as plain-text
ext: [".txt", ".html", /docs\/README/i]
ext: [".txt", ".html", /docs\/README/i]
}
}
});
Expand Down Expand Up @@ -103,7 +102,7 @@ Here is a simple example of a custom parser. For more complex examples refer to

// This parser only parses .foo files
myCustomParser.ext = '.foo';

// This parser runs first (before any other parsers)
myCustomParser.order = 1;

Expand All @@ -129,28 +128,13 @@ $RefParser.dereference("my-schema.yaml", { resolve: { http: false } });
#### `order`
Resolvers run in a specific order, relative to other resolvers. For example, a resolver with `order: 5` will run _before_ a resolver with `order: 10`. If a resolver is unable to successfully resolve a path, then the next resolver is tried, until one succeeds or they all fail.

You can change the order in which resolvers run, which is useful if you know that most of your file references will be a certain type, or if you add your own custom resolver that you want to run _first_.
You can change the order in which resolvers run, which is useful if you know that most of your file references will be a certain type, or if you add your own custom resolver that you want to run _first_.

```javascript
// Run the HTTP resolver first
$RefParser.dereference("my-schema.yaml", { resolve: { http: { order: 1 } } });
```

#### `cache`
JSON Schema $Ref Parser can automatically cache files, so they don't need to be re-downloaded or re-read every time. You can specify a different cache duration (in milliseconds) for each resolver, or you can disable caching for a given resolver by setting the duration to zero.

```javascript
$RefParser.dereference("my-schema.yaml", {
resolve: {
// Cache downloaded files for 30 seconds
http: { cache: 30000 },

// Don't cache local files (re-read them every time)
file: { cache: 0 }
}
});
```

#### Resolver-specific options
Resolvers can have other options that are specific to that resolver. For example, the [HTTP resolver](../lib/read/http.js) has options that allow you to customize the HTTP headers, timeout, credentials, etc.

Expand All @@ -166,13 +150,13 @@ To add your own custom resolver, just define a function that accepts the followi
Here is a simple example of a custom resolver. For more complex examples refer to any of [the built-in resolvers](../lib/read).

```javascript
// A custom resolver that reads from a MongoDB database
// A custom resolver that reads from a MongoDB database
function mongoDb(path, options, callback) {
// If it's not a MongoDB URL, then error-out, so the next resolver can be tried
if (path.substr(0, 10) !== "mongodb://") {
callback("Not a MongoDB URL");
}

mongoClient.connect(path, function(err, db) {
if (err) {
callback(err);
Expand All @@ -185,7 +169,7 @@ Here is a simple example of a custom resolver. For more complex examples refer

// This parser only parses .foo files
myCustomParser.ext = '.foo';

// This parser runs first (before any other parsers)
myCustomParser.order = 1;

Expand All @@ -203,9 +187,6 @@ Here is a simple example of a custom resolver. For more complex examples refer
|Option |Type |Default |Description
|:---------------------|:--------|:---------|:----------
|`$refs.circular` |bool or "ignore" |true |Determines whether [circular `$ref` pointers](README.md#circular-refs) are allowed. If `false`, then a `ReferenceError` will be thrown if the schema contains a circular reference.<br><br> If set to `"ignore"`, then circular references will _not_ be dereferenced, even when calling [`dereference()`](ref-parser.md#dereferenceschema-options-callback). No error will be thrown, but the [`$Refs.circular`](refs.md#circular) property will still be set to `true`.
|`cache.fs` |number |60 |<a name="caching"></a>The length of time (in seconds) to cache local files. The default is one minute. Setting to zero will cache forever.
|`cache.http` |number |300 |The length of time (in seconds) to cache HTTP URLs. The default is five minutes. Setting to zero will cache forever.
|`cache.https` |number |300 |The length of time (in seconds) to cache HTTPS URLs. The default is five minutes. Setting to zero will cache forever.
|`http.headers` |object |`{}` |HTTP to send when downloading files<br> (note: [some headers are protected](https://developer.mozilla.org/en-US/docs/Glossary/Forbidden_header_name) and cannot be set)
|`http.timeout` |number |5000 |The HTTP request timeout (in milliseconds)
|`http.redirects` |number |5 |The maximum number of HTTP redirects to follow. To disable automatic following of redirects, set this to zero.
Expand Down
43 changes: 0 additions & 43 deletions docs/refs.md
Expand Up @@ -12,8 +12,6 @@ This object is a map of JSON References and their resolved values. It also has
##### Methods
- [`paths()`](#pathstypes)
- [`values()`](#valuestypes)
- [`isExpired()`](#isexpiredref)
- [`expire()`](#expireref)
- [`exists()`](#existsref)
- [`get()`](#getref)
- [`set()`](#setref-value)
Expand Down Expand Up @@ -80,47 +78,6 @@ $RefParser.resolve("my-schema.json")
```


### `isExpired($ref)`

- **$ref** (_required_) - `string`<br>
The JSON Reference path, optionally with a JSON Pointer in the hash

- **Return Value:** `boolean`<br>
Returns `true` if the given JSON reference has expired (or if it doesn't exist); otherwise, returns `false`

```javascript
$RefParser.resolve("my-schema.json")
.then(function($refs) {
// Hasn't expired yet
$refs.isExpired("schemas/places.yaml"); // => false

// Check again after 10 minutes
setTimeout(function() {
$refs.isExpired("schemas/places.yaml"); // => true
}, 600000);
});
```


### `expire($ref)`

- **$ref** (_required_) - `string`<br>
The JSON Reference path, optionally with a JSON Pointer in the hash

Immediately expires the given JSON reference, so the next time you call a method such as [`parse`](ref-parser.md#parseschema-options-callback) or [`dereference`](ref-parser.md#dereferenceschema-options-callback), the file will be refreshed rather than reusing the cached value.

```javascript
$RefParser.resolve("my-schema.json")
.then(function($refs) {
$refs.isExpired("schemas/places.yaml"); // => false

$refs.expire("schemas/places.yaml");

$refs.isExpired("schemas/places.yaml"); // => true
});
```


### `exists($ref)`

- **$ref** (_required_) - `string`<br>
Expand Down
20 changes: 8 additions & 12 deletions lib/options.js
Expand Up @@ -11,7 +11,7 @@ var jsonParser = require('./parsers/json'),
module.exports = $RefParserOptions;

/**
* Options that determine how JSON schemas are parsed, dereferenced, and cached.
* Options that determine how JSON schemas are parsed, resolved, and dereferenced.
*
* @param {object|$RefParserOptions} [options] - Overridden options
* @constructor
Expand Down Expand Up @@ -39,12 +39,12 @@ $RefParserOptions.defaults = {
* "Empty" includes zero-byte files, as well as JSON/YAML files that
* don't contain any keys.
*/
this.parse = {
parse: {
json: jsonParser,
yaml: yamlParser,
text: textParser,
binary: binaryParser,
};
},

/**
* Determines how JSON References will be resolved.
Expand All @@ -56,13 +56,9 @@ $RefParserOptions.defaults = {
*
* order {number} - The order in which the resolver will run
*
* cache {number} - How long to cache files (in milliseconds)
* The default cache duration is different for each resolver.
* Setting the cache duration to zero disables caching for that resolver.
*
* The HTTP resolver has additional options. See read/http.js for details.
*/
this.resolve = {
resolve: {
file: fileResolver,
http: httpResolver,

Expand All @@ -74,12 +70,12 @@ $RefParserOptions.defaults = {
* @type {boolean}
*/
external: true,
};
},

/**
* Determines the types of JSON references that are allowed.
*/
this.dereference = {
dereference: {
/**
* Dereference circular (recursive) JSON references?
* If false, then a {@link ReferenceError} will be thrown if a circular reference is found.
Expand All @@ -88,11 +84,11 @@ $RefParserOptions.defaults = {
* @type {boolean|string}
*/
circular: true
};
},
};

/**
* Merges the specified options into the target object.
* Merges the properties of the source object into the target object.
*
* @param {object} target - The object that we're populating
* @param {?object} source - The options that are being merged
Expand Down
80 changes: 12 additions & 68 deletions lib/ref.js
Expand Up @@ -2,28 +2,14 @@

module.exports = $Ref;

var Pointer = require('./pointer'),
url = require('./util/url');
var Pointer = require('./pointer');

/**
* This class represents a single JSON reference and its resolved value.
*
* @param {$Refs} $refs
* @param {string} path
* @constructor
*/
function $Ref($refs, path) {
path = url.stripHash(path);

// Add this $ref to its parent collection
$refs._$refs[path] = this;

/**
* The {@link $Refs} object that contains this {@link $Ref} object.
* @type {$Refs}
*/
this.$refs = $refs;

function $Ref() {
/**
* The file path or URL of the referenced file.
* This path is relative to the path of the main JSON schema file.
Expand All @@ -34,13 +20,7 @@ function $Ref($refs, path) {
*
* @type {string}
*/
this.path = path;

/**
* Indicates the type of {@link $Ref#path} (e.g. "file", "http", etc.)
* @type {?string}
*/
this.pathType = undefined;
this.path = undefined;

/**
* The resolved value of the JSON reference.
Expand All @@ -50,53 +30,17 @@ function $Ref($refs, path) {
this.value = undefined;

/**
* The date/time that the cached value will expire.
* @type {?Date}
* The {@link $Refs} object that contains this {@link $Ref} object.
* @type {$Refs}
*/
this.expires = undefined;
}

/**
* Determines whether the {@link $Ref#value} has expired.
*
* @returns {boolean}
*/
$Ref.prototype.isExpired = function() {
// If no expiration has been set yet (i.e. `this.expires === undefined`) then it's NOT expried
return !!(this.expires && this.expires <= Date.now());
};

/**
* Immediately expires the {@link $Ref#value}.
*/
$Ref.prototype.expire = function() {
this.expires = new Date();
};
this.$refs = undefined;

/**
* Sets the {@link $Ref#expires}, based on the {@link $Ref#pathType}.
*
* @param {$RefParserOptions} options
*/
$Ref.prototype.setExpiration = function(options) {
var resolver = options.resolve[this.pathType];
if (resolver) {
var cacheDuration = resolver.cache;
if (cacheDuration > 0) {
// Extend the cache expiration
var expires = Date.now() + cacheDuration;
this.expires = new Date(expires);
}
else {
// Expire immediately
this.expires = new Date();
}
}
else {
// Never expire
this.expires = undefined;
}
};
/**
* Indicates the type of {@link $Ref#path} (e.g. "file", "http", etc.)
* @type {?string}
*/
this.pathType = undefined;
}

/**
* Determines whether the given JSON reference exists within this {@link $Ref#value}.
Expand Down

0 comments on commit 1f42601

Please sign in to comment.