Skip to content

Commit

Permalink
Working on the templating branch
Browse files Browse the repository at this point in the history
  • Loading branch information
@kriszyp
kriszyp committed Dec 12, 2010
1 parent e48ec9d commit 0ae6751
Show file tree
Hide file tree
Showing 6 changed files with 209 additions and 74 deletions.
40 changes: 20 additions & 20 deletions README.md
View file
Expand Up @@ -103,33 +103,33 @@ each department:

aggregate(departmentId,sum(sales))
Here is a definition of the common operators (individual stores may have support
for more less operators):

Here are definitions of the common operators (individual stores may have support
for more less operators). First are the filtering and sorting operations:

* eq(<property?>,<value>+) - Filters for objects where the specified property's value is equal to the provided value or if the value is an array, equal to any of the values in the array.
* ne(<property?>,<value>+) - Filters for objects where the specified property's value is not equal to the provided value or if the value is an array, not equal to any of the values in the array.
* lt(<property?>,<value>) - Filters for objects where the specified property's value is less than the provided value
* le(<property?>,<value>) - Filters for objects where the specified property's value is less than or equal to the provided value
* gt(<property?>,<value>) - Filters for objects where the specified property's value is greater than the provided value
* ge(<property?>,<value>) - Filters for objects where the specified property's value is greater than or equal to the provided value
* and(<query>,<query>,...) - Applies all the given queries
* or(<query>,<query>,...) - The union of the given queries
* contains(<property?>,<value | expression>) - Filters for objects where the specified property's value is an array and the array contains any value that equals the provided value or satisfies the provided expression.
* excludes(<property?>,<value | expression>) - Filters for objects where the specified property's value is an array and the array does not contain any of value that equals the provided value or satisfies the provided expression.
* type(<property?>,<type>) - Filters for objects where the specified property's value is of the given type.
* rql(<property?>,<type>) - Filters for objects where the specified property's value matches the given RQL template.
* sort(<+|-><property) - Sorts by the given property in order specified by the prefix (+ for ascending, - for descending)
* recurse(<property?>) - Recursively searches, looking in children of the object as objects in arrays in the given property value

The next group of operators act on the list of the objects returned from the filtering and sorting operations:
* limit(count,start,maxTotalCount) - Returns the given range of objects from the result set, where "count" is the maximum number of objects to return, "start" is an optional starting offset, and maxTotalCount is an optional limit on the max total count that needs to be computed.
* select(<property>,<property>,...) - Trims each object down to the set of properties defined in the arguments
* values(<property>) - Returns an array of the given property value for each object
* aggregate(<property|function>,...) - Aggregates the array, grouping by objects that are distinct for the provided properties, and then reduces the remaining other property values using the provided functions
* distinct() - Returns a result set with duplicates removed
* in(<property>,<array-of-values>) - Filters for objects where the specified property's value is in the provided array
* out(<property>,<array-of-values>) - Filters for objects where the specified property's value is not in the provided array
* contains(<property>,<value | expression>) - Filters for objects where the specified property's value is an array and the array contains any value that equals the provided value or satisfies the provided expression.
* excludes(<property>,<value | expression>) - Filters for objects where the specified property's value is an array and the array does not contain any of value that equals the provided value or satisfies the provided expression.
* limit(count,start,maxCount) - Returns the given range of objects from the result set
* and(<query>,<query>,...) - Applies all the given queries
* or(<query>,<query>,...) - The union of the given queries
* eq(<property>,<value>) - Filters for objects where the specified property's value is equal to the provided value
* lt(<property>,<value>) - Filters for objects where the specified property's value is less than the provided value
* le(<property>,<value>) - Filters for objects where the specified property's value is less than or equal to the provided value
* gt(<property>,<value>) - Filters for objects where the specified property's value is greater than the provided value
* ge(<property>,<value>) - Filters for objects where the specified property's value is greater than or equal to the provided value
* ne(<property>,<value>) - Filters for objects where the specified property's value is not equal to the provided value
* rel(<relation name?>,<query>) - Applies the provided query against the linked data of the provided relation name.
* aggregate(<property|function>,...) - Aggregates the array, grouping by objects that are distinct for the provided properties, and then reduces the remaining other property values using the provided functions. When called with no arguments, returns a result set with duplicates removed
* sum(<property?>) - Finds the sum of every value in the array or if the property argument is provided, returns the sum of the value of property for every object in the array
* mean(<property?>) - Finds the mean of every value in the array or if the property argument is provided, returns the mean of the value of property for every object in the array
* max(<property?>) - Finds the maximum of every value in the array or if the property argument is provided, returns the maximum of the value of property for every object in the array
* min(<property?>) - Finds the minimum of every value in the array or if the property argument is provided, returns the minimum of the value of property for every object in the array
* recurse(<property?>) - Recursively searches, looking in children of the object as objects in arrays in the given property value
* first() - Returns the first record of the query's result set
* one() - Returns the first and only record of the query's result set, or produces an error if the query's result set has more or less than one record in it.
* count() - Returns the count of the number of records in the query's result set
Expand Down
25 changes: 16 additions & 9 deletions lib/js-array.js
View file
Expand Up @@ -46,13 +46,13 @@ exports.operators = {
});
return this;
},
match: filter(function(value, regex){
return new RegExp(regex).test(value);
match: filter(function(value, regex, flags){
return new RegExp(regex, flags).test(value);
}),
"in": filter(function(value, values){
eq: filter(function(value, values){
return values.indexOf(value) > -1;
}),
out: filter(function(value, values){
ne: filter(function(value, values){
return values.indexOf(value) == -1;
}),
contains: filter(function(array, value){
Expand Down Expand Up @@ -268,12 +268,15 @@ exports.operators = {
throw new TypeError("More than one object found");
}
return this[0];
}
},
rql: filter(function(value, template){
template.isValid(value, template);
})
};
exports.filter = filter;
function filter(condition, not){
// convert to boolean right now
var filter = function(property, second){
var filter = function(property, second, third){
if(typeof second == "undefined"){
second = property;
property = undefined;
Expand All @@ -282,7 +285,7 @@ function filter(condition, not){
var filtered = [];
for(var i = 0, length = this.length; i < length; i++){
var item = this[i];
if(condition(evaluateProperty(item, property), second)){
if(condition(evaluateProperty(item, property), second, third)){
filtered.push(item);
}
}
Expand Down Expand Up @@ -353,10 +356,10 @@ function query(query, options, target){
return '[' + value.map(queryToJS) + ']';
}else{
var jsOperator = exports.jsOperatorMap[value.name];
if(jsOperator){
var target = value.args[1];
if(jsOperator && !(target instanceof Array)){
// item['foo.bar'] ==> (item && item.foo && item.foo.bar && ...)
var path = value.args[0];
var target = value.args[1];
if (typeof target == "undefined"){
var item = "item";
target = path;
Expand Down Expand Up @@ -395,5 +398,9 @@ function throwMaxIterations(){
throw new Error("Query has taken too much computation, and the user is not allowed to execute resource-intense queries. Increase maxIterations in your config file to allow longer running non-indexed queries to be processed.");
}
exports.maxIterations = 10000;
exports.operators.and.commutative = true;
exports.operators.or.commutative = true;
exports.operators.select.commutative = true;

return exports;
});
109 changes: 66 additions & 43 deletions lib/parser.js
View file
Expand Up @@ -43,14 +43,14 @@ function parse(/*String|Object*/query, parameters){
if(exports.jsonQueryCompatible){
query = query.replace(/%3C=/g,"=le=").replace(/%3E=/g,"=ge=").replace(/%3C/g,"=lt=").replace(/%3E/g,"=gt=");
}
// convert slash delimited text to arrays
if(query.indexOf("/") > -1){ // performance guard
// convert slash delimited text to arrays
query = query.replace(/[\+\*\$\-:\w%\._]*\/[\+\*\$\-:\w%\._\/]*/g, function(slashed){
return "(" + slashed.replace(/\//g, ",") + ")";
});
}
// convert FIQL to normalized call syntax form
query = query.replace(/(\([\+\*\$\-:\w%\._,]+\)|[\+\*\$\-:\w%\._]*|)([<>!]?=(?:[\w]*=)?|>|<)(\([\+\*\$\-:\w%\._,]+\)|[\+\*\$\-:\w%\._]*|)/g,
query = query.replace(/(\([\{\}\+\*\$\-:\w%\._,]+\)|[\+\*\$\-:\w%\._]*|)([<>!]?=(?:[\w]*=)?|>|<)(\([\{\}\+\*\$\-:\w%\._,]+\)|[\{\}\+\*\$\-:\w%\._]*|)/g,
//<--------- property -----------><------ operator -----><---------------- value ------------------>
function(t, property, operator, value){
if(operator.length < 3){
Expand All @@ -67,46 +67,64 @@ function parse(/*String|Object*/query, parameters){
if(query.charAt(0)=="?"){
query = query.substring(1);
}
var leftoverCharacters = query.replace(/(\))|([&\|,])?([\+\*\$\-:\w%\._]*)(\(?)/g,
// <-closedParan->|<-delim-- propertyOrValue -----(> |
function(t, closedParan, delim, propertyOrValue, openParan){
if(delim){
if(delim === "&"){
setConjunction("and");
}
if(delim === "|"){
setConjunction("or");
}
}
if(openParan){
var newTerm = new exports.Query();
newTerm.name = propertyOrValue;
newTerm.parent = term;
call(newTerm);
}
else if(closedParan){
var isArray = !term.name;
term = term.parent;
if(!term){
throw new URIError("Closing paranthesis without an opening paranthesis");
}
if(isArray){
term.args.push(term.args.pop().args);
}
}
else if(propertyOrValue || delim === ','){
term.args.push(stringToValue(propertyOrValue, parameters));

// cache the last seen sort(), select(), values() and limit()
if (exports.lastSeen.indexOf(term.name) >= 0) {
topTerm.cache[term.name] = term.args;
}
// cache the last seen id equality
if (term.name === 'eq' && term.args[0] === exports.primaryKeyName) {
var id = term.args[1];
if (id && !(id instanceof RegExp)) id = id.toString();
topTerm.cache[exports.primaryKeyName] = id;
}
var lastName;
var leftoverCharacters = query.replace(/\)|\(|\{[\-\w%\._]+\:?|\}|[&\|,]?([\+\*\$\-:\w%\._]*)/g,
// <-close->|<-delim--propertyOrValue ----(> |<-- variable-->|close>
function(t, propertyOrValue){
switch(t.charAt(0)){
case ")": case "}":
var isArray = !term.name && !term.variable;
term = term.parent;
if(!term){
throw new URIError("Closing paranthesis without an opening paranthesis");
}
if(isArray){
term.args.push(term.args.pop().args);
}
break;
case "{":
// a variable that can be substituted
var colon = t.indexOf(":");
t = t.substring(1, colon > -1 ? colon : t.length);
var newTerm = {
variable: t,
parent: term,
args: []
};
if(parameters && parameters.hasOwnProperty(t)){
// parameter found, substitute
term.args.push(parameters[t]);
}else{
// not found, use the variable placeholder
term.args.push(newTerm);
}
term = newTerm;
break;
case "(":
var newTerm = new exports.Query();
newTerm.name = lastName === undefined ? undefined : term.args.pop();
newTerm.parent = term;
call(newTerm);
break;
case "&": case "|":
lastName = undefined;
setConjunction(t.charAt(0) == "&" ? "and" : "or");
// fall through
default:
if(propertyOrValue || t.charAt(0) === ','){
lastName = null;
term.args.push(stringToValue(propertyOrValue, parameters));
// cache the last seen sort(), select(), values() and limit()
if (exports.lastSeen.indexOf(term.name) >= 0) {
topTerm.cache[term.name] = term.args;
}
// cache the last seen id equality
if (term.name === 'eq' && term.args[0] === exports.primaryKeyName) {
var id = term.args[1];
if (id && !(id instanceof RegExp)) id = id.toString();
topTerm.cache[exports.primaryKeyName] = id;
}
}
}
return "";
});
Expand All @@ -131,7 +149,7 @@ function parse(/*String|Object*/query, parameters){
term.name = operator;
}
else if(term.name !== operator){
throw new Error("Can not mix conjunctions within a group, use paranthesis around each set of same conjuctions (& and |)");
throw new Error("Can not mix conjunctions within a group, use paranthesis around each set of same conjuctions (& and |). Was in '" + term.name + "' but operator was '" + operator + "'.");
}
}
function removeParentProperty(obj) {
Expand Down Expand Up @@ -256,6 +274,11 @@ exports.converters = {
string: function(string){
return decodeURIComponent(string);
},
rel: function(x){ // relation type
return {
rel: x
};
},
re: function(x){
return new RegExp(decodeURIComponent(x), 'i');
},
Expand Down
53 changes: 53 additions & 0 deletions lib/template.js
View file
@@ -0,0 +1,53 @@
/**
* This module provides RQL parsing. For example:
* var parsed = require("./parser").parse("b=3&le(c,5)");
*/
(function(define){
define(["./js-array", "./parser"], function(jsArray, parser){
var executeQuery = jsArray.executeQuery,
operators = jsArray.operators,
parseQuery = parser.parseQuery;
return {
isValid: function(query, template){
query = parseQuery(query);
template = parseQuery(template);
if(query && query.name){
return query.name == template.name &&
operators[name].commutative ?
setMatch(query.args, template.args) :
query.args.every(function(arg, i){ return isValid(arg, template.args[i]);});
}
if(query === template){
return true;
}
if(template && template.variable){
return executeQuery(template, {}, [value]).length;
}
},
substitute: function(template, variables){
return template.replace(/\{([^\{]*)\}/g, function(t, name){
return encodeURIComponent(variables[name]);
});
}
};
function setMatch(queryArgs, templateArgs){
templateArgs = templateArgs.concat(); // clone
var found = [];
var allMatched = queryArgs.every(function(arg){
return templateArgs.some(function(templateArg, i){
if(isValid(arg, templateArg)){
if(templateArg.multiple){
found.push(templateArg);
}else{
templateArgs.splice(i, 1);
}
return true;
}
});
});
return allMatched && !templateArgs.some(function(arg){
return !arg.optional && found.indexOf(arg) == -1;
});
}
});
})(typeof define!="undefined"?define:function(factory){module.exports = factory(require("./js-array"))});
4 changes: 2 additions & 2 deletions test/js-array.js
View file
Expand Up @@ -23,8 +23,8 @@ exports.testFiltering = function() {
assert.equal(executeQuery("price=lt=11", {}, data).length, 2);
assert.equal(executeQuery("nested/property=value", {}, data).length, 1);
assert.equal(executeQuery("with%2Fslash=slashed", {}, data).length, 1);
assert.equal(executeQuery("out(price,(5,10))", {}, data).length, 0);
assert.equal(executeQuery("out(price,(5))", {}, data).length, 1);
assert.equal(executeQuery("ne(price,(5,10))", {}, data).length, 0);
assert.equal(executeQuery("ne(price,(5))", {}, data).length, 1);
assert.equal(executeQuery("contains(tags,even)", {}, data).length, 1);
assert.equal(executeQuery("contains(tags,fun)", {}, data).length, 2);
assert.equal(executeQuery("excludes(tags,fun)", {}, data).length, 0);
Expand Down

0 comments on commit 0ae6751

Please sign in to comment.