Skip to content
Browse files

Colossal performances increase

  • Loading branch information...
1 parent dd0a998 commit 8d49b628c4f4fb47f9d7fcc43e6e4b5a1b62c9d5 @Ayms committed Mar 13, 2012
Showing with 39 additions and 9 deletions.
  1. +30 −4 README.md
  2. +9 −5 lib/bot.js
View
34 README.md
@@ -44,7 +44,13 @@ Complementary modules :
}
};
- var params='search='+$E('nike shoes')'+'&name='+$E(nike_shoes)+'&regexp='+$E('\\$|€');
+ with node-googleSearch
+
+ var params='search='+$E('nike shoes')'+'&name='+$E(nike_shoes)+'&regexp='+$E('\\$|€')+'&nbmax=20';
+
+ without node-googleSearch
+
+ var params='url=http://store.nike.com/us/en_us/%3Fl%3Dshop,pdp,ctr-inline/cid-1/pid-417121/pgid-437002&search='+$E('nike shoes')'+'&name='+$E(nike_shoes)+'&regexp='+$E('\\$|€')+'&nbmax=20';
getElements(params,response);
````
@@ -68,18 +74,30 @@ Complementary modules :
````
To call it directly :
-http://myserver:myport/getelements?name=nike_shoes&search='nikestore nike lebron9'&regexp=\$|€
+with node-googleSearch
+
+http://myserver:myport/getelements?name=nike_shoes&search='nikestore nike lebron9'&regexp=\$|€&nbmax=20
+
+without node-googleSearch
+
+http://myserver:myport/getelements?url=http://store.nike.com/us/en_us/%3Fl%3Dshop,pdp,ctr-inline/cid-1/pid-417121/pgid-437002&name=nike_shoes&search='nikestore nike lebron9'&regexp=\$|€&nbmax=20
Example with encoded parameters to retrieve the price of "lebron9" shoes on nike store :
+with node-googleSearch
+
http://213.246.53.127:1341/getelements?name=nike_shoes&search=nike%20lebron%209&regexp=%5C%24%7C%C3%A2%E2%80%9A%C2%AC
+without node-googleSearch
+
+http://213.246.53.127:1341/getelements?url=http%3A%2F%2Fstore.nike.com%2Fus%2Fen_us%2F%253Fl%253Dshop%2Cpdp%2Cctr-inline%2Fcid-1%2Fpid-417121%2Fpgid-437002&name=nike_shoes&search=nike%20lebron%209&regexp=%5C%24%7C%C3%A2%E2%80%9A%C2%AC&nbmax=20
+
To call it from a script :
````
var xscript=document.createElement('SCRIPT');
xscript.type="text/javascript";
- var params='name=nike_shoes'+'&search='+$E(nike shoes nikestore)+'&regexp='+$E('\\$|€');
+ var params='name=nike_shoes'+'&search='+$E(nike shoes nikestore)+'&regexp='+$E('\\$|€')+'&nbmax=20'; //add url parameter if you already know it (do not use node-googleSearch)
xscript.src='http://myserver:myport/getelements?'+params;
document.head.appendChild(xscript);
@@ -121,6 +139,8 @@ regexp : while building the DOM, node-dom will use that regular expression to de
search : indicates that once the gadgets have been selected with the regexp, you can filter these gadgets based on the value of search field (example : "nikestore nike shoes" url can contain other products than shoes, node-bot will return only the results matching "nike shoes").
+nbmax : important parameter for performances, the value does specify a limit for the weight of searched gadgets so node-bot does not spend a lot of time processing gadgets that are not relevant. The default value is 100, recommended value is 20.
+
## Output :
The output is an Array of :
@@ -137,7 +157,13 @@ See https://github.com/Ayms/node-gadgets/ documentation for more details.
jCore server (http://www.jcore.fr) : http://213.246.53.127:1341/getelements?params
-http://213.246.53.127:1341/getelements?name=nike_shoes&search=nike%20lebron%209&regexp=%5C%24%7C%C3%A2%E2%80%9A%C2%AC
+Example with node-googleSearch :
+
+http://213.246.53.127:1341/getelements?name=nike_shoes&search=nike%20lebron%209&regexp=%5C%24%7C%C3%A2%E2%80%9A%C2%AC&nbmax=20
+
+Example without node-googleSearch :
+
+http://213.246.53.127:1341/getelements?url=http%3A%2F%2Fstore.nike.com%2Fus%2Fen_us%2F%253Fl%253Dshop%2Cpdp%2Cctr-inline%2Fcid-1%2Fpid-417121%2Fpgid-437002&name=nike_shoes&search=nike%20lebron%209&regexp=%5C%24%7C%C3%A2%E2%80%9A%C2%AC&nbmax=20
You can use the API on jCore server : http://213.246.53.127:1341 (if by any unforeseen reasons the server is down, please advise).
View
14 lib/bot.js
@@ -34,14 +34,16 @@ var getElements = function(query,response){
};
//Example : babyliss g910e cdiscount
//becomes : babyliss g910e
+
+ var nbmax=params.nbmax?params.nbmax:'100';
if (params.gadget) {
- p +='&fetch='+$E('{"img":"", "input":""}')+'&proc='+$E('{"img":"","input":""}')+'&gadget=true&price=true&search='+$E(search)+'&regexp='+$E(params.regexp);
+ p +='&fetch='+$E('{"img":"", "input":""}')+'&proc='+$E('{"img":"","input":""}')+'&gadget=true&price=true&search='+$E(search)+'&regexp='+$E(params.regexp)+'&nbmax='+nbmax;
} else {
- p +='&fetch='+$E('{"img":"", "input":""}')+'&proc='+$E('{"img":"","input":""}')+'&price=true&search='+$E(search)+'&regexp='+$E(params.regexp);
+ p +='&fetch='+$E('{"img":"", "input":""}')+'&proc='+$E('{"img":"","input":""}')+'&price=true&search='+$E(search)+'&regexp='+$E(params.regexp)+'&nbmax='+nbmax;
};
-
- //console.log(p);
+
+ console.log(p);
var resp2={
@@ -53,6 +55,8 @@ var getElements = function(query,response){
response.writeHead(200, head);
+ console.log(params.name+'='+JSON.stringify(res)+';');
+
response.end(params.name+'='+JSON.stringify(res)+';');
/*var l=res.length;
@@ -73,7 +77,7 @@ var getElements = function(query,response){
}
}
};
-
+
if (!params.url) {
googleSearch('search='+$E(params.search),resp);
} else {

0 comments on commit 8d49b62

Please sign in to comment.
Something went wrong with that request. Please try again.