-
Notifications
You must be signed in to change notification settings - Fork 144
Field monitoring #2
Comments
Correct. It assumes we are monitoring records of data that will be added, deleted, and changed over time. Can you explain the use case for monitoring a single field? Can't you just field.match(//) against it if your needs are this simple? |
My table has a number of fields, I only want to search against specific fields. I have limited experience with elastic search so pardon my ignorance if I have this back-to-front. |
Specify the fields you want to search in config.js using the fields array |
Perhaps my original explanation was a little rushed, but this is what I had done and it doesn't seem to work. With config.js specifying |
That could be a bug. Would it be possible to share your config.js, data structure, and client code? |
I think Config is the only file I have changed from the cloned code. I am really only just testing this out. Config.js /**
* This config file is provided as a convenience for development. You can either
* set the environment variables on your server or modify the values here.
*
* At a minimum, you must set FB_URL and Paths to Monitor. Everything else is optional, assuming your
* ElasticSearch server is at localhost:9200.
*/
/** Firebase Settings
***************************************************/
// Your Firebase instance where we will listen and write search results
exports.FB_URL = 'https://economatic.firebaseio.com/';
// Either your Firebase secret or a token you create with no expiry, used to authenticate
// To Firebase and access search data.
exports.FB_TOKEN = process.env.FB_TOKEN || null;
// The path in your Firebase where clients will write search requests
exports.FB_REQ = process.env.FB_REQ || 'search/request';
// The path in your Firebase where this app will write the results
exports.FB_RES = process.env.FB_RES || 'search/response';
/** ElasticSearch Settings
*********************************************/
if (process.env.BONSAI_URL) {
processBonsaiUrl(exports, process.env.BONSAI_URL);
} else {
// ElasticSearch server's host URL
exports.ES_HOST = process.env.ES_HOST || 'localhost';
// ElasticSearch server's host port
exports.ES_PORT = process.env.ES_PORT || '9200';
// ElasticSearch username for http auth
exports.ES_USER = process.env.ES_USER || null;
// ElasticSearch password for http auth
exports.ES_PASS = process.env.ES_PASS || null;
}
/** Paths to Monitor
*
* Each path can have these keys:
* {string} path: [required] the Firebase path to be monitored, for example, `users/profiles`
* would monitor https://<instance>.firebaseio.com/users/profiles
* {string} index: [required] the name of the ES index to write data into
* {string} type: [required] name of the ES object type this document will be stored as
* {Array} fields: list of fields to be monitored (defaults to all fields)
* {Function} filter: if provided, only records that return true are indexed
* {Function} parser: if provided, the results of this function are passed to ES, rather than the raw data (fields is ignored if this is used)
****************************************************/
exports.paths = [{
path: "cars",
index: "firebase",
type: "car",
fields: ['MANUFACTURER', 'MODEL', 'YEAR']
}];
/** Config Options
***************************************************/
// How often should the script remove unclaimed search results? probably just leave this alone
exports.CLEANUP_INTERVAL =
process.env.NODE_ENV === 'production' ?
3600 * 1000 /* once an hour */ :
60 * 1000 /* once a minute */ ;
function processBonsaiUrl(exports, url) {
var matches = url.match(/^https?:\/\/([^:]+):([^@]+)@([^/]+)\/?$/);
exports.ES_HOST = matches[3];
exports.ES_PORT = 80;
exports.ES_USER = matches[1];
exports.ES_PASS = matches[2];
console.log('Configured using BONSAI_URL environment variable', url, exports);
} |
So all we would need is a short sample of your data and the client-side code you're using to read the results and determine that it searches on all fields. Then I should be able to pinpoint why you're not getting the desired results. Feel free to email data to wulf at firebase or just sanitize sensitive fields; a few records should suffice. |
Data emailed to you, thanks. |
I have the same issue. No matter what I put in the fields array be it
I always get back a match on all fields when I would think if I don't have any key of 'foobar', I would get no matches. |
Thanks for the update; will have a look at this soon. |
So, I dug into this, and it looks like the "fields" param in export.paths isn't referenced anywhere else in Flashlight. The logical place would be in PathMonitor at line 6 (where the Firebase ref is being constructed), but I think that even that would be unnecessary. Perhaps just eliminate "fields" as a parameter altogether, and just create more complete paths. Example: path: "profiles" could be path: "profiles/foobar" In either case, having "fields" in the config.js comments is confusing, since it doesn't do anything as it currently stands. |
Fixed in latest head branch. Thanks gents! |
w00t! 👍 |
Added ability to index links
Added ability to index links
Added ability to index links
Stargazing search indexing via firebase
Configuring the app to monitor only specific fields doesn't seem to work. Even though I have specified a 'fields' array, search still returns results as if all fields are being monitored. Perhaps I have misunderstood how this is supposed to work?
The text was updated successfully, but these errors were encountered: