-
Notifications
You must be signed in to change notification settings - Fork 38
Collector Tester
The collector tester is a program that can be used after deploying the collector, to test the collector independently of the other modules.
It only requires the aidr-collector module to be deployed.
The collector tester is run through the following command:
mvn test -Dtest=CollectorTesterTest PARAMETERS
This parameter is mandatory:
-DcollectionTask=FILE
These parameters are optional:
-Dconfig=FILE
-Dtime=SECONDS (default 60)
-Dquiet=TRUE/FALSE (default false)
The config is the name of the collector configuration (or of a centralized configuration) to read properties that the tester needs to know to perform the testing: the URL of the collector's API, the name of the REDIS queue where the collector will write.
The collection-task is a file containing a CollectionTask object serialized in .properties format. A simple collection-task.properties file is included which describes the following test (listen for tweets in English containing the keyword "uk", which are many):
collectionCode = collector_tester
keywords = uk
languageFilter = en
persist = false
The time is the number of seconds during which the test has to be done.
The quiet option suppresses the print of the tweets. All other messages are printed even in quiet mode.
The collector tester should perform the following steps:
- Read the collection-task
- Subscribe to the REDIS queue where the collector writes data. This thread will print to the screen the received tweets (just the timestamp and first 40 characters of the text), unless the quiet option is specified. If subscription to REDIS fails, FAIL.
- Call the collector's
/startmethod to start a collection; if the return code is not OK, FAIL. - Wait for 1/2 of the time
- Call the collector's
/statusmethod to get the status of the collection. - If the status returned is not OK, FAIL.
- Wait for 1/2 of the time
- Call the collector's
/stopmethod; if the return code is not OK, FAIL. - Call the collector's
/statusmethod, if the status returned is different from "not running", FAIL. - Print the number of tweets received so far, if the number is zero, FAIL.
- Exit with exit code 0 (success) otherwise.
FAIL means printing a clear and informative message describing the condition and exiting with code 1 (non success).
On interrupt by the user, the collector tester should attempt to call the collector's /stop method and exit with non-success code.
- Home
- [What is AIDR?](AIDR Overview)
- The science behind AIDR
- [Operator's manual](AIDR Operator's Manual)
- [Public API documentation](API documentation)
- High-level overview
- Common
- DB Manager
- DTO standards
- Database schema
- Manager
- Manager API
- Collector
- Collector API
- Reconnect strategy
- Collector Tester
- Output
- Output API
- Output Buffered
- Output Streaming
- Output Tester
- Persister
- Persister API
- Persister Tester
- Tagger
- Tagger Concepts
- Tagger API
- Tagger Tester
- Trainer
- Trainer API
- PyBossa Trainer