Spidercrab: i06t02, i06t03, i06t04, i06t07 #118
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Note: pass node_id to get_configuration and test if it works (use don on ocean-don.no-ip.biz) |
|
|
Added checking for empty model nodes. Something can be wrong we neo4j. We will return to this problem if it occurs again
Now master and slave Spidercrabs are looping. If there is need to terminate them after work is done - use `"terminate_on_end": 1` parameter in config.json.
|
Looping now works as described in 1bd2c1b. |
Add --no-corleone option to spidercrab scripts.
ktalik
changed the title
ktalik
changed the title
|
Add last fixes and improvements. This pull request is ready to test and merge. It closes following tasks:
Please comment if there is anything missing. |
Improved Spidercrab master and slave functionalities: * Fix bug of loop in recursion of finding another content source when a particular source is no longer available on given url * Improved _news_is_supported() method * Fixed Corleone config fetching for command line options inside spidecrab init scripts * Add logging stats into json file inside logs/ directory * Add -t option (for stats exporting) to spidercrab init scripts This commit closes tasks #103 and #106.
Related to task #105.
|
Spidercrab is passing the test but I am thinking about more restrictive version of this test now. |
kudkudak
added a commit
that referenced
this pull request
Apr 24, 2014
Spidercrab: i06t02, i06t03, i06t04, i06t07
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
don_utils.pyMerging this request would close issue #105.