Rogue node detection. #27
Conversation
|
So, why are tests failing here? |
|
|
||
| from nodewatcher import celery | ||
|
|
||
| from nodewatcher.modules.monitor.http.survey.management.commands.export_survey_data import extract_survey_graph |
There was a problem hiding this comment.
I think it would be better if this particular function would be somewhere else, and then both the management command and this task would import it.
|
It's no longer failing. Had to rename a couple of modules. |
| graph=input_graph['graph'], | ||
| friendly_nodes=input_graph['friendly_nodes'], | ||
| ) | ||
| # append -results to the end of the filename |
|
Hm, do you think it is useful to store whole results for your tests? Maybe. But isn't the idea that you detect rogue nodes, but slight changes in values might not be so problematic? So maybe more important is that class rogue/not does not change? |
| """ | ||
|
|
||
| nx_graph = nx.Graph() | ||
| nx_graph.add_nodes_from([(node['i'], {'b': node['b']}) if 'b' in node else node['i'] for node in graph["v"]]) |
|
When you say "results", you mean the output of the detection algorithm? Meaning a bunch of nodes and the probability of each being rogue? So what should we do? |
|
Yes, I implemented a custom test suite. Please do a code review. |
|
We could also try to ensure that the numbers are in a certain range, as you proposed, but it is not within the scope of the algorithm to determine the "classification" of a rogue node. That is done by the maintainers. So we might be breaking abstraction barriers. But more importantly, I don't foresee anyone tampering with this algorithm in the near future so I wouldn't spend too much time trying to predict how they will work on it. |
| """ | ||
|
|
||
| nx_graph = nx.Graph() | ||
| nx_graph.add_nodes_from([(node['i'], {'b': node['b']}) if 'b' in node else node['i'] for node in graph['v']]) |
There was a problem hiding this comment.
I do not get it here. Nodes are or tuple of (ID, {b: BSSID}) or just ID? This hurts my mental type-checking. :-)
There was a problem hiding this comment.
Yes, that's how you create nodes in nx:
http://networkx.readthedocs.io/en/latest/reference/generated/networkx.Graph.add_node.html#networkx.Graph.add_node
It has to be a tuple in case you're storing additional information.
|
I reviewed the pull request. Please answer questions I made. Also see changes I made. See if everything still works. |
|
Hi, not all tests are running with your code. Django only reports running one test: comparing 'behind-couch-....' with its results. How do we fix this? |
| for filename in files: | ||
| # Test every JSON file that does not contain "results" in the filename. | ||
| if os.path.splitext(filename)[1] == '.json' and 'results' not in filename: | ||
| test_cases.addTest(RogueNodeTestCase('run_test', os.path.join(path, filename))) |
There was a problem hiding this comment.
Can you check if this is called for all files?
There was a problem hiding this comment.
Before you modified the code, I was able to use print statements to debug. It seems that stdout is now being redirected. I also can't use pdb, it seems to skip over the whole thing. How do I verify that this is called?
There was a problem hiding this comment.
Print statements in load_tests should work, because that is called when tests are being loaded (the same as before in your code). So does for loop work correctly?
Inside tests stdout is collected and you get it to output only if there is an error. So one simple way is to throw an exception with your message. ;-)
There was a problem hiding this comment.
OK, the for loop works correctly and finds all the test cases. test_cases is an array of 7 elements, one for every test we have.
But run_test only runs once in total (only with the first test_filename).
There was a problem hiding this comment.
Not sure. Research. Maybe the issue is that we have the same name for tests all the time. So id returns the same. You should probably override id and get it to return module name + class name + test_filename or something.
There was a problem hiding this comment.
See how it is officially defined. You could also just call super and append fest_filename to the id.
There was a problem hiding this comment.
I decided it would be easier to construct a test_case with a directory and the test would go into the directory and check each entry. This also actually works.
There was a problem hiding this comment.
Have you tried id method approach?
Are you sure it works? That tests don't now stop on first error? That you run all tests and allow some of them to fail?
There was a problem hiding this comment.
Hi,
the overriding id doesn't work.
Tests probably do stop on the first error, but at least all of them run.
There was a problem hiding this comment.
ok. I found out that I have to override the hashing function. It works now. will push new code shortly.
| with io.open(os.path.join(path, results_filename), encoding='utf-8') as asserted_output_file: | ||
| asserted_output = json.load(asserted_output_file) | ||
|
|
||
| self.assertEqual(algorithm_output, asserted_output) |
There was a problem hiding this comment.
If any test fails now, other tests are not run.
|
@mitar Can you do a code review of this module? |
|
Perfect. Merging! |
This PR should be used to discuss an implementation of the rogue node algorithm implementation.