Handle timeout errors when calling fact_extractor, related to #820 #852
Conversation
…d#820 Timeouts are handled by catching all requests exceptions because requests raise ConnectionError on a timeout.
src/unpacker/unpack.py
Outdated
| file_object.processed_analysis['unpacker'] = { | ||
| 'plugin_used': 'None', 'number_of_unpacked_files': 0, 'plugin_version': '0.0', 'analysis_date': time(), | ||
| 'info': 'Unpacking stopped because extractor raised a exception (possible timeout)', | ||
| } | ||
| tag_dict = {'unpacker': {'extractor error': {'value': 'possible extractor timeout', 'color': TagColor.ORANGE, 'propagate': False}}} | ||
| file_object.analysis_tags.update(tag_dict) |
There was a problem hiding this comment.
The tags don't seem to be saved correctly in the database. But something like this should work:
| file_object.processed_analysis['unpacker'] = { | |
| 'plugin_used': 'None', 'number_of_unpacked_files': 0, 'plugin_version': '0.0', 'analysis_date': time(), | |
| 'info': 'Unpacking stopped because extractor raised a exception (possible timeout)', | |
| } | |
| tag_dict = {'unpacker': {'extractor error': {'value': 'possible extractor timeout', 'color': TagColor.ORANGE, 'propagate': False}}} | |
| file_object.analysis_tags.update(tag_dict) | |
| file_object.processed_analysis['unpacker'] = { | |
| 'plugin_used': 'None', 'number_of_unpacked_files': 0, 'plugin_version': '0.0', 'analysis_date': time(), | |
| 'info': 'Unpacking stopped because extractor raised a exception (possible timeout)', | |
| 'tags': {'extractor error': {'value': 'possible extractor timeout', 'color': TagColor.ORANGE, 'propagate': False}}, | |
| } |
This also applies to the _store_unpacking_depth_skip_info() method below
There was a problem hiding this comment.
I pushed the changed suggested (untested).
There was a problem hiding this comment.
Converting to draft as the changes are going do cause unneeded timeouts when using fact_extractor in standalone mode. I think the correct way to implement this should be by getting the timeout value from fact_core as a parameter.
There was a problem hiding this comment.
Converting to draft as the changes are going do cause unneeded timeouts when using fact_extractor in standalone mode.
I'm not sure I can follow. The changes in this PR should not affect the extractor. Did you maybe mean fkie-cad/fact_extractor#94?
I just tested the changes again and everything seems to work fine (with one exception: analysis tags are not always rendered correctly for firmware containers but that is something that is out of scope for this PR).
There was a problem hiding this comment.
You are correct, I meant to mark the other pull request as draft. This pull request is fine (I can not remove the draft status myself).
|
What is more, it would probably be a good idea to add |
Timeouts are handled by catching all requests exceptions because requests raise ConnectionError on a timeout.