Handle FileNotFoundError error in run_yara_query_rule_on_versions[_chunk]

[diox]

  File "/data/olympia/src/olympia/scanners/tasks.py", line 316, in run_yara_query_rule_on_versions_chunk
    _run_yara_query_rule_on_version(version, rule)
  File "/data/olympia/src/olympia/scanners/tasks.py", line 331, in _run_yara_query_rule_on_version
    definition=rule.definition)
  File "/data/olympia/src/olympia/scanners/tasks.py", line 214, in _run_yara_for_path
    zip_file = SafeZip(source=path)
  File "/data/olympia/src/olympia/files/utils.py", line 739, in __init__
    self.is_valid = self.initialize_and_validate()
  File "/data/olympia/src/olympia/files/utils.py", line 752, in initialize_and_validate
    zip_file = zipfile.ZipFile(self.source, self.mode)
  File "/usr/local/lib/python3.6/zipfile.py", line 1113, in __init__
    self.fp = io.open(file, filemode)
<class 'FileNotFoundError'>
FileNotFoundError(2, 'No such file or directory')

[diox]

Kinda weird, but since we're looking at deleted/disabled stuff I guess that's a possibility.

[wagnerand]

Maybe relevant: I recently experienced "File not found" issues shortly after the file status changed to STATUS_DISABLED, probably before the fixing cronjobs had a chance to run.

[diox]

Yeah. We're using current_file_path which deals with disabled files but if the file hasn't been moved yet it would trigger a File Not Found exception. I could set up the task to be retried a few times, but I suspect it would not be moved in time and the result would be the same. Maybe just ignoring this issue for now is enough, it's an edge case.

[diox]

On the test we did on prod it was happening for ~ 20 versions. Instead of simply retrying, we could retry immediately with the other path (disabled path if we were using the regular one and vice-versa). That should be enough.

[diox]

STR:

• Upload an add-on
• Once it's approved, wait for a few minutes after the 25th minute of the hour has passed (e.g. if it was 14h20 when you started, wait until 14h30) and disable the add-on (don't touch the versions/files).
• Run a code search using scanners query rule admin, using a yara definition that should find the add-on
• Make sure it's found in the results

[AlexandraMoga]

@diox here are my test method and results:

• submitted 4 add-ons, 2 listed and 2 unlisted
• I approved the listed and waited for the auto-approval to run on the unlisted
• I forced disabled one of the listed and one of the unlisted add-ons
• waited until 12h28 and changed the status for them back to Approved
• at 12h31 I've deleted the remaining unlisted add-on
• at 12h31 I've set the remaining listed add-on to Invisible
• at 12h33 I've started the query rule run

Result: all 4 add-ons have been identified by the query rule run