Handle 404 NOT FOUND error for download_and_extract_antismash_data

[gcroci2]

With this PR:

• Now the hash of the project's json file (MSV000079284.json in our test case) is checked, and if the file changes an error is raised
• Now two try-except statements handle better the creation of antismash extracted folders, which happens only if the corresponding zip folder has been downloaded. Also, the genome_status.json file now contains an empty string value for the key bgc_path in the cases in which no zip folder has been downloaded and no folder has been extracted.
• I added one test in tests/genomics/antismash/test_antismash_downloader.py to assess that download_and_extract_antismash_data does not create an empty folder when the id does not exist in NCBI and when the id does exist there but not in the antismash database (404 error).
• I added two tests in tests/pairedomics/test_podp_antismash_downloader.py to assess that podp_download_and_extract_antismash_data does not create an empty folder when the id does not exist in NCBI and when the id does exist there but not in the antismash database (404 error); they also check for the correctness of the genome status file.

In conclusion, we were already handling cases in which the genome id was non-existing, but we weren't handling cases in which the genome id was existing in NCBI but not in the antismash database. Now we handle both cases.

Notes for @CunliangGeng:

• tests/test_nplinker_local.py still fails, but it's an error unrelated to this PR (maybe it will be solved with Create StrainMappingLoader #116?):

ERROR tests/test_nplinker_local.py::test_load_data - Exception: Failed to find *ANY* strains, missing strain_mappings.json?

• running mypy on src/nplinker/genomics/antismash/antismash_downloader.py gives me an error about os.scandir, but I didn't find how to solve it:

src/nplinker/genomics/antismash/antismash_downloader.py:91: error: Value of type variable "AnyStr" of "scandir" cannot be "Union[PathLike[Any], Any]"  [type-var]

[CunliangGeng]

We didn't catch such errors because the possibility of having a broken URL wasn't tested. Should we create a unit test for that?

Yes please!

[gcroci2]

See my updated comment above @CunliangGeng

[CunliangGeng]

I guess we need to define what "non-existent ID" is.
For the function download_and_extract_antismash_data, the non-existent ID (fake ID, not exist in NCBI) or broken ID (not exist in antismash database) are actually the same thing, both will trigger a 404 error. It'd be easier to treat them as the same case I guess.

[CunliangGeng]

the os.scandir type issue is fine, we can ignore it.

[CunliangGeng]

the os.scandir type issue is fine, we can ignore it.

you can change if any(os.scandir(extract_path)) to if any(os.scandir(str(extract_path))) to fix the mypy issue. os.scandir need string as input but we give a Pathlike.

[CunliangGeng]

The mypy issues should be fixed in my suggested changes. Apply them, then you can merge this PR 🎉