403 errors on csv files / master files missing

[jfaunt]

I imagine this is related to the previous issue that was posted here regarding scraper status. At this point, the master files are unavailable (for example the master entities.csv file used to be found at http://data.opensanctions.org/v1/sources/master/latest/entities.csv) for download and all currently listed files generate a 403 (access forbidden) error when clicked for download. We were just getting ready to launch a site that utilizes the data that you provide but this has forced us to stop the rollout. Is there any update on what is happening with your system? Can we depend on this site for reliable data access in the future or should we go back to building/consolidating our own lists? By the way, when it is working this is an excellent resource. Thanks for any information you can provide.

John

[pudo]

Hi @jfaunt. Apologies for this being broken for such a long time. We basically moved the crawlers to use a scraping framework (memorious) and moved the actual operation into @occrp infrastructure. Unfortunately, I hadn't finished an export function yet. In any case, from now on files should be uploaded every night to archive.org; they will be linked at http://opensanctions.org and also at: https://archive.org/details/opensanctions .