Handle missing harvest source without mass un-publishing all content from the source.

[janette]

connects #2824

When a harvest source is temporarily unavailable, all content from that harvest is unpublished and marked as 'orphaned'. On large harvests this can mean huge amounts of unnecessary processing that fails to finish before the next harvest and the source is available again.

Let's instead just leave the content as-is. If the source is truly gone, the catalog maintainer can delete the content via the Harvest UI.

QA Steps

missing source

1. Create a harvest source from https://s3.amazonaws.com/dkan-default-content-files/files/data_harvest_orphan_test.json, and run the harvest.
2. Log in to S3 and make the file private
3. Re-run the harvest, note that rather than unpublishing all of the datasets, you only get the error message:

Items to import is 0. Looks like source is missing. No updates will be made at this time.

Actual change in the source

1. Create a harvest source on a flavor build (http://janette.dkandemo.nuamsdev.com/data.json) and run the harvest.
2. Go to the source site and delete a dataset.
3. Run the harvest again
4. Confirm that the upstream deleted dataset was unpublished and marked as an 'orphan'

Reminders

• There is test for the issue.
• Coding standards checked.
• Review docs.getdkan.com (or in /docs) to see if it still covers the scope of the PR and update if needed.

[dafeder]

This PR will prevent all orphaning from happening, so isn't quite there yet. What we need to do is have the orphan function skip only if the source is missing. This means:

Ideally, if caching fails, the cache for this source should some how be flagged to be in a failed state, which would cause HarvestMigrate::processImport() to return FALSE, and therefore skip all postMigrate functionality.

Another option would be to simply have processImport() return FALSE whenever the cache is empty, which would include instances where a filter had removed all datasets. This would be less than ideal because presumably if a filter were to filter out all datasets from a source, orphaning everything would be the desired behavior.

[janette]

@dafeder Fixed! this version will now allow normal orphaning when needed.