Remove duplicate "Setting status of job to COMPLETE" Workflow Manager log line and other improvements

[jrobble]

Duplicate "Setting status of job to COMPLETE" log line:

From @jrobble :

Our Camel route invokes JobStatusCalculator.calculateStatus() (which in turn invokes inProgressJobs.setJobStatus()) before the job is sent to the JobCompleteProcessorImpl. Once it's there, we invoke createOutputObject() followed by inProgressJobs.setJobStatus().

createOutputObject() invokes storageService.store() which can change the job status if there is an issue writing out the JSON output object (possibly to the object store). For that reason we have to set the job status again afterward.

An end user is trying to create Kibana visualizations and was trying to get some job counts with associated status. Using the "Setting status of job to COMPLETE" log line would result in duplicate counts.

From @brosenberg42 :

I think the problem is actually that JobStatusCalculator.nextStatus() should actually be using BUILDING_OUTPUT_OBJECT instead of COMPLETE*.

JobCompleteProcessor would have to change the state from BUILDING_OUTPUT_OBJECT to COMPLETE*. This is so that the first log line shows BUILDING_OUTPUT_OBJECT but COMPLETE is used before generating the JSON output object.

Also, the job status REST endpoint should show BUILDING_OUTPUT_OBJECT while the JSON output object is being written.

---

Other improvements:

• Have the media inspection work queue log the job id, media url, data type, and mime type for each media