MR Jobs failing to List after analyzing

[shahrukhkhan489]

As per Steve - Not all MR jobs listed in Dr.elephant dashboard, Dr. elephant log displays the following message, it appears the success fetch, however the two jobs are NOT listed in Dr. elephant dashboard in the end

07-21-2016 10:32:10 INFO  com.linkedin.drelephant.ElephantRunner : Job queue size is 2
07-21-2016 10:32:10 INFO  com.linkedin.drelephant.ElephantRunner : Executor thread 1 analyzing MAPREDUCE application_1469039591592_0008
07-21-2016 10:32:10 INFO  com.linkedin.drelephant.ElephantRunner : Executor thread 3 analyzing MAPREDUCE application_1469039591592_0007
07-21-2016 10:32:10 INFO  com.linkedin.drelephant.mapreduce.ThreadContextMR2 : Executor 2 updates its AuthenticatedToken.
07-21-2016 10:32:11 INFO  com.linkedin.drelephant.mapreduce.ThreadContextMR2 : Executor 3 updates its AuthenticatedToken.

[shahrukhkhan489]

Plamen Jeliazkov seems to have resolved the issue by commenting out HttpURLConnection conn object.

  private JsonNode readJsonNode(URL url)
      throws IOException, AuthenticationException {
//    HttpURLConnection conn = _authenticatedURL.openConnection(url, _token);
    return _objectMapper.readTree(url.openStream());
  }

Plamen noticed the same symptom on his Dr. Elephant instance running on a secure cluster.

I debugged into it and found that if I performed the AuthenticatedURL I got back a JSON object that had "startedTime" and "finishedTime" both set to 0.
I took the URL toString provided by my remote debug and pasted it into my browser and the JSON object I saw there HAD valid "startedTime" and "finishedTime" times.

[shahrukhkhan489]

Created Pull request #302 for the same