Authentication 401 Error on River #29

Chadwiki · 2014-04-17T21:07:51Z

Authentication is failing on the river:
I was able to request using a CURL>

curl --user username@account:password "https://companytemp.saas.appdynamics.com/controller/rest/applications/Service%20Management%20-
%20SEA%20PR/metric-data?metric-path=Overall%20Application%20Performance%7CAverag
e%20Response%20Time%20%28ms%29&time-range-type=BEFORE_NOW&duration-in-mins=15&ou
tput=json" -k

####### output

[{
"frequency": "ONE_MIN",
"metricPath": "Overall Application Performance|Average Response Time (ms)",
"metricValues": [ {
"current": 234,
"max": 274717,
"min": 0,
"startTimeInMillis": 1397762100000,
"value": 238
}]
}]

####### end output

I used the index/username and password fields in the river.

I tested two ways in the config:

"username" : "username@account ",
 "password" : "password",

"username" : "username@account:password ",
 "password" : "",

####### ElasticSearch Log

407][DEBUG][org.apache.http.client.protocol.RequestAddCookies] CookieSpec selected: best-match
407][DEBUG][org.apache.http.client.protocol.RequestAuthCache] Re-using cached 'basic' auth scheme for https://companytemp.saas.appdynamics.com:443
408][DEBUG][org.apache.http.client.protocol.RequestAuthCache] No credentials for preemptive authentication
408][DEBUG][org.apache.http.client.protocol.RequestTargetAuthentication] Target auth state: UNCHALLENGED
408][DEBUG][org.apache.http.client.protocol.RequestProxyAuthentication] Proxy auth state: UNCHALLENGED
408][DEBUG][org.apache.http.impl.client.DefaultHttpClient] Attempt 1 to execute request
409][DEBUG][org.apache.http.impl.conn.DefaultClientConnection] Sending request: GET /controller/rest/applications/Service%20Management%20-%20SEA%20PR/metric-data?metric-path=Overall%20Application%20Performance%7CAverage%20Response%20Time%20%28ms%29&time-range-type=BEFORE_NOW&duration-in-mins=15&output=json HTTP/1.1
410][DEBUG][org.apache.http.wire ] >> "GET /controller/rest/applications/Service%20Management%20-%20SEA%20PR/metric-data?metric-path=Overall%20Application%20Performance%7CAverage%20Response%20Time%20%28ms%29&time-range-type=BEFORE_NOW&duration-in-mins=15&output=json HTTP/1.1[\r][\n]"
410][DEBUG][org.apache.http.wire ] >> "Accept: application/json[\r][\n]"
411][DEBUG][org.apache.http.wire ] >> "Host: companytemp.saas.appdynamics.com[\r][\n]"
411][DEBUG][org.apache.http.wire ] >> "Connection: Keep-Alive[\r][\n]"
411][DEBUG][org.apache.http.wire ] >> "User-Agent: Apache-HttpClient/4.2.3 (java 1.5)[\r][\n]"
411][DEBUG][org.apache.http.wire ] >> "[\r][\n]"
411][DEBUG][org.apache.http.headers ] >> GET /controller/rest/applications/Service%20Management%20-%20SEA%20PR/metric-data?metric-path=Overall%20Application%20Performance%7CAverage%20Response%20Time%20%28ms%29&time-range-type=BEFORE_NOW&duration-in-mins=15&output=json HTTP/1.1
412][DEBUG][org.apache.http.headers ] >> Accept: application/json
412][DEBUG][org.apache.http.headers ] >> Host: companytemp.saas.appdynamics.com
413][DEBUG][org.apache.http.headers ] >> Connection: Keep-Alive
413][DEBUG][org.apache.http.headers ] >> User-Agent: Apache-HttpClient/4.2.3 (java 1.5)
467][DEBUG][org.apache.http.wire ] << "HTTP/1.1 401 Unauthorized[\r][\n]"
467][DEBUG][org.apache.http.wire ] << "X-Powered-By: Servlet/3.0 JSP/2.2 (GlassFish Server Open Source Edition 3.1.2.2 Java/Sun Microsystems Inc./1.6)[\r][\n]"
468][DEBUG][org.apache.http.wire ] << "Server: GlassFish Server Open Source Edition 3.1.2.2[\r][\n]"
468][DEBUG][org.apache.http.wire ] << "Pragma: No-cache[\r][\n]"
468][DEBUG][org.apache.http.wire ] << "Cache-Control: no-cache[\r][\n]"
469][DEBUG][org.apache.http.wire ] << "Expires: Wed, 31 Dec 1969 16:00:00 PST[\r][\n]"
469][DEBUG][org.apache.http.wire ] << "WWW-Authenticate: Basic realm="controller_realm"[\r][\n]"
469][DEBUG][org.apache.http.wire ] << "Content-Type: text/html[\r][\n]"
469][DEBUG][org.apache.http.wire ] << "Content-Length: 1073[\r][\n]"
470][DEBUG][org.apache.http.wire ] << "Date: Thu, 17 Apr 2014 20:40:07 GMT[\r][\n]"
470][DEBUG][org.apache.http.wire ] << "X-Varnish: 1905701652[\r][\n]"
470][DEBUG][org.apache.http.wire ] << "Age: 0[\r][\n]"
470][DEBUG][org.apache.http.wire ] << "Via: 1.1 varnish[\r][\n]"
471][DEBUG][org.apache.http.wire ] << "Connection: keep-alive[\r][\n]"
471][DEBUG][org.apache.http.wire ] << "[\r][\n]"
471][DEBUG][org.apache.http.impl.conn.DefaultClientConnection] Receiving response: HTTP/1.1 401 Unauthorized
471][DEBUG][org.apache.http.headers ] << HTTP/1.1 401 Unauthorized
472][DEBUG][org.apache.http.headers ] << X-Powered-By: Servlet/3.0 JSP/2.2 (GlassFish Server Open Source Edition 3.1.2.2 Java/Sun Microsystems Inc./1.6)
472][DEBUG][org.apache.http.headers ] << Server: GlassFish Server Open Source Edition 3.1.2.2
472][DEBUG][org.apache.http.headers ] << Pragma: No-cache
472][DEBUG][org.apache.http.headers ] << Cache-Control: no-cache
472][DEBUG][org.apache.http.headers ] << Expires: Wed, 31 Dec 1969 16:00:00 PST
473][DEBUG][org.apache.http.headers ] << WWW-Authenticate: Basic realm="controller_realm"
473][DEBUG][org.apache.http.headers ] << Content-Type: text/html
473][DEBUG][org.apache.http.headers ] << Content-Length: 1073
473][DEBUG][org.apache.http.headers ] << Date: Thu, 17 Apr 2014 20:40:07 GMT
473][DEBUG][org.apache.http.headers ] << X-Varnish: 1905701652
474][DEBUG][org.apache.http.headers ] << Age: 0
474][DEBUG][org.apache.http.headers ] << Via: 1.1 varnish
474][DEBUG][org.apache.http.headers ] << Connection: keep-alive
474][DEBUG][org.apache.http.impl.client.DefaultHttpClient] Connection can be kept alive indefinitely
475][DEBUG][org.apache.http.impl.client.DefaultHttpClient] Authentication required
475][DEBUG][org.apache.http.impl.client.DefaultHttpClient] companytemp.saas.appdynamics.com:443 requested authentication
475][DEBUG][org.apache.http.impl.client.TargetAuthenticationStrategy] Authentication schemes in the order of preference: [negotiate, Kerberos, NTLM, Digest, Basic]
475][DEBUG][org.apache.http.impl.client.TargetAuthenticationStrategy] Challenge for negotiate authentication scheme not available
476][DEBUG][org.apache.http.impl.client.TargetAuthenticationStrategy] Challenge for Kerberos authentication scheme not available
476][DEBUG][org.apache.http.impl.client.TargetAuthenticationStrategy] Challenge for NTLM authentication scheme not available
476][DEBUG][org.apache.http.impl.client.TargetAuthenticationStrategy] Challenge for Digest authentication scheme not available

The text was updated successfully, but these errors were encountered:

velias · 2014-04-18T07:09:48Z

Are you sure you configured username and password correctly? It have to be in remote part of configuration, not index as you wrote. You can look for INFO log message Configured GET JSON remote client. S... where you can see configured username.
River supports HTTP Basic authentication for now only. I see some sights of NTLM in your log. Are you sure your curl used Basic auth not NTLM (as it seems curl supports NTLM)

Chadwiki · 2014-04-18T07:49:21Z

I have this set in the remote part of the config, Right beneath the URL.
The target URL accepts basic authentication.
I will check the Curl command in the morning.

Any other suggestions,
I started to looking CA cert issue. I had tested another URL with curl and was forced to use the -k or insecure switch. Not on the URL I'm using for this config.

I can paste my config later.

Btw thanks for the help on the previous issues.
I should have known the fields issue. Uses the same style of config in the jira river. ;-) basically same code base.

velias · 2014-04-18T08:48:22Z

Try curl with --basic which should force it to use Basic authentication only.
Other thing is that river used Basic authentication in Pre-emptive mode, which means it automatically sends necessary auth header in first request, without waiting for server to challenge it over 401. Most REST API's need this mode because they return anonymously available data in preemptive mode is not used. I'm not sure if it is possible to simulate this mode over curl to check you server support it.

Regarding CA. If you use https server which certificate is not signed by common trusted authority, then you have to import https server certificate into cacerts store of JVM used to run river. Eg. http://codebistro.com/2010/03/25/adding-cacert-to-the-java-trusted-store/

Yep, this river is generalization of a bit older jira river, shares base ideas and some codebase definitely.

Chadwiki · 2014-04-18T19:13:58Z

CURL with forced --basic works

Chadwiki · 2014-04-23T15:26:27Z

This issue is on hold until API URL target is back online

Chadwiki · 2014-05-13T19:47:13Z

Here is an update on curl, output, river config - I am still receiving an 401. I'm not sure if its the username field or something else. I did remove the data Dir on test system and rebuilt the river.
I have checked the Cert Chain, I'm at a loss...

CURL:
curl --user username@companyid:password "https://companyid.saas.appdynamics.com/controller/rest/applications/ServiceMgmnt/metric-data?metric-path=Overall%20Application%20Performance%7CErrors%20per%20Minute&time-range-type=BEFORE_NOW&duration-in-mins=15&output=json" -k --basic

OUTPUT:

[{
"frequency": "ONE_MIN",
"metricName": "BTM|Application Summary|Errors per Minute",
"metricPath": "Overall Application Performance|Errors per Minute",
"metricValues": [ {
"count": 1356,
"current": 8,
"max": 0,
"min": 0,
"occurrences": 0,
"standardDeviation": 0,
"startTimeInMillis": 1400007600000,
"sum": 388,
"useRange": false,
"value": 26
}]
}]

RIVER CONFIG:

{
"type" : "remote",
"remote" : {
"urlGetDocuments" : "https://companyid.saas.appdynamics.com/controller/rest/applications/ServiceMgmnt/metric-data?metric-path=Overall%20Application%20Performance%7CErrors%20per%20Minute&time-range-type=BEFORE_NOW&duration-in-mins=15&output=json",
"getDocsResFieldDocuments" : "metricValues",
"username" : "username@companyid",
"password" : "password",
"timeout" : "30s",
"spacesIndexed" : "MAIN",
"spaceKeysExcluded" : "",
"indexUpdatePeriod" : "5m",
"indexFullUpdatePeriod" : "4h",
"simpleGetDocuments" : "true",
"maxIndexingThreads" : 1
},
"index" : {
"index" : "remote_river_index",
"type" : "appdynamics",
"remote_field_document_id" : "metricPath",
"fields" : {
"frequency" : {"remote_field" : "frequency"},
"metricPath" : {"remote_field" : "metricPath"},
"count" : {"remote_field" : "count"},
"current" : {"remote_field" : "current"},
"max" : {"remote_field" : "max"},
"min" : {"remote_field" : "min"},
"occurances" : {"remote_field" : "occurances"},
"standardDiviation" : {"remote_field" : "standardDiviation"},
"startTimeInMillis" : {"remote_field" : "startTimeInMillis"},
"sum" : {"remote_field" : "sum"},
"useRange" : {"remote_field" : "useRange"},
"value" : {"remote_field" : "value"}
}

},
"activity_log": {
    "index" : "remote_river_activity",
    "type"  : "remote_river_indexupdate"
}

}

ERROR in River:
error_message: "Failed remote system HTTP GET request to the url 'https://companyid.saas.appdynamics.com/controller/rest/applications/ServiceMgmnt/metric-data?metric-path=Overall%20Application%20Performance%7CErrors%20per%20Minute&time-range-type=BEFORE_NOW&duration-in-mins=15&output=json'. HTTP error code: 401 Response body: <title>GlassFish Server Open Source Edition 3.1.2.2 - Error report</title><style type="text/css"></style>

HTTP Status 401 -

type Status report

message

descriptionThis request requires HTTP authentication ().

GlassFish Server Open Source Edition 3.1.2.2

"

Chadwiki · 2014-05-14T22:19:58Z

Maybe this will help. it was a successful test using Java...

URL url = new URL("https://company.saas.appdynamics.com/controller/rest/applications/");
HttpsURLConnection conn = (HttpsURLConnection) url.openConnection();
conn.setRequestMethod("GET");
conn.setRequestProperty("Accept", "application/json");
String userpass = " username@companyid:password ";
String basicAuth = "Basic " + javax.xml.bind.DatatypeConverter.printBase64Binary(userpass.getBytes());
conn.setRequestProperty ("Authorization", basicAuth);

            if (conn.getResponseCode() != 200) {
                    throw new RuntimeException("Failed : HTTP error code : "
                                    + conn.getResponseCode());
            }

            BufferedReader br = new BufferedReader(new InputStreamReader(
                    (conn.getInputStream())));

            String output;
            while ((output = br.readLine()) != null) {
                    System.out.println(output);
            }

            conn.disconnect();

velias · 2014-05-16T07:03:12Z

River uses HttpClient for calls. Exact code can be found at https://github.com/searchisko/elasticsearch-river-remote/blob/master/src/main/java/org/jboss/elasticsearch/river/remote/HttpRemoteSystemClientBase.java
Exact version of HttpClient is:

 <dependency>
      <groupId>org.apache.httpcomponents</groupId>
      <artifactId>httpclient</artifactId>
      <version>4.2.6</version>
  </dependency>

If you should test/debug exactly this code and httpclient version with your server and credentials it should help probably.

Chadwiki · 2014-09-04T16:43:22Z

I'm back to this issue.
ES - 1.3.2 + Remote_plugin 1.5.0
I have tested forcing the -basic auth in CURL, which works correctly.

FYI - I am working on a clean no old _river system.
I was able to connect to the open API test endpoint. This one does not require authentication. This proved the plugin works... {{ http://docs.appdynamics.com/download/attachments/20187207/REST_WildCardBT_metric-dataJSON.txt?version=1&modificationDate=1394226069000&api=v2 }}

The error has not changed and no other details are available in the logs...

Chadwiki · 2014-09-04T16:48:17Z

error_message: "Failed remote system HTTP GET request to the url 'https://company.saas.appdynamics.com/controller/rest/applications/SEAP%20-%20CE%20-%20Production%20-%201/metric-data?metric-path=Backends%7CDefault%20Web%20Site/ClaqServices%7CAverage%20Response%20Time%20%28ms%29&time-range-type=BEFORE_NOW&duration-in-mins=15&output=json'. HTTP error code: 401 Response body: <html xmlns="http://www.w3.org/1999/xhtml\"><title>GlassFish Server Open Source Edition 3.1.2.2 - Error report</title><style type="text/css"></style>

HTTP Status 401 -

type Status report

message

descriptionThis request requires HTTP authentication ().

GlassFish Server Open Source Edition 3.1.2.2

"

velias · 2014-09-05T14:14:49Z

I just realized that you probably store password into incorrect field of river configuration. pwd should be used instead of password

Chadwiki · 2014-09-05T14:49:38Z

{
"type" : "remote",
"remote" : {
"urlGetDocuments" : "https://company.saas.appdynamics.com/controller/rest/applications/SEAP%20-%20CE%20-%20Production%20-%201/metric-data?metric-path=Backends%7CDefault%20Web%20Site/ClaqServices%7CAverage%20Response%20Time%20%28ms%29&time-range-type=BEFORE_NOW&duration-in-mins=15&output=json",
"username" : "company@servicemgmt",
"pwd" : "passw0rd",
"timeout" : "5s",
"spacesIndexed" : "MAIN",
"spaceKeysExcluded" : "",
"indexUpdatePeriod" : "1m",
"indexFullUpdatePeriod" : "0",
"simpleGetDocuments" : "true",
"maxIndexingThreads" : 2
}, ......

velias added the question label Apr 18, 2014

Chadwiki closed this as completed May 15, 2014

Chadwiki reopened this May 15, 2014

velias closed this as completed Aug 29, 2014

Chadwiki mentioned this issue Sep 5, 2014

Authentication 401 Error on River from #29 #43

Closed

velias reopened this Sep 5, 2014

Chadwiki closed this as completed Sep 11, 2014

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Authentication 401 Error on River #29

Authentication 401 Error on River #29

Chadwiki commented Apr 17, 2014

velias commented Apr 18, 2014

Chadwiki commented Apr 18, 2014

velias commented Apr 18, 2014

Chadwiki commented Apr 18, 2014

Chadwiki commented Apr 23, 2014

Chadwiki commented May 13, 2014

Chadwiki commented May 14, 2014

velias commented May 16, 2014

Chadwiki commented Sep 4, 2014

Chadwiki commented Sep 4, 2014

velias commented Sep 5, 2014

Chadwiki commented Sep 5, 2014

Authentication 401 Error on River #29

Authentication 401 Error on River #29

Comments

Chadwiki commented Apr 17, 2014

####### output

####### end output

####### ElasticSearch Log

velias commented Apr 18, 2014

Chadwiki commented Apr 18, 2014

velias commented Apr 18, 2014

Chadwiki commented Apr 18, 2014

Chadwiki commented Apr 23, 2014

Chadwiki commented May 13, 2014

HTTP Status 401 -

GlassFish Server Open Source Edition 3.1.2.2

Chadwiki commented May 14, 2014

velias commented May 16, 2014

Chadwiki commented Sep 4, 2014

Chadwiki commented Sep 4, 2014

HTTP Status 401 -

GlassFish Server Open Source Edition 3.1.2.2

velias commented Sep 5, 2014

Chadwiki commented Sep 5, 2014