when preemptive auth is disabled HTTP Sampler does not automatically respond to Basic Auth challenge #5265
Comments
|
Peter Lynch (migrated from Bugzilla): |
|
Peter Lynch (migrated from Bugzilla): Created attachment jmeter-3.3-working-automatic-challenge-response.log: log file proof that jmeter version 3.3 did what I expected |
|
@pmouawad (migrated from Bugzilla): |
|
Peter Lynch (migrated from Bugzilla):
I don't understand your claim. How do you explain what jmeter 3.3 did? https://bz.apache.org/bugzilla/attachment.cgi?id=37123 And how do you explain the documentation contradicting the behaviour? |
|
@pmouawad (migrated from Bugzilla):
Can you clarify ?
Which documentation ? Can you point me to it please ? |
|
Peter Lynch (migrated from Bugzilla):
The following logging extracted from the attachments on this bug. ======= JMeter 3.3 jmeter 3.3 request made: jmeter 3.3 response challenged: jmeter 3.3 httpclient replies to challenge: jmeter 3.3 remote confirms with success jmeter asserts a successful response: ============= JMeter 5 jmeter 5 request made: jmeter 5 response challenge: jmeter 5 HttpClient, knows it was challenged jmeter 5 rather than responding to challenge, httpclient releases the connection back to pool: jmeter response assertion fails immediately on 401 status code, instead of sending the expected challenge answer:
Are you not seeing the opening comments of this report? Are you not looking at the attachments? This page: https://jmeter.apache.org/usermanual/component_reference.html#HTTP_Authorization_Manager |
|
@FSchumacher (migrated from Bugzilla): The attached patch fixes the issue (at least on my machine :) ), but I am a bit unsure, whether this is the right way to fix it. I had to put my hand deep into some foreign classes, to get the authorization right. But it would be nice, if you could try the patch on your side and report back, if it fixes your problem. Created attachment 01.diff: Fix non-preemptive authentication for http sampler 01.diffdiff --git a/bin/jmeter.properties b/bin/jmeter.properties
diff --git a/src/protocol/http/src/main/java/org/apache/jmeter/protocol/http/sampler/HTTPHC4Impl.java b/src/protocol/http/src/main/java/org/apache/jmeter/protocol/http/sampler/HTTPHC4Impl.java
index b24cb66bf2..3808bb9e74 100644
--- a/src/protocol/http/src/main/java/org/apache/jmeter/protocol/http/sampler/HTTPHC4Impl.java
+++ b/src/protocol/http/src/main/java/org/apache/jmeter/protocol/http/sampler/HTTPHC4Impl.java
@@ -24,12 +24,14 @@ import java.io.OutputStream;
import java.io.UnsupportedEncodingException;
import java.net.InetAddress;
import java.net.InetSocketAddress;
+import java.net.MalformedURLException;
import java.net.URI;
import java.net.URISyntaxException;
import java.net.URL;
import java.net.URLDecoder;
import java.nio.charset.Charset;
import java.security.GeneralSecurityException;
+import java.security.Principal;
import java.security.PrivilegedActionException;
import java.security.PrivilegedExceptionAction;
import java.util.ArrayList;
@@ -62,6 +64,7 @@ import org.apache.http.auth.AuthScope;
import org.apache.http.auth.AuthState;
import org.apache.http.auth.Credentials;
import org.apache.http.auth.NTCredentials;
+import org.apache.http.auth.UsernamePasswordCredentials;
import org.apache.http.client.AuthCache;
import org.apache.http.client.AuthenticationStrategy;
import org.apache.http.client.CredentialsProvider;
@@ -119,6 +122,7 @@ import org.apache.http.impl.client.HttpClients;
import org.apache.http.impl.client.LaxRedirectStrategy;
import org.apache.http.impl.client.ProxyAuthenticationStrategy;
import org.apache.http.impl.client.StandardHttpRequestRetryHandler;
+import org.apache.http.impl.client.TargetAuthenticationStrategy;
import org.apache.http.impl.conn.DefaultHttpClientConnectionOperator;
import org.apache.http.impl.conn.DefaultSchemePortResolver;
import org.apache.http.impl.conn.PoolingHttpClientConnectionManager;
@@ -205,6 +209,80 @@ public class HTTPHC4Impl extends HTTPHCAbstractImpl {
private static final InputStreamFactory BROTLI = BrotliInputStream::new;
+ private static final class ManagedCredentialsProvider implements CredentialsProvider {
+ private AuthManager authManager;
+ private Credentials proxyCredentials;
+ private AuthScope proxyAuthScope;
+
+ public ManagedCredentialsProvider(AuthManager authManager, AuthScope proxyAuthScope, Credentials proxyCredentials) {
+ this.authManager = authManager;
+ this.proxyAuthScope = proxyAuthScope;
+ this.proxyCredentials = proxyCredentials;
+ }
+
+ @Override
+ public void setCredentials(AuthScope authscope, Credentials credentials) {
+ log.debug("Store creds {} for {}", credentials, authscope);
+ }
+
+ @Override
+ public Credentials getCredentials(AuthScope authScope) {
+ log.info("Get creds for {}", authScope);
+ if (this.proxyAuthScope != null && authScope.equals(proxyAuthScope)) {
+ return proxyCredentials;
+ }
+ final Authorization authorization = getAuthorizationForAuthScope(authScope);
+ if (authorization == null) {
+ return null;
+ }
+ return new UsernamePasswordCredentials(authorization.getUser(), authorization.getPass());
+ }
+
+ /**
+ * Find the Authorization for the given AuthScope. We can't ask the AuthManager
+ * by the URL, as we didn't get the scheme or path of the URL. Therefore we do a
+ * best guess on the information we have
+ *
+ * @param authScope information which destination we want to get credentials for
+ * @return matching authorization information entry from the AuthManager
+ */
+ private Authorization getAuthorizationForAuthScope(AuthScope authScope) {
+ if (authScope == null) {
+ return null;
+ }
+ for (JMeterProperty authProp : authManager.getAuthObjects()) {
+ Object authObject = authProp.getObjectValue();
+ if (authObject instanceof Authorization) {
+ Authorization auth = (Authorization) authObject;
+ if (!authScope.getRealm().equals(auth.getRealm())) {
+ continue;
+ }
+ try {
+ URL authUrl = new URL(auth.getURL());
+ if (authUrl.getHost().equals(authScope.getHost()) && getPort(authUrl) == authScope.getPort()) {
+ return auth;
+ }
+ } catch (MalformedURLException e) {
+ log.debug("Invalid URL {} in authManager", auth.getURL());
+ }
+ }
+ }
+ return null;
+ }
+
+ private int getPort(URL url) {
+ if (url.getPort() == -1) {
+ return url.getProtocol().equals("https") ? 443 : 80;
+ }
+ return url.getPort();
+ }
+
+ @Override
+ public void clear() {
+ log.debug("clear creds");
+ }
+ }
+
private static final class PreemptiveAuthRequestInterceptor implements HttpRequestInterceptor {
@Override
public void process(HttpRequest request, HttpContext context) throws HttpException, IOException {
@@ -1050,21 +1131,27 @@ public class HTTPHC4Impl extends HTTPHCAbstractImpl {
}
// Set up proxy details
+ AuthScope proxyAuthScope = null;
+ NTCredentials proxyCredentials = null;
if (key.hasProxy) {
HttpHost proxy = new HttpHost(key.proxyHost, key.proxyPort, key.proxyScheme);
builder.setProxy(proxy);
CredentialsProvider credsProvider = new BasicCredentialsProvider();
if (!key.proxyUser.isEmpty()) {
+ proxyAuthScope = new AuthScope(key.proxyHost, key.proxyPort);
+ proxyCredentials = new NTCredentials(key.proxyUser, key.proxyPass, LOCALHOST, PROXY_DOMAIN);
credsProvider.setCredentials(
- new AuthScope(key.proxyHost, key.proxyPort),
- new NTCredentials(key.proxyUser, key.proxyPass, LOCALHOST, PROXY_DOMAIN));
+ proxyAuthScope,
+ proxyCredentials);
}
builder.setDefaultCredentialsProvider(credsProvider);
}
builder.disableContentCompression().addInterceptorLast(RESPONSE_CONTENT_ENCODING);
if(BASIC_AUTH_PREEMPTIVE) {
builder.addInterceptorFirst(PREEMPTIVE_AUTH_INTERCEPTOR);
+ } else {
+ builder.setDefaultCredentialsProvider(new ManagedCredentialsProvider(getAuthManager(), proxyAuthScope, proxyCredentials));
}
httpClient = builder.build();
if (log.isDebugEnabled()) { |
|
@pmouawad (migrated from Bugzilla):
Hello Felix, |
|
@FSchumacher (migrated from Bugzilla):
But I will commit it, as release time is near and a fix is a fix. |
|
@FSchumacher (migrated from Bugzilla): commit 740a206 .../jmeter/protocol/http/sampler/HTTPHC4Impl.java | 86 +++++++++++++++++++++- |
|
Peter Lynch (migrated from Bugzilla): I tried latest snapshot with the test case attached to this bug report. Initially it didn't work - looks like this was the logged cause: 2020-05-10 08:13:49,560 DEBUG o.a.j.p.h.s.HTTPHC4Impl: Invalid URL http in authManager I have "http" as the Base URL in HTTP Authorization Manager - which I believe is a valid value according to the HTTP Authorization manager docs, but the patched code rejects it: 740a206#diff-98b4f752302231261a91b5c0ee96ea9dR264 Once I changed the Base URL value in the HTTP Authorization Manager to a parseable URL object (https://httpbin.org) , then the test case passes. So while the specific problem seems to have been improved - I am concerned the new patch makes the incorrect assumption that the Authorization.getURL() method returns a proper URL in all cases. This would change the semantics used to match Base URL for Auth documented for the the HTTP Authorization Manager. https://jmeter.apache.org/usermanual/component_reference.html#HTTP_Authorization_Manager states "Base URL" contains "A partial or complete URL". So it seems the patch would need to use similar startsWith matching logic as in I did not perform any other auth related tests given he above noted issue. |
|
@FSchumacher (migrated from Bugzilla):
I always understood the documentation to mean a valid URL, that is shortened on the path and not necessarily on the host name, but I get your point.
startsWith is not easily usable, as httpclient is not giving us information about the used protocol. We could guess it and try all combinations (http and https), or try to get the hostname and the port from the URL by means of guessing. Both approaches have their weaknesses. Which one would you prefer? |
|
@FSchumacher (migrated from Bugzilla): Created attachment 01.diff: Try harder to match even partial URLs from AuthManager 01.diffdiff --git a/src/protocol/http/src/main/java/org/apache/jmeter/protocol/http/sampler/HTTPHC4Impl.java b/src/protocol/http/src/main/java/org/apache/jmeter/protocol/http/sampler/HTTPHC4Impl.java
index 4824ed6a70..fab565a59b 100644
--- a/src/protocol/http/src/main/java/org/apache/jmeter/protocol/http/sampler/HTTPHC4Impl.java
+++ b/src/protocol/http/src/main/java/org/apache/jmeter/protocol/http/sampler/HTTPHC4Impl.java
@@ -255,19 +255,32 @@ public class HTTPHC4Impl extends HTTPHCAbstractImpl {
if (!authScope.getRealm().equals(auth.getRealm())) {
continue;
}
- try {
- URL authUrl = new URL(auth.getURL());
- if (authUrl.getHost().equals(authScope.getHost()) && getPort(authUrl) == authScope.getPort()) {
- return auth;
- }
- } catch (MalformedURLException e) {
- log.debug("Invalid URL {} in authManager", auth.getURL());
+ if (matchesOnUrl(authScope, auth)) {
+ return auth;
}
}
}
return null;
}
+ private boolean matchesOnUrl(AuthScope authScope, Authorization auth) {
+ String authUrlString = auth.getURL().toLowerCase();
+ try {
+ URL authUrl = new URL(authUrlString);
+ if (authUrl.getHost().equals(authScope.getHost()) && getPort(authUrl) == authScope.getPort()) {
+ return true;
+ }
+ } catch (MalformedURLException e) {
+ for (String prefix: Arrays.asList("http://", "https://")) {
+ if ((prefix + authScope.getHost() + ":" + authScope.getPort()).startsWith(authUrlString)) {
+ return true;
+ }
+ }
+ log.debug("Invalid URL {} in authManager", authUrlString);
+ }
+ return false;
+ }
+
private int getPort(URL url) {
if (url.getPort() == -1) {
return url.getProtocol().equals("https") ? 443 : 80; |
|
@pmouawad (migrated from Bugzilla):
Hello Peter, Thank you |
|
Peter Lynch (migrated from Bugzilla): |
Peter Lynch (Bug 64267):
This page: https://jmeter.apache.org/usermanual/component_reference.html#HTTP_Authorization_Manager
States:
"The HttpComponents (HC 4.5.X) implementation defaults to pre-emptive since 3.2 and the header will be shown. To disable this, set the values as below, in which case authentication will only be performed in response to a challenge.
In the file jmeter.properties set httpclient4.auth.preemptive=false
"
This statement suggests to me that when the property is set to false, that the HTTP sampler using HC will automatically respond to an auth challenge if it has access to an HTTP Authorization Manager with appropriate credentials.
The attached test case and jmeter.log suggests this is not the case. I added a Response Assertion with Ignore Status enabled checking for a 200 response code. The assertion fails because 401 is received and the logs show no challenge response is automatically sent.
Expected:
When httpclient4.auth.preemptive=false and HC HTTP Sampler is used, then HTTP sampler should automatically respond to an Auth challenge if credentials are present via an HTTP Authorization Manager. Subsequently a response assertion testing for status code, should be able to test against the response code of the second request sent by the sampler with its credentials - instead of the first response challenge (401).
I realize there might be use cases the verify a challenge response code/headers/etc. and I don't wish to remove that ability. But I do believe the automatic challenge request by the HTTP Sampler also worked at one point in jmeter history, thus I filed this as bug, not enhancement.
Created attachment basic-auth-challenge-test.jmx: reproduce test case
basic-auth-challenge-test.jmx
OS: All
The text was updated successfully, but these errors were encountered: