HTTP Proxy Server throws an exception when path contains "|" character #2978
Comments
|
Marek (migrated from Bugzilla): Anyway I will report problem to application owners. |
|
@pmouawad (migrated from Bugzilla): All unsafe characters must always be encoded within a URL." |
|
@pmouawad (migrated from Bugzilla): |
|
Marek (migrated from Bugzilla): I understand that this character is classified as "unwise" ("unsafe"), and all new web application must not use it.
I can understand that you have different opinion and that is why decided to mark report as "WONTFIX", but can you at least give me a general hint how/where can I fix it? This way I could create my own private branch of JMeter. |
|
Marek (migrated from Bugzilla): I've encounter this problem when using Fire Fox. |
|
@pmouawad (migrated from Bugzilla): Regarding your previous comment, we usually do what you describe and tend to help as much as possible users with workarounds. Regards |
|
Marek (migrated from Bugzilla): |
|
Sebb (migrated from Bugzilla): However Opera, Chrome and IE do encode the "|" character when entered in the location field. == The Proxy intercepts the outgoing request, which should already have been encoded. In general it's not possible to safely re-encode a URL, because the encoding process introduces % characters which themselves need encoding. For example /| should be presented as /%7C. If this is re-encoded, it will encode the % again. |
|
Marek (migrated from Bugzilla): HTTPHC4Impl.java.patchIndex: src/protocol/http/org/apache/jmeter/protocol/http/sampler/HTTPHC4Impl.java
===================================================================
--- src/protocol/http/org/apache/jmeter/protocol/http/sampler/HTTPHC4Impl.java (revision 1418369)
+++ src/protocol/http/org/apache/jmeter/protocol/http/sampler/HTTPHC4Impl.java (working copy)
@@ -231,7 +231,14 @@
HttpRequestBase httpRequest = null;
try {
- URI uri = url.toURI();
+ URI uri = new URI(url.getProtocol(),
+ null /*userInfo*/,
+ url.getHost(),
+ url.getPort(),
+ url.getPath(),
+ url.getQuery(),
+ null /*fragment*/);
+
if (method.equals(HTTPConstants.POST)) {
httpRequest = new HttpPost(uri);
} else if (method.equals(HTTPConstants.PUT)) {
@@ -249,7 +256,10 @@
} else if (method.equals(HTTPConstants.PATCH)) {
httpRequest = new HttpPatch(uri);
} else {
- throw new IllegalArgumentException("Unexpected method: "+method);
+ throw new IllegalArgumentException(String.format("Unexpected method: \"%s\" for url: \"%s\" frameDepth=%d",
+ method,
+ url.toString(),
+ frameDepth));
}
setupRequest(url, httpRequest, res); // can throw IOException
} catch (Exception e) { |
|
Marek (migrated from Bugzilla): I've included patch which fixes this issue. BR, Marek Ruszczak |
|
Marek (migrated from Bugzilla): I've noticed that there is problem with encoding url. I've notice that this also can be fixed in: org.apache.jmeter.protocol.http.sampler.HTTPSamplerBase.getUrl BR, Marek Created attachment HTTPHC4Impl.java.patch: Corrected patch - double encoding problem HTTPHC4Impl.java.patchIndex: src/protocol/http/org/apache/jmeter/protocol/http/sampler/HTTPHC4Impl.java
===================================================================
--- src/protocol/http/org/apache/jmeter/protocol/http/sampler/HTTPHC4Impl.java (revision 1418369)
+++ src/protocol/http/org/apache/jmeter/protocol/http/sampler/HTTPHC4Impl.java (working copy)
@@ -231,7 +231,21 @@
HttpRequestBase httpRequest = null;
try {
- URI uri = url.toURI();
+ String urlContentEncoding = getContentEncodingOrNull();
+ if(urlContentEncoding == null || urlContentEncoding.length() == 0) {
+ // Use the default encoding for urls
+ urlContentEncoding = EncoderCache.URL_ARGUMENT_ENCODING;
+ }
+
+ URI uri = new URI(url.getProtocol(),
+ null /*userInfo*/,
+ url.getHost(),
+ url.getPort(),
+ // path and query might be already encoded so to prevent double encoding first it is decoded
+ URLDecoder.decode(url.getPath(), urlContentEncoding),
+ URLDecoder.decode(url.getQuery(), urlContentEncoding),
+ null /*fragment*/);
+
if (method.equals(HTTPConstants.POST)) {
httpRequest = new HttpPost(uri);
} else if (method.equals(HTTPConstants.PUT)) {
@@ -249,7 +263,10 @@
} else if (method.equals(HTTPConstants.PATCH)) {
httpRequest = new HttpPatch(uri);
} else {
- throw new IllegalArgumentException("Unexpected method: "+method);
+ throw new IllegalArgumentException(String.format("Unexpected method: \"%s\" for url: \"%s\" frameDepth=%d",
+ method,
+ url.toString(),
+ frameDepth));
}
setupRequest(url, httpRequest, res); // can throw IOException
} catch (Exception e) { |
|
Sebb (migrated from Bugzilla): However the patch affects all usage of the HC4 implementation. I think this is wrong:
Any patch that is applied needs to be for the Proxy Server only. But I do agree that decoding and then re-encoding the URL should avoid the double-encoding issue as mentioned in comment 8. It would be sensible to check if the URL is valid before attempting to fix it, rather than unconditionally "fixing" each URL. This would avoid unnecessary conversions possibly changing the URL. This would be slightly more work for the invalid case - but the work is done by the Proxy Server, not as part of a test - and is the minimal work required to work round what is a browser bug. |
|
@pmouawad (migrated from Bugzilla): |
|
@pmouawad (migrated from Bugzilla): |
|
@pmouawad (migrated from Bugzilla): URL: http://svn.apache.org/viewvc?rev=1442395&view=rev Modified: |
|
Marek (migrated from Bugzilla): BR, Marek PS. I'm not sure if I should change status to VERIFIED/FIXED so I leave it as it is. |
|
@pmouawad (migrated from Bugzilla): |
|
@pmouawad (migrated from Bugzilla): URL: http://svn.apache.org/r1467074 Modified: |
|
@pmouawad (migrated from Bugzilla): My issue with this patch is that it fixes the issues for HttpClient3 and 4 but it also fixes for Java which could break Created attachment HttpRequestHdr.java.patch: Patch for issue fixing unsafe URLs during parsing HttpRequestHdr.java.patchIndex: src/protocol/http/org/apache/jmeter/protocol/http/proxy/HttpRequestHdr.java
===================================================================
--- src/protocol/http/org/apache/jmeter/protocol/http/proxy/HttpRequestHdr.java (revision 1509948)
+++ src/protocol/http/org/apache/jmeter/protocol/http/proxy/HttpRequestHdr.java (working copy)
@@ -21,7 +21,9 @@
import java.io.ByteArrayOutputStream;
import java.io.IOException;
import java.io.InputStream;
+import java.net.URI;
import java.net.URL;
+import java.net.URLDecoder;
import java.util.HashMap;
import java.util.Map;
import java.util.StringTokenizer;
@@ -167,9 +169,31 @@
if (url.startsWith("/")) {
url = HTTPS + "://" + paramHttps + url; // $NON-NLS-1$
}
+ try {
+ URI testCleanUri = new URI(url);
+ } catch (Exception e) {
+ log.info("Url contains unsafe characters");
+ try {
+ url = escapeIllegalURLCharacters(url);
+ } catch (Exception e1) {
+ log.error("Error sanitzing URL:"+url);
+ }
+ }
log.debug("First Line: " + url);
}
+ /**
+ * @param url
+ * @return
+ * @throws Exception
+ */
+ public static String escapeIllegalURLCharacters(String url) throws Exception{
+ String decodeUrl = URLDecoder.decode(url,"UTF-8");
+ URL urlString = new URL(decodeUrl);
+ URI uri = new URI(urlString.getProtocol(), urlString.getUserInfo(), urlString.getHost(), urlString.getPort(), urlString.getPath(), urlString.getQuery(), urlString.getRef());
+ return uri.toString();
+ }
+
/*
* Split line into name/value pairs and store in headers if relevant
* If name = "content-length", then return value as int, else return 0 |
|
Sebb (migrated from Bugzilla):
Seems OK.
Not sure I follow. The purpose of this patch is to fix up URLs generated by browsers that don't encode all unsafe characters. It should be equivalent to using a well-behaved browser currently. |
|
@pmouawad (migrated from Bugzilla): URL: http://svn.apache.org/r1511503 Modified: |
Marek (Bug 54142):
I've have some application to test which is working fine, but during recording process with JMeter when I select some link, proxy rises an exception (visible only in web browser):
java.net.URISyntaxException: Illegal character in path at index 51: http://10.133.47.78:8080/comarch-cm/cm/showAccounts|C|21737.treeGroupNode|G|21731.treeGroupNode|G|21691.?clickContext=tree
at java.net.URI$Parser.fail(Unknown Source)
at java.net.URI$Parser.checkChars(Unknown Source)
at java.net.URI$Parser.parseHierarchical(Unknown Source)
at java.net.URI$Parser.parse(Unknown Source)
at java.net.URI.<init>(Unknown Source)
at java.net.URL.toURI(Unknown Source)
at org.apache.jmeter.protocol.http.sampler.HTTPHC4Impl.sample(HTTPHC4Impl.java:232)
at org.apache.jmeter.protocol.http.sampler.HTTPSamplerProxy.sample(HTTPSamplerProxy.java:62)
at org.apache.jmeter.protocol.http.sampler.HTTPSamplerBase.sample(HTTPSamplerBase.java:1075)
at org.apache.jmeter.protocol.http.proxy.Proxy.run(Proxy.java:212)
I'm including a short version of recording.
Below is the content of problematic GET request from "View Results Tree" attached to "HTTP Proxy Server" (HTTP view):
Method GET
Protocol http
Host 10.133.47.78
Port 8080
Path /comarch-cm/cm/showAccounts|C|21737.treeGroupNode|G|21731.treeGroupNode|G|21691.
Looks like "|" character character is problematic here.
Created attachment bugReportRecord.jmx: Result of recording when exception was raised
bugReportRecord.jmx
Votes in Bugzilla: 1
Severity: major
OS: All
Depends on:
The text was updated successfully, but these errors were encountered: