[PATCH]ResponseAssertion and HttpSampler don't work for national charsets #1624

asfimport · 2005-10-18T11:41:16Z

Henryk Paluch (Bug 37140):
Hi!
When attempting to use ReponseAssertion to search for national characters (in
UTF-8) found two bugs that prevent it:

HttpSampler forget to copy contentType and cataEncoding in case of following
redirects
ResponseAssertion converts byte[] to String using just default constructor
(thus converting characters using Java system default encoding) - wich is wrong.
Above patch fixes both issues.

Severity: normal
OS: other

asfimport · 2005-10-18T11:42:15Z

Henryk Paluch (migrated from Bugzilla):
Patch to fix bugs mentioned in description.

Created attachment jmeter_i18n_assertion.patch: jmeter_i18n_assertion.patch

jmeter_i18n_assertion.patch

diff -ur jakarta-jmeter-2.0.3-orig/src/components/org/apache/jmeter/assertions/ResponseAssertion.java jakarta-jmeter-2.0.3-src/src/components/org/apache/jmeter/assertions/ResponseAssertion.java
--- jakarta-jmeter-2.0.3-orig/src/components/org/apache/jmeter/assertions/ResponseAssertion.java	2004-06-11 13:04:46.000000000 +0200
+++ jakarta-jmeter-2.0.3-src/src/components/org/apache/jmeter/assertions/ResponseAssertion.java	2005-10-18 11:25:34.000000000 +0200
@@ -18,6 +18,7 @@
 
 package org.apache.jmeter.assertions;
 import java.io.Serializable;
+import java.io.UnsupportedEncodingException;
 import java.util.ArrayList;
 
 import org.apache.jmeter.samplers.SampleResult;
@@ -34,6 +35,8 @@
 import org.apache.oro.text.regex.Pattern;
 import org.apache.oro.text.regex.Perl5Compiler;
 import org.apache.oro.text.regex.Perl5Matcher;
+import org.apache.jorphan.logging.LoggingManager;
+import org.apache.log.Logger; 
 
 /**
  *
@@ -43,6 +46,8 @@
    extends AbstractTestElement
    implements Serializable, Assertion
 {
+   transient private static Logger log = LoggingManager.getLoggerForClass();
+ 
    public final static String TEST_FIELD = "Assertion.test_field";
    // Values for TEST_FIELD
    public final static String SAMPLE_LABEL = "Assertion.sample_label";
@@ -263,7 +268,29 @@
       // What are we testing against?
       if (ResponseAssertion.RESPONSE_DATA.equals(getTestField()))
       {
-		toCheck = new StringBuffer(response.getResponseHeaders()).append(new String(response.responseDataAsBA())).toString();
+           String responseData = null;
+	   if ( response.getContentType()!=null 
+			   && response.getContentType().startsWith("text/") ){
+		   if ( log.isDebugEnabled()){
+			   log.debug("attempting to use charset '"+response.getDataEncoding()+"' for contentType '"+response.getContentType()+"' of "+response.getURL());
+		   }
+		   try{
+		      // use supplied encoding from ContentType
+		      responseData = new String(response.responseDataAsBA(),response.getDataEncoding());
+		   }catch(UnsupportedEncodingException ex){
+			   if ( log.isErrorEnabled){
+				   log.error("Error "+ex.getMessage()+" while converting charset '"+response.getDataEncoding()+"' for contentType '"+response.getContentType()+"' of "+response.getURL()+". Falling back to system encoding.");
+			   }
+		   }
+	   }
+
+	   
+	   if (responseData == null) {
+		   // fallback to old behaviour
+		   responseData = new String(response.responseDataAsBA());
+	   }
+
+		toCheck = new StringBuffer(response.getResponseHeaders()).append(responseData).toString();
       }
       else if (ResponseAssertion.RESPONSE_CODE.equals(getTestField()))
       {
Only in jakarta-jmeter-2.0.3-src/src/core/org/apache/jmeter/util: catalog
diff -ur jakarta-jmeter-2.0.3-orig/src/protocol/http/org/apache/jmeter/protocol/http/sampler/HTTPSampler.java jakarta-jmeter-2.0.3-src/src/protocol/http/org/apache/jmeter/protocol/http/sampler/HTTPSampler.java
--- jakarta-jmeter-2.0.3-orig/src/protocol/http/org/apache/jmeter/protocol/http/sampler/HTTPSampler.java	2004-10-03 16:23:30.000000000 +0200
+++ jakarta-jmeter-2.0.3-src/src/protocol/http/org/apache/jmeter/protocol/http/sampler/HTTPSampler.java	2005-10-18 11:31:12.000000000 +0200
@@ -675,6 +675,9 @@
         totalRes.setResponseMessage(lastRes.getResponseMessage());
         totalRes.setDataType(lastRes.getDataType());
         totalRes.setResponseHeaders(lastRes.getResponseHeaders());
+        totalRes.setContentType(lastRes.getContentType());
+        totalRes.setDataEncoding(lastRes.getDataEncoding());
+
         return totalRes;
     }

asfimport · 2005-10-18T12:12:04Z

Henryk Paluch (migrated from Bugzilla):
Hi!
Found more problematic objects (for example RegexFunction does use
String(byte[]) again). Working on better patch. I have an idea to add new method
SampleResult.getResponseDataAsString() that would be handy for proper charset
conversion (as does that patch for ResponseAssertion now).

asfimport · 2005-10-18T13:41:52Z

Henryk Paluch (migrated from Bugzilla):
Hi!
Here is more advanced patch - it fixes charset handling for both
ResponseAssertion and RegexFunction. There is new convenience method
SampleResult.responseDataAsString() that does the trick.
Note: that patch works for me, but all daring testers are encouraged to try
this piece of code.

Created attachment jmeter_i18n_rev2.patch: jmeter_i18n_rev2.patch

jmeter_i18n_rev2.patch

diff -u -r jakarta-jmeter-2.0.3-orig/src/components/org/apache/jmeter/assertions/ResponseAssertion.java jakarta-jmeter-2.0.3-src/src/components/org/apache/jmeter/assertions/ResponseAssertion.java
--- jakarta-jmeter-2.0.3-orig/src/components/org/apache/jmeter/assertions/ResponseAssertion.java	2004-06-11 13:04:46.000000000 +0200
+++ jakarta-jmeter-2.0.3-src/src/components/org/apache/jmeter/assertions/ResponseAssertion.java	2005-10-18 13:35:03.000000000 +0200
@@ -263,7 +263,8 @@
       // What are we testing against?
       if (ResponseAssertion.RESPONSE_DATA.equals(getTestField()))
       {
-		toCheck = new StringBuffer(response.getResponseHeaders()).append(new String(response.responseDataAsBA())).toString();
+
+		toCheck = new StringBuffer(response.getResponseHeaders()).append(response.responseDataAsString()).toString();
       }
       else if (ResponseAssertion.RESPONSE_CODE.equals(getTestField()))
       {
Only in jakarta-jmeter-2.0.3-src/src/core/org/apache/jmeter: NewDriver.java
diff -u -r jakarta-jmeter-2.0.3-orig/src/core/org/apache/jmeter/samplers/SampleResult.java jakarta-jmeter-2.0.3-src/src/core/org/apache/jmeter/samplers/SampleResult.java
--- jakarta-jmeter-2.0.3-orig/src/core/org/apache/jmeter/samplers/SampleResult.java	2005-03-12 11:38:40.000000000 +0100
+++ jakarta-jmeter-2.0.3-src/src/core/org/apache/jmeter/samplers/SampleResult.java	2005-10-18 13:21:31.000000000 +0200
@@ -20,6 +20,7 @@
 
 import java.io.Serializable;
 import java.io.StringWriter;
+import java.io.UnsupportedEncodingException;
 import java.net.URL;
 import java.util.ArrayList;
 import java.util.HashSet;
@@ -369,6 +370,39 @@
 	{
 		return responseData == null ? EMPTY_BA : responseData;
 	}
+	
+	/**
+	 * Convenience method to get responseData String that
+	 * cares character set of response.
+	 * 
+	 * @return the responseData. If responseData is null
+	 * then an empty String is returned
+	 * 
+	 */
+	public String responseDataAsString()
+	{
+       String str = null;
+ 	   if ( getContentType()!=null 
+ 			   && getContentType().startsWith("text/") ){
+ 		   if ( log.isDebugEnabled()){
+ 			   log.debug("attempting to use charset '"+getDataEncoding()+"' for contentType '"+getContentType()+"' of "+getURL());
+ 		   }
+ 		   try{
+ 		      // use supplied encoding from ContentType
+ 		      str = new String(responseDataAsBA(),getDataEncoding());
+ 		   }catch(UnsupportedEncodingException ex){
+ 			   if ( log.isErrorEnabled()){
+ 				   log.error("Error "+ex.getMessage()+" while converting charset '"+getDataEncoding()+"' for contentType '"+getContentType()+"' of "+getURL()+". Falling back to system encoding.");
+ 			   }
+ 		   }
+ 	   }
+ 	   
+ 	   if (str == null) {
+ 		   // fallback to old behaviour
+ 		   str = new String(responseDataAsBA());
+ 	   }
+ 	   return str;
+	}
 
     public void setSamplerData(String s)
     {
Only in jakarta-jmeter-2.0.3-src/src/core/org/apache/jmeter/util: JMeterVersion.java
Only in jakarta-jmeter-2.0.3-src/src/core/org/apache/jmeter/util: XPathUtil.java
Only in jakarta-jmeter-2.0.3-src/src/core/org/apache/jmeter/util: catalog
diff -u -r jakarta-jmeter-2.0.3-orig/src/functions/org/apache/jmeter/functions/RegexFunction.java jakarta-jmeter-2.0.3-src/src/functions/org/apache/jmeter/functions/RegexFunction.java
--- jakarta-jmeter-2.0.3-orig/src/functions/org/apache/jmeter/functions/RegexFunction.java	2004-12-11 12:12:18.000000000 +0100
+++ jakarta-jmeter-2.0.3-src/src/functions/org/apache/jmeter/functions/RegexFunction.java	2005-10-18 13:24:42.000000000 +0200
@@ -143,7 +143,7 @@
         try
         {
             PatternMatcher matcher = (PatternMatcher) localMatcher.get();
-            String responseText = new String(previousResult.getResponseData());
+            String responseText = previousResult.responseDataAsString();
             PatternMatcherInput input = new PatternMatcherInput(responseText);
             while (matcher.contains(input, searchPattern))
             {
diff -u -r jakarta-jmeter-2.0.3-orig/src/protocol/http/org/apache/jmeter/protocol/http/sampler/HTTPSampler.java jakarta-jmeter-2.0.3-src/src/protocol/http/org/apache/jmeter/protocol/http/sampler/HTTPSampler.java
--- jakarta-jmeter-2.0.3-orig/src/protocol/http/org/apache/jmeter/protocol/http/sampler/HTTPSampler.java	2004-10-03 16:23:30.000000000 +0200
+++ jakarta-jmeter-2.0.3-src/src/protocol/http/org/apache/jmeter/protocol/http/sampler/HTTPSampler.java	2005-10-18 13:32:12.000000000 +0200
@@ -675,6 +675,9 @@
         totalRes.setResponseMessage(lastRes.getResponseMessage());
         totalRes.setDataType(lastRes.getDataType());
         totalRes.setResponseHeaders(lastRes.getResponseHeaders());
+        totalRes.setContentType(lastRes.getContentType());
+        totalRes.setDataEncoding(lastRes.getDataEncoding());
+
         return totalRes;
     }

asfimport · 2005-10-18T13:48:19Z

Henryk Paluch (migrated from Bugzilla):
Created attachment jmeter_i18n_rev3.patch: jmeter_i18n_rev3.patch - just fixed rogue "Only in..." diff messages

jmeter_i18n_rev3.patch

diff -u -r jakarta-jmeter-2.0.3-orig/src/components/org/apache/jmeter/assertions/ResponseAssertion.java jakarta-jmeter-2.0.3-src/src/components/org/apache/jmeter/assertions/ResponseAssertion.java
--- jakarta-jmeter-2.0.3-orig/src/components/org/apache/jmeter/assertions/ResponseAssertion.java	2004-06-11 13:04:46.000000000 +0200
+++ jakarta-jmeter-2.0.3-src/src/components/org/apache/jmeter/assertions/ResponseAssertion.java	2005-10-18 13:35:03.000000000 +0200
@@ -263,7 +263,8 @@
       // What are we testing against?
       if (ResponseAssertion.RESPONSE_DATA.equals(getTestField()))
       {
-		toCheck = new StringBuffer(response.getResponseHeaders()).append(new String(response.responseDataAsBA())).toString();
+
+		toCheck = new StringBuffer(response.getResponseHeaders()).append(response.responseDataAsString()).toString();
       }
       else if (ResponseAssertion.RESPONSE_CODE.equals(getTestField()))
       {
diff -u -r jakarta-jmeter-2.0.3-orig/src/core/org/apache/jmeter/samplers/SampleResult.java jakarta-jmeter-2.0.3-src/src/core/org/apache/jmeter/samplers/SampleResult.java
--- jakarta-jmeter-2.0.3-orig/src/core/org/apache/jmeter/samplers/SampleResult.java	2005-03-12 11:38:40.000000000 +0100
+++ jakarta-jmeter-2.0.3-src/src/core/org/apache/jmeter/samplers/SampleResult.java	2005-10-18 13:21:31.000000000 +0200
@@ -20,6 +20,7 @@
 
 import java.io.Serializable;
 import java.io.StringWriter;
+import java.io.UnsupportedEncodingException;
 import java.net.URL;
 import java.util.ArrayList;
 import java.util.HashSet;
@@ -369,6 +370,39 @@
 	{
 		return responseData == null ? EMPTY_BA : responseData;
 	}
+	
+	/**
+	 * Convenience method to get responseData String that
+	 * cares character set of response.
+	 * 
+	 * @return the responseData. If responseData is null
+	 * then an empty String is returned
+	 * 
+	 */
+	public String responseDataAsString()
+	{
+       String str = null;
+ 	   if ( getContentType()!=null 
+ 			   && getContentType().startsWith("text/") ){
+ 		   if ( log.isDebugEnabled()){
+ 			   log.debug("attempting to use charset '"+getDataEncoding()+"' for contentType '"+getContentType()+"' of "+getURL());
+ 		   }
+ 		   try{
+ 		      // use supplied encoding from ContentType
+ 		      str = new String(responseDataAsBA(),getDataEncoding());
+ 		   }catch(UnsupportedEncodingException ex){
+ 			   if ( log.isErrorEnabled()){
+ 				   log.error("Error "+ex.getMessage()+" while converting charset '"+getDataEncoding()+"' for contentType '"+getContentType()+"' of "+getURL()+". Falling back to system encoding.");
+ 			   }
+ 		   }
+ 	   }
+ 	   
+ 	   if (str == null) {
+ 		   // fallback to old behaviour
+ 		   str = new String(responseDataAsBA());
+ 	   }
+ 	   return str;
+	}
 
     public void setSamplerData(String s)
     {
diff -u -r jakarta-jmeter-2.0.3-orig/src/functions/org/apache/jmeter/functions/RegexFunction.java jakarta-jmeter-2.0.3-src/src/functions/org/apache/jmeter/functions/RegexFunction.java
--- jakarta-jmeter-2.0.3-orig/src/functions/org/apache/jmeter/functions/RegexFunction.java	2004-12-11 12:12:18.000000000 +0100
+++ jakarta-jmeter-2.0.3-src/src/functions/org/apache/jmeter/functions/RegexFunction.java	2005-10-18 13:24:42.000000000 +0200
@@ -143,7 +143,7 @@
         try
         {
             PatternMatcher matcher = (PatternMatcher) localMatcher.get();
-            String responseText = new String(previousResult.getResponseData());
+            String responseText = previousResult.responseDataAsString();
             PatternMatcherInput input = new PatternMatcherInput(responseText);
             while (matcher.contains(input, searchPattern))
             {
diff -u -r jakarta-jmeter-2.0.3-orig/src/protocol/http/org/apache/jmeter/protocol/http/sampler/HTTPSampler.java jakarta-jmeter-2.0.3-src/src/protocol/http/org/apache/jmeter/protocol/http/sampler/HTTPSampler.java
--- jakarta-jmeter-2.0.3-orig/src/protocol/http/org/apache/jmeter/protocol/http/sampler/HTTPSampler.java	2004-10-03 16:23:30.000000000 +0200
+++ jakarta-jmeter-2.0.3-src/src/protocol/http/org/apache/jmeter/protocol/http/sampler/HTTPSampler.java	2005-10-18 13:32:12.000000000 +0200
@@ -675,6 +675,9 @@
         totalRes.setResponseMessage(lastRes.getResponseMessage());
         totalRes.setDataType(lastRes.getDataType());
         totalRes.setResponseHeaders(lastRes.getResponseHeaders());
+        totalRes.setContentType(lastRes.getContentType());
+        totalRes.setDataEncoding(lastRes.getDataEncoding());
+
         return totalRes;
     }

asfimport · 2005-10-18T15:21:31Z

peter lin (migrated from Bugzilla):
thanks for the information. I'll try to get to it tonight if I have time.

peter lin

asfimport · 2005-10-20T02:15:46Z

Sebb (migrated from Bugzilla):
Thanks.

The Response Assertion problem was already fixed in #1258.

Other fixes applied (though I did not add the convenience function).

asfimport · 2005-10-20T05:13:23Z

peter lin (migrated from Bugzilla):
I've added your name to the contributer page, thanks again for submitting the patch.

http://wiki.apache.org/jakarta-jmeter/JMeterCommitters?action=show

peter lin

asfimport · 2005-10-20T14:42:56Z

Henryk Paluch (migrated from Bugzilla):
Hi!
Tested rel-2-1 branch - it works well. Closing that bug.
Thank you!

asfimport closed this as completed Oct 20, 2005

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[PATCH]ResponseAssertion and HttpSampler don't work for national charsets #1624

[PATCH]ResponseAssertion and HttpSampler don't work for national charsets #1624

asfimport commented Oct 18, 2005

asfimport commented Oct 18, 2005

asfimport commented Oct 18, 2005

asfimport commented Oct 18, 2005

asfimport commented Oct 18, 2005

asfimport commented Oct 18, 2005

asfimport commented Oct 20, 2005

asfimport commented Oct 20, 2005

asfimport commented Oct 20, 2005

[PATCH]ResponseAssertion and HttpSampler don't work for national charsets #1624

[PATCH]ResponseAssertion and HttpSampler don't work for national charsets #1624

Comments

asfimport commented Oct 18, 2005

asfimport commented Oct 18, 2005

asfimport commented Oct 18, 2005

asfimport commented Oct 18, 2005

asfimport commented Oct 18, 2005

asfimport commented Oct 18, 2005

asfimport commented Oct 20, 2005

asfimport commented Oct 20, 2005

asfimport commented Oct 20, 2005