Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Timer : Add a factor to apply on pauses #4069

Closed
asfimport opened this issue Aug 18, 2016 · 12 comments
Closed

Timer : Add a factor to apply on pauses #4069

asfimport opened this issue Aug 18, 2016 · 12 comments

Comments

@asfimport
Copy link
Collaborator

UbikLoadPack support (Bug 60018):
This bug is related partly to #3809
Sometimes you have to adapt the timers values to be half or double what they are currently.

When you have configured timers at every place this can be cumbersome as you have to touch tens of Test Elements.

It would be nice to be able to say: Take current computed pauses and multiply them by a factor.

Any other idea is welcome

If you agree to this idea, we can submit a patch.

Regards
@ubikloadpack

OS: All

Duplicates:

@asfimport
Copy link
Collaborator Author

Sebb (migrated from Bugzilla):
I'm not sure that I see a genuine use case for this. Why would every single pause have to be doubled? That does not seem like something a genuine user would do.

Most (all?) the timers can take variables, so it's possible to implement this anyway.

@asfimport
Copy link
Collaborator Author

UbikLoadPack support (migrated from Bugzilla):
The use case is testing different throughputs easily.

This frequently happens at the beginning of a campaign when you're trying to find the good think times to reach some load and you don't want to increase threads.
Frankly I believe it's a very frequent use case.

Yes it can be done by using __BeanShell or __groovy to use a property containing factor but whenever you have different think times between the samplers (We use TestAction with pause time at 0 and its child is a Timer) in a realistic test plan, it becomes a lot of work and makes test plan harder to maintain.

It's not a big piece of code (1 line in JMeterThread#delay ) and it's only a matter of adding a property:

  • thinktime_factor (for example)

Regards

@asfimport
Copy link
Collaborator Author

benoit.wiart (migrated from Bugzilla):
As a "genuine" user that's something I frequently do.

I also think timers should be moved out of the sampler and become "first level" elements.

@asfimport
Copy link
Collaborator Author

Sebb (migrated from Bugzilla):
(In reply to UbikLoadPack support from comment 2)

The use case is testing different throughputs easily.

This frequently happens at the beginning of a campaign when you're trying to
find the good think times to reach some load and you don't want to increase
threads.
Frankly I believe it's a very frequent use case.

Yes it can be done by using __BeanShell or __groovy to use a property
containing factor but whenever you have different think times between the
samplers (We use TestAction with pause time at 0 and its child is a Timer)
in a realistic test plan, it becomes a lot of work and makes test plan
harder to maintain.

It's not a big piece of code (1 line in JMeterThread#delay ) and it's only a
matter of adding a property:

  • thinktime_factor (for example)

Also documentation, testing.

Every additional function increases user support costs somewhat.

Regards

@asfimport
Copy link
Collaborator Author

@pmouawad (migrated from Bugzilla):
(In reply to Sebb from comment 4)

(In reply to UbikLoadPack support from comment 2)
> The use case is testing different throughputs easily.
>
> This frequently happens at the beginning of a campaign when you're trying to
> find the good think times to reach some load and you don't want to increase
> threads.
> Frankly I believe it's a very frequent use case.
>
>
> Yes it can be done by using __BeanShell or __groovy to use a property
> containing factor but whenever you have different think times between the
> samplers (We use TestAction with pause time at 0 and its child is a Timer)
> in a realistic test plan, it becomes a lot of work and makes test plan
> harder to maintain.
>
> It's not a big piece of code (1 line in JMeterThread#delay ) and it's only a
> matter of adding a property:
> - thinktime_factor (for example)

Also documentation, testing.

Every additional function increases user support costs somewhat.

If this contributor contributes the full feature that they find useful, why would be refuse it ?

I don't share the reasoning regarding additional function.
If we follow it then we would stop any development on JMeter ?

>
> Regards

@asfimport
Copy link
Collaborator Author

Sebb (migrated from Bugzilla):
(In reply to Philippe Mouawad from comment 5)

(In reply to Sebb from comment 4)
> (In reply to UbikLoadPack support from comment 2)
> > It's not a big piece of code (1 line in JMeterThread#delay ) and it's only a
> > matter of adding a property:
> > - thinktime_factor (for example)
>
> Also documentation, testing.
>
> Every additional function increases user support costs somewhat.

I don't share the reasoning regarding additional function.

I was just pointing out that the amount of work involved is more than you wrote.

When deciding whether to implement a new feature, it's important to do a cost-benefit analysis. Ignoring some of the cost invalidates the analysis.

If we follow it then we would stop any development on JMeter ?

That does not follow from what I wrote.

@asfimport
Copy link
Collaborator Author

@vlsi (migrated from Bugzilla):
ULP> This frequently happens at the beginning of a campaign when you're trying to find the good think times to reach some load and you don't want to increase threads.
ULP> Frankly I believe it's a very frequent use case.

I think the following approach should be used:

  1. Use "Constant Throughput Timer" or alike to achieve the target load (number of iterations per hour)
  2. Set "number of threads" == "number of callcenter operators" (or other kind of persons working with the system).
  3. For open system (that is the case like "anybody from the internet can access the system" where the number of users is virtually unlimited), set "number of threads" == "expected maximal number of concurrent requests".

ULP>reach some load and you don't want to increase threads.

Suppose we are dealing with callcenter of 20 operators. We specify 20 threads.
If that is not enough to reach the target load, that just means "application is not responsive enough for real users to reach the desired load".
It is just a valid finding: "not enough operators to drive the load".
It is not something to be "cured" by "artificially decreasing think times".

Does it make sense?

@asfimport
Copy link
Collaborator Author

UbikLoadPack support (migrated from Bugzilla):
(In reply to Vladimir Sitnikov from comment 7)

ULP> This frequently happens at the beginning of a campaign when you're
trying to find the good think times to reach some load and you don't want to
increase threads.
ULP> Frankly I believe it's a very frequent use case.

I think the following approach should be used:

  1. Use "Constant Throughput Timer" or alike to achieve the target load
    (number of iterations per hour)

If you read http://jmeter.apache.org/usermanual/component_reference.html#Constant_Throughput_Timer,

Of course the throughput will be lower if the server is not capable of handling it, or if >>>>other timers<<<< or time-consuming test elements prevent it.

  1. Set "number of threads" == "number of callcenter operators" (or other
    kind of persons working with the system).
  2. For open system (that is the case like "anybody from the internet can
    access the system" where the number of users is virtually unlimited), set
    "number of threads" == "expected maximal number of concurrent requests".

ULP>reach some load and you don't want to increase threads.

Suppose we are dealing with callcenter of 20 operators. We specify 20
threads.
If that is not enough to reach the target load, that just means "application
is not responsive enough for real users to reach the desired load".
It is just a valid finding: "not enough operators to drive the load".
It is not something to be "cured" by "artificially decreasing think times".

Does it make sense?

Yes.
But if may be not enough because for example in an E-Commerce load test, you can use an approach where you model the average visit.
You are given pause times , you configure them, you compute a target number of Threads.
You make a first run and see that you don't reach the number of visits expected in an hour, that might be due to response times, but that can also be due to wrong Pauses.

Isn't it interesting to be able to validate the hypothesis by multiplying all pause times (except CTP (read below proposition)) by a double (not int) factor and test again ? instead of having to do that in scripting.

We could implement it this way:

  • Either we introduce a TimerConfig element or just a property
  • We add a method in Timer interface canBeMultiplied (a better name is to be found) , if it returns true, we can apply the factor, if not we use timer delay as is.
  • CTT would implement the method by returning false
  • Other timers would implement the method by returning true

Also, as noted by Antonio Gomes Rodrigues in mailing list, this feature exists in alternative products in the same field, so it must be useful no ?
At least in our experience it would be.

Regards

@asfimport
Copy link
Collaborator Author

UbikLoadPack support (migrated from Bugzilla):
Hello,
Find attached a patch implementing the following:

  • Add a method on Timer interface that allows it to tell wether a ratio can be applied on it
  • Implement it in subclasses
  • Add a new property timer.factor
  • Apply factor in JMeterThread

The patch is already usable like this and it was very useful during a load testing campaign this week to adjust timer and reach a behaviour similar to production.

Next step can be:

  • Add a new Timer Config element that controls timer.factor
  • Add a checkbox on all Timer GUIs to enable them to refuse adjustment

Regards
UbikLoadPack Team

Created attachment BUG_60018.patch: Patch implementing first part of the feature

BUG_60018.patch
Index: bin/jmeter.properties
===================================================================
--- bin/jmeter.properties	(revision 1759038)
+++ bin/jmeter.properties	(working copy)
@@ -1242,3 +1242,7 @@
 
 # Number of iterations to use to validate a Thread Group
 #testplan_validation.number_iterations=1
+
+# Timer factor that can be applied to Timer except for Constant ThroughputTimer and SynchronizingTimer
+# as long as any timer return false in  org.apache.jmeter.timers.Timer#isModifiable
+#timer.factor=1.0
\ No newline at end of file
Index: src/components/org/apache/jmeter/timers/BSFTimer.java
===================================================================
--- src/components/org/apache/jmeter/timers/BSFTimer.java	(revision 1754657)
+++ src/components/org/apache/jmeter/timers/BSFTimer.java	(working copy)
@@ -52,4 +52,9 @@
         }
         return delay;
     }
+    
+    @Override
+    public boolean isModifiable() {
+        return true;
+    }
 }
Index: src/components/org/apache/jmeter/timers/BeanShellTimer.java
===================================================================
--- src/components/org/apache/jmeter/timers/BeanShellTimer.java	(revision 1754657)
+++ src/components/org/apache/jmeter/timers/BeanShellTimer.java	(working copy)
@@ -62,4 +62,9 @@
             return 0;
         }
     }
+
+    @Override
+    public boolean isModifiable() {
+        return true;
+    }
 }
Index: src/components/org/apache/jmeter/timers/ConstantThroughputTimer.java
===================================================================
--- src/components/org/apache/jmeter/timers/ConstantThroughputTimer.java	(revision 1754657)
+++ src/components/org/apache/jmeter/timers/ConstantThroughputTimer.java	(working copy)
@@ -217,8 +217,8 @@
         //Synchronize on the info object's MUTEX to ensure
         //Multiple threads don't update the scheduled time simultaneously
         synchronized (info.MUTEX) {
-            final long nextRequstTime = info.lastScheduledTime + milliSecPerRequest;
-            info.lastScheduledTime = Math.max(now, nextRequstTime);
+            final long nextRequestTime = info.lastScheduledTime + milliSecPerRequest;
+            info.lastScheduledTime = Math.max(now, nextRequestTime);
             calculatedDelay = info.lastScheduledTime - now;
         }
 
@@ -329,5 +329,9 @@
     void setMode(Mode newMode) {
         mode = newMode;
     }
-
+    
+    @Override
+    public boolean isModifiable() {
+        return false;
+    }
 }
Index: src/components/org/apache/jmeter/timers/ConstantTimer.java
===================================================================
--- src/components/org/apache/jmeter/timers/ConstantTimer.java	(revision 1754657)
+++ src/components/org/apache/jmeter/timers/ConstantTimer.java	(working copy)
@@ -109,4 +109,9 @@
         delay = getPropertyAsLong(DELAY);
 
     }
+    
+    @Override
+    public boolean isModifiable() {
+        return true;
+    }
 }
Index: src/components/org/apache/jmeter/timers/JSR223Timer.java
===================================================================
--- src/components/org/apache/jmeter/timers/JSR223Timer.java	(revision 1754657)
+++ src/components/org/apache/jmeter/timers/JSR223Timer.java	(working copy)
@@ -50,4 +50,9 @@
         }
         return delay;
     }
+
+    @Override
+    public boolean isModifiable() {
+        return true;
+    }
 }
Index: src/components/org/apache/jmeter/timers/SyncTimer.java
===================================================================
--- src/components/org/apache/jmeter/timers/SyncTimer.java	(revision 1754657)
+++ src/components/org/apache/jmeter/timers/SyncTimer.java	(working copy)
@@ -275,4 +275,9 @@
     public void setTimeoutInMs(long timeoutInMs) {
         this.timeoutInMs = timeoutInMs;
     }
+
+    @Override
+    public boolean isModifiable() {
+        return false;
+    }
 }
Index: src/components/org/apache/jmeter/timers/UniformRandomTimer.java
===================================================================
--- src/components/org/apache/jmeter/timers/UniformRandomTimer.java	(revision 1754657)
+++ src/components/org/apache/jmeter/timers/UniformRandomTimer.java	(working copy)
@@ -20,7 +20,7 @@
 
 import java.io.Serializable;
 
-import org.apache.jmeter.util.JMeterUtils;
+import org.apache.commons.lang3.builder.ToStringBuilder;
 
 /**
  * This class implements those methods needed by RandomTimer to be instantiable
@@ -38,7 +38,7 @@
 
     @Override
     public String toString() {
-        return JMeterUtils.getResString("uniform_timer_memo"); //$NON-NLS-1$
+        //return JMeterUtils.getResString("uniform_timer_memo"); //$NON-NLS-1$
+        return ToStringBuilder.reflectionToString(this);
     }
-
 }
Index: src/core/org/apache/jmeter/threads/JMeterThread.java
===================================================================
--- src/core/org/apache/jmeter/threads/JMeterThread.java	(revision 1758348)
+++ src/core/org/apache/jmeter/threads/JMeterThread.java	(working copy)
@@ -75,6 +75,8 @@
     private static final int RAMPUP_GRANULARITY =
             JMeterUtils.getPropDefault("jmeterthread.rampup.granularity", 1000); // $NON-NLS-1$
 
+    private static final double TIMER_FACTOR = JMeterUtils.getPropDefault("timer.factor", 1.0d);
+
     private final Controller threadGroupLoopController;
 
     private final HashTree testTree;
@@ -795,7 +797,17 @@
         long totalDelay = 0;
         for (Timer timer : timers) {
             TestBeanHelper.prepare((TestElement) timer);
-            totalDelay += timer.delay();
+            long delay = timer.delay();
+            if(TIMER_FACTOR != 1.0d && timer.isModifiable()) {
+                if(log.isDebugEnabled()) {
+                    log.debug("Applying TIMER_FACTOR:"
+                            +TIMER_FACTOR + " on timer:"
+                            +((TestElement)timer).getName()
+                            + " for thread:"+getThreadName());
+                }
+                delay = Math.round(delay * TIMER_FACTOR);
+            }
+            totalDelay += delay;
         }
         if (totalDelay > 0) {
             try {
Index: src/core/org/apache/jmeter/timers/Timer.java
===================================================================
--- src/core/org/apache/jmeter/timers/Timer.java	(revision 1754657)
+++ src/core/org/apache/jmeter/timers/Timer.java	(working copy)
@@ -33,4 +33,9 @@
      * @return the computed delay value.
      */
     long delay();
+
+    /**
+     * @return true if factor can be applied to it
+     */
+    boolean isModifiable();
 }
Index: src/core/org/apache/jmeter/util/JMeterUtils.java
===================================================================
--- src/core/org/apache/jmeter/util/JMeterUtils.java	(revision 1756527)
+++ src/core/org/apache/jmeter/util/JMeterUtils.java	(working copy)
@@ -856,6 +856,26 @@
         }
         return ans;
     }
+    
+    /**
+     * Get a double value with default if not present.
+     *
+     * @param propName
+     *            the name of the property.
+     * @param defaultVal
+     *            the default value.
+     * @return The PropDefault value
+     */
+    public static double getPropDefault(String propName, double defaultVal) {
+        double ans;
+        try {
+            ans = Double.parseDouble(appProperties.getProperty(propName, Double.toString(defaultVal)).trim());
+        } catch (Exception e) {
+            log.warn("Exception '"+ e.getMessage()+ "' occurred when fetching double property:'"+propName+"', defaulting to:"+defaultVal);
+            ans = defaultVal;
+        }
+        return ans;
+    }
 
     /**
      * Get a String value with default if not present.

@asfimport
Copy link
Collaborator Author

@pmouawad (migrated from Bugzilla):
Author: pmouawad
Date: Tue Sep 20 12:28:37 2016
New Revision: 1761563

URL: http://svn.apache.org/viewvc?rev=1761563&view=rev
Log:
#4069 - Timer : Add a factor to apply on pauses
#4069

Added:
jmeter/trunk/src/core/org/apache/jmeter/timers/ModifiableTimer.java (with props)
Modified:
jmeter/trunk/bin/jmeter.properties
jmeter/trunk/src/components/org/apache/jmeter/timers/RandomTimer.java
jmeter/trunk/src/core/org/apache/jmeter/threads/JMeterThread.java
jmeter/trunk/src/core/org/apache/jmeter/util/JMeterUtils.java
jmeter/trunk/xdocs/changes.xml
jmeter/trunk/xdocs/usermanual/component_reference.xml

Author: pmouawad
Date: Tue Sep 20 12:30:19 2016
New Revision: 1761564

URL: http://svn.apache.org/viewvc?rev=1761564&view=rev
Log:
#4069 - Timer : Add a factor to apply on pauses
eol
#4069

Modified:
jmeter/trunk/src/core/org/apache/jmeter/timers/ModifiableTimer.java (props changed)

@asfimport
Copy link
Collaborator Author

@pmouawad (migrated from Bugzilla):
Author: pmouawad
Date: Sun Sep 25 16:28:59 2016
New Revision: 1762223

URL: http://svn.apache.org/viewvc?rev=1762223&view=rev
Log:
#4069 - Timer : Add a factor to apply on pauses
Document property
#4069

Modified:
jmeter/trunk/xdocs/usermanual/properties_reference.xml

@asfimport
Copy link
Collaborator Author

@pmouawad (migrated from Bugzilla):
Author: pmouawad
Date: Fri Dec 23 21:45:29 2016
New Revision: 1775911

URL: http://svn.apache.org/viewvc?rev=1775911&view=rev
Log:
Sonar : Fix errors, code smells and false-positive
Also use Timer.isModifiable() introduced within #4069
#4069

Modified:
jmeter/trunk/src/core/org/apache/jmeter/threads/JMeterThread.java

Author: pmouawad
Date: Fri Dec 23 21:32:13 2016
New Revision: 1775901

URL: http://svn.apache.org/viewvc?rev=1775901&view=rev
Log:
#4069 - Timer : Add a factor to apply on pauses
Use Java8 default method
#4069

Modified:
jmeter/trunk/src/components/org/apache/jmeter/timers/RandomTimer.java
jmeter/trunk/src/core/org/apache/jmeter/timers/ModifiableTimer.java
jmeter/trunk/src/core/org/apache/jmeter/timers/Timer.java

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant