Create this tool [bounty: $2200]

[robingustafsson]

Converting JMeter JMX files to k6 JS

View on BountySource

The goal of this project is to write a separate tool from k6 (similar to the existing Postman converter) that can convert JMeter .jmx files to k6 JS. The aim is to cover as many of the most common features in JMeter as possible to make the tool a great aid when migrating from JMeter to k6.

Requirements

The tool must be written in either JS or Go, which are the two languages used by k6.

The tool must support the following JMX/XML tags unless it turns out to be infeasible after discussion in this issue (the Component Reference will be very helpful here).:

Config

• hashTree
• jmeterTestPlan
• TestPlan
• ConfigTestElement
• kg.apc.jmeter.threads.SteppingThreadGroup
  • This should be converted to the equivalent stages option in k6 JS. Each step should be a { target: X, duration: Y } entry.
• ThreadGroup
  • This should be converted to the equivalent stages option in k6 JS ("Number of Threads" = number of VUs, aka the "target" in the stages option, "Ramp-up Period" = the "duration" of the ramp-up in the stages option). If more than one ThreadGroup has been defined they should be merged into one stages option (potentially with several { target: X, duration: Y } entries) and the correct distribution of VUs should be handled in the export default function using __VU and if/else JS statements.
  • The special case of a setUp ThreadGroup should be converted to a corresponding export function setup() in k6 JS.
  • The special case of a tearDown ThreadGroup should be converted to a corresponding export function teardown() in k6 JS.

Managers

• DNSCacheManager
  • The static host table part of this element should be converted to the corresponding hosts option in k6 JS.
• HeaderManager
  • This should be converted (all HTTPHeaderManagers should be merged into one) to a JS Object in global scope that can be used as the basis for the request headers of each http.get/post/put/patch/del/options/batch/request call in k6 JS.
• CookieManager
  • This should be converted to equivalent Cookie API calls in k6 JS.
• AuthManager
  • This should be converted to use the equivalent auth request parameter option for basic, digest and ntlm in k6 JS (setting the username and password in the URL itself).

Assertions (should be converted to k6 checks)

• com.atlantbh.jmeter.plugins.jsonutils.jsonpathassertion.JSONPathAssertion
• DurationAssertion
  • This should be converted to the equivalent check() entry looking at res.timings.duration.
• ResponseAssertion
  • This should be converted to the equivalent check() entry looking at res.body and res.headers.
• XPathAssertion (as we don’t have any native XPath APIs in k6 yet, this should be converted to a comment stating that "There's currently no XPath API in k6 so a pure JS solution has to be used. Try https://github.com/google/wicked-good-xpath.")

Logic controllers

• ForeachController
  • This should be converted to the equivalent JS for statement (or Array.forEach(function(...) { ... }); statement).
• GenericController
• IfController
  • This should be converted to the equivalent JS if statement using the specified condition, to control whether to run the child controllers or not.
• InterleaveControl
  • This should be converted to use something like a JS switch statement: switch (__ITER % NUM_OF_SUB_CONTOLLERS) { case 1: ... case 2: ... etc. } to alternative between the child controllers.
• LoopController
  • This should be converted into the equivalent JS for loop in k6 JS.
• OnceOnlyController
  • This should be converted to if (__ITER === 1) { ... } in k6 JS. If in a looping container it should also be restricted to only run on the first iteration.
• WhileController
  • This should be converted to the equivalent JS while statement using the specified condition.
• RunTimeController
  • This should be converted to use a global timestamp var, something like const TEST_START_TIME = Date().now();, and then in the script an if check like if (RUNTIME_CONTROLLER_SECS * 1000 > Date.now() - TEST_START_TIME) { ...execute child controllers... }.
• TransactionController
  • This should be converted to the equivalent group("TRANSACTION_CONTROLLER_NAME", function() { ... }) call in k6 JS. The group duration will be the same as the equivalent generated transaction sample in JMeter.
• SimpleController
  • This should be converted to the equivalent group("SIMPLE_CONTROLLER_NAME", function() { ... }) call in k6 JS.
• RandomController
  • This should be converted to pick a child controller at random every execution using something like let rndIndex = Math.floor(Math.random() * (NUM_OF_SUB_CONTOLLERS + 1)).

Timers

• ConstantTimer
  • This should be converted to the equivalent sleep(...) call in k6 JS.

Data

• CSVDataSet (might need to be removed as we don’t have any native CSV APIs yet, maybe if CSV streaming PR is included we could have this?)
• Arguments

Post-processors

• JSONPostProcessor
• BoundaryExtractor
• com.atlantbh.jmeter.plugins.jsonutils.jsonpathextractor.JSONPathExtractor (might need to be removed if not covered by the newly included JSON path extracting in HTTPResponse.json(selector))
• HtmlExtractor/CSSSelectorExtractor
  • This should be converted to the equivalent HTML parsing and CSS selector querying in k6 JS.
• RegexExtractor
  • Convert to the equivalent JS regex code in k6 JS.
• ResultStatusActionHandler
  • This should be converted to check the response status code of a HTTP response and then calling the k6 fail() API to stop the test, something like: if (res.status < 299 and res.status < 400) { fail(); }
• XPathExtractor (as we don’t have any native XPath APIs in k6 yet, this should be converted to a comment stating that "There's currently no XPath API in k6 so a pure JS solution has to be used. Try https://github.com/google/wicked-good-xpath.")

For the following tags it’s enough to dump the code contents as a comment into the JS output:

• JSR223PreProcessor
• JSR223PostProcessor
• BeanShellPreProcessor
• BeanShellPostProcessor

A completed bounty must include a test suite to make sure the converter can properly handle the required tags specified above. It should also include instructions on how to download, build and run the converter in the README.

Resources

• JMeter user manual: https://jmeter.apache.org/usermanual/index.html
• Component reference: http://jmeter.apache.org/usermanual/component_reference.html
• JMX to Yaml converter: https://github.com/Blazemeter/taurus/blob/master/bzt/jmx2yaml.py
• JMX test cases: https://github.com/Blazemeter/taurus/tree/master/tests/resources/jmeter/jmx

[bookmoons]

This is really well specified.

bookmoons/v1 has something toward it. The core elements jmeterTestPlan hashTree TestPlan ThreadGroup SetupThreadGroup PostThreadGroup are converting. Passing tests run with npm test. CircleCI is building commits.

There's a simple example. It produces a script with scaffolding. Empty logic at the moment.

node bin/jmeter-to-k6 example/minimal.xml

<jmeterTestPlan version="1.2" properties="5.0" jmeter="5.0 r1840935">
  <hashTree>
    <TestPlan testname="Search page test">
      <ThreadGroup>
        <stringProp name="ThreadGroup.num_threads">1</stringProp>
        <stringProp name="ThreadGroup.ramp_time">2</stringProp>
      </ThreadGroup>
    </TestPlan>
  </hashTree>
</jmeterTestPlan>

// Search page test

export let options = {
    stages: [{"target":1,"duration":"2s"}]
}

export default function (data) {
  switch (__VU) {
    case 1:
      
      break
    default: throw new Error('Unexpected VU: ' + __VU)
  }
}

[bookmoons]

This seems to be more than $1,000 of work. Would the team consider an increased bounty?

[bookmoons]

Added handling of defaults from ConfigTestElement and the custom SteppingThreadGroup.

There's also a BEHAVIOR document that specifies chosen behavior for each element.

---

..

TestPlan

Attribute

Name	Description	Action
guiclass	?	Ignore.
testclass	?	Ignore.
testname	Plaintext name for test.	Comment.
enabled	Boolean.	Ignore.

Property

Name	Type	Description	Action
comments	string	Contents freeform comments.	Comment.
functional_mode	bool	Enable data saving.	?
tearDown_on_shutdown	bool	Run teardown after shutdown.	?
serialize_threadgroups	bool	Serialize thread groups.	?
user_defined_variables	element	Defines variables.	Variables.
user_define_classpath	string	?	Ignore.

..

[robingustafsson]

@bookmoons Sorry for the late reply. You seems to have a good structure of the project going, nice 👏! Thanks also for the updates, that's great!

Yes, the bounty could possibly be increased. We've increased bounties in the passed. That said, it's been a judgement on the quality of the solution and time put into it. Is there any specific part of the specification above that you think will be more time consuming?

Also, there's a channel in k6 slack (#jmeter-to-k6) where there's been some more discussion on this project/specification with another person that has started an implementation.

[bookmoons]

Thanks @robingustafsson. Going to do a few more elements and come up with something to propose.

The latest has these elements converting:
jmeterTestPlan hashTree TestPlan ThreadGroup SetupThreadGroup PostThreadGroup SteppingThreadGroup ConfigTestElement FtpConfigGui HttpDefaultsGui LdapExtConfigGui LdapConfigGui LoginConfigGui SimpleConfigGui TCPConfigGui DNSCacheManager HeaderManager CookieManager AuthManager DurationAssertion ResponseAssertion XPathAssertion

[bookmoons]

OK, I have a couple of the logic controllers converting. The main time is in dealing with the XML. Finding the structure, determining how values are encoded. Based on time so far $2,200 would make it possible for me. Will set that as a target on Bountysource. Thank you for considering.

There's a larger example with some control structures in full.xml. It converts to:

// Search page test

const constants = {}

const vars = {}
vars["INIT"] = "1" /* Run initialization */
vars["CONTINUE"] = "true"
vars["CHECK"] = "true"

export let options = {
  [{"target":1,"duration":"2s"}]
}

export function setup () {
  /* Open resources */
}

export default function (data) {
  if (__VU >= 1 && __VU <= 1) {
    if (eval(`${vars["INIT"]} > 0`)) {
      // There's currently no XPath API in k6 so a pure JS solution has to be used.
      // Try https://github.com/google/wicked-good-xpath.
    }

    while (`${vars["CONTINUE"]}` !== "false") {
      if (eval(`${vars["CHECK"]}`)) {
        // There's currently no XPath API in k6 so a pure JS solution has to be used.
        // Try https://github.com/google/wicked-good-xpath.
      }
    }
  } else throw new Error('Unexpected VU: ' + __VU)
}

export function teardown (data) {
  /* Close resources */
}

[robingustafsson]

@bookmoons Great! We've talked internally regarding the bounty $ amount and as I said we're happy to consider raising bounties if we feel it's warranted. In this case we think it is, but I'd like to ask you what you think we'd have to cut to get it down to $1800?

[bookmoons]

Thanks guys.

OK, these categories are done:

• Config (core elements)
• Manager
• Assertion
• JMeter variable evaluation (used in controller conditions)

These pieces are pending:

• Sampler
• Controllers
• Timers
• Data
• Processors
• Docs

The processors seem like a good chunk of work. I think if they were dropped it would work. Or alternatively the remaining controllers, they're simpler but there are more of them. The IfController and WhileController are finished.

[robingustafsson]

@bookmoons Thanks for that, we've decided to go ahead with the full bounty for $2,200. So, please continue the great work you've done so far! 😃

[bookmoons]

Understood, thank you. Continuing on.

[bookmoons]

I've been poking around with CSVDataSet the past few days. In JMeter it has a multithreaded characteristic, where the file is opened once centrally and each read from a thread takes the next line. So the data gets distributed across all threads.

I'm not sure if there's a way to duplicate this in k6. Is there any way to achieve cross-VU communication?

[robingustafsson]

No, unfortunately there's no way to share data between VUs at the moment. I suggest you just convert the CSVDataSet to something like this (including the link in the import and NOTE comments):

// See https://support.loadimpact.com/4.0/test-scripting/examples/#reading-parameterization-data-from-a-csv-file
import papaparse from "./papaparse.js";

// NOTE: In JMeter all Virtual Users (aka Threads) can read from the same CSVDataSet.
// In k6 there's no data sharing between VUs. Instead you can use the __VU global variable
// to help partition the data (if running in the Load Impact cloud you'll also have to
// use LI_INSTANCE_ID).
const csvData = papaparse.parse(open('/path/to/data.csv'), {header: true}); // "header" will depend on the CSVDataSet.ignoreFirstLine setting

export default function() {
    // Use the CSV data in whatever way necessary
    let randomUser = csvData.data[Math.floor(Math.random() * csvData.data.length)];
}

[bookmoons]

Copy that.

[bookmoons]

Have submitted a functional tool. All elements on the list are converting.

example/full.xml includes a large example with all elements, generated with JMeter. Converting produces a good example with varied code.

node bin/jmeter-to-k6.js example/full.xml

Some logic looks convoluted, with a lot of template literals. Those pieces are handling runtime evaluation of JMeter variables, which is supposed to be available in nearly all fields.

[bookmoons]

I see the CSVDataSet is not quite working due to bundling issues. Just disabling that element gets me a successful run of the example file. Will be looking into this tomorrow.

[bookmoons]

Switched it to papaparse as recommended for CSV. CSVDataSet successfully processes a CSV file.