Pushing the name field (instead of the url) into influxdb name tag to prevent index flood

[nio101]

Hello,
We are using the converter, but upon running the generated script with k6, the name tags in influxdb for our requests are set to the url of the request, and as our url changes everytime, it floods influxdb (index with so many different values) and make the script unusable.
Is there any way to easily set the pushed tag name, or to disable this url to name tag redirection mechanism, or to have the converter use the proper name field of the request instead of the always changing url?
Thanks.

[nio101]

We successfully added manually a name tag to each request.
Could it be possible for the converter to automatically push as name tag the postman name (pm.info.requestName) of the request?
That would be a handful feature!

[thim81]

@nio101 I'm experiencing similar problems with K6 and the reporting in InfluxDB/grafana.

I was already working on a PR to allow such behaviour, but I what would you prefer:

• use the request name like defined in postman, example "Show all accounts"
• use the request name like defined in postman + the HTTP verb, example "GET - Show all accounts"
• use the request name + postman folder name "Accounts - Show all accounts"

[nio101]

@thim81 : as to me, the latter (request name + postman folder name) would be the best solution.
Thanks! 👍

[thim81]

@nio101
Could you share a code example how you added a name tag manually to each request?

That way I'm sure that what is generated is matching with the correct tags for influxdb.

[flegris-orange]

Hello @thim81, a example of tag added manually :

postman[Request]({
     ...
      name: "save order content without customer - Error 40017",
      headers: {
        Accept: "application/json"
      },
      tags: { name: 'save order content without customer - Error 40017'},
      ...
    });

[thim81]

@nio101 @flegris-orange the basic code to include tags, is pretty much finished.

I wanted some input on the cli config and was playing around with providing a tagging strategy CLI parameter.

Example: --requestTagging=request

To support the 3 strategies:

1. none (no automatic tags, default)
2. request (= request name, example "Show all accounts")
3. folder-request (folder name + request name, example: "Accounts - Show all accounts")

What do you think or what did you expected as config option?

[nio101]

That seems great to me! :)

[thim81]

@nio101 @flegris-orange

I have provided a forked NPM package version that contains the feature to set K6 request tags.
Have a look at readme: https://www.npmjs.com/package/@apideck/postman-to-k6#k6-request-tag that explains how to use it.

Replace in your packages.json:

"dependencies": {
    "postman-to-k6": "^1.5.0"
  }

to

"dependencies": {
    "@apideck/postman-to-k6": "^1.8.1"
  }

Question for you: I'm curious which dashboards you are using? I'm using one of the public K6 Grafana dashboard, but looking for more complete ones.

[nio101]

I have provided a forked NPM package version that contains the feature to set K6 request tags.
Have a look at readme: https://www.npmjs.com/package/@apideck/postman-to-k6#k6-request-tag that explains how to use it.

Great! Thank you! :)

Question for you: I'm curious which dashboards you are using? I'm using one of the public K6 Grafana dashboard, but looking for more complete ones.

Here is the grafana dashboard I did, I'm using it on a daily basis: https://gist.github.com/nio101/08ed091ea154a7dcb664c218a41cd25f
You'll have to clean up some metrics (Openshift metrics and others specifics), but there are basic cool metrics for each scenario/group/request tag names and also global k6 metrics.

[thim81]

@nio101 Thanks for sharing your Grafana Dashboard. It looks great, although I havent been able to get it show actual data, since when altering the InfluxDB sources, the browser keeps stuck at loading.

You should publish it on the Grafana https://grafana.com/grafana/dashboards?search=K6, since others might also like it that struggle to display K6 results.

This is my current dashboard:
https://gist.github.com/thim81/832b5e2b566e782f00c2e7c87788d593

I'm trying to find a way to show the checks as well, but it is not easy.

[thim81]

@nio101 Could you also give a bit more info on the "scenario UC1 & UC2"? I'm curious to see how you use it.

[nio101]

@nio101 Could you also give a bit more info on the "scenario UC1 & UC2"? I'm curious to see how you use it.

Yes, if you have a look at the Variables definition, you'll see that the dashboard is extracting all the different scenarios, groups and requests name tag values to fill in the corresponding variables.
Those variables will allow you to filter/aggregate scenario metrics, group metrics and requests based on those values.
Request errors and checks are also shown by error type.

You can ignore the container variable and the openshift metrics...
You'll see a log/loki tab also. I'm using grafana loki to aggregate and show the injection logs inside this dashboard, it's quite handful.

The debug details tab will bring long requests details, and also low level and http errors details for debugging.

note: if you already have some data in your influxdb with lots of requests whose name tag has not been set in your scripts, they may have been set at the url by default, and if your url is varying each time, you index is full of different values and the dashboard may then freeze when trying to request influxb for the name tag when loading ... in that case, just clear influxdb and set the name tag for every request in your scripts... :)

'Scenario UC1&UC2' might just be the last value used when exporting the dashboard json...
Defining and using your own scenarios (https://k6.io/docs/using-k6/scenarios/) will overwrite this...

And yes, I should share it with the community through the grafana website, I never took the time to do so, some cleaning and info should come along with it also... :)

[thim81]

@nio101
I got it to work for most parts. It really is a very good dashboard + plus it shows data in the checks and errors which I was missing in all public dashboards

I'm still running into 2 things that I dont know how fix:

Which widget, needs the promotheus DBM? Since I only have InfluxDB, I want to remove the widget

How do you alter these filters?

[nio101]

Well InfluxDB-k6 is the name of the datasource I used for k6 influxdb metrics
I guess that you could simply create such named datasource pointing to your own k6 influxdb database, or just rename in the JSON definition of your dashboard every InfluxDB-k6 to point to your own datasource?

You can remove the variables related to prometheus or access/alter the filters definition in the Settings/Variables part of your dashboard.

Namespace/container/pod can be removed in your case, along with the 'OpenShift metrics for $container' row of graphs.

The idea of this dashboard is that just by using scenarios/groups/requests name tags, you won't have most of the time to deal with custom metrics in your scripts. Every info/data should already be available in this generic dashboard.

[thim81]

@nio101 It works now 🚀 , after I removed the Namespace/container/pod and changed the variables datasource.

It all works without any Grafana errors. My browers still has difficulties loading all the data, but that can be sorted out. Thanks for sharing and helping out.

[nio101]

@nio101 It works now 🚀 , after I removed the Namespace/container/pod and changed the variables datasource.

It all works without any Grafana errors. My browers still has difficulties loading all the data, but that can be sorted out. Thanks for sharing and helping out.

Great! Thanks for helping us sorting out this converter limitation! :)

[nio101]

I finally took some time: https://grafana.com/grafana/dashboards/15030 :)

[flegris-orange]

@nio101 @flegris-orange the basic code to include tags, is pretty much finished.

I wanted some input on the cli config and was playing around with providing a tagging strategy CLI parameter.

Example: --requestTagging=request

To support the 3 strategies:

1. none (no automatic tags, default)

2. `request` (= request name, example "Show all accounts")

3. `folder-request` (folder name + request name, example: "Accounts - Show all accounts")

What do you think or what did you expected as config option?

@thim81 I have tested the two proposed strategies (request and folder-request) it works as expected. Thanks a lot.

[sasha240100]

@thim81 Great work, thanks! Could you please add support for test_run_id tag or any other? (provide tag name in options)

[thim81]

@sasha240100 I ll have a look. Could you share a sample what you would expect from the generated K6 script, a single request would be sufficient?

[thim81]

@sasha240100 I had a look at the available https://k6.io/docs/using-k6/tags-and-groups/ but test_run_id is not part of it? Would you like to set a fixed "tag" & "value through the options?

Example: --requestTag=test_run_id --requestTagValue=1234567

[ppcano]

Apologies for the inactivity on this project.

Due to the inability to properly support this project, the k6 team has decided not to continue its development. The primary reason is that k6 scope has grown significantly with the launch of k6 extensions. The team is prioritizing extending k6 capabilities over the converters.

We suggest you post your issue at api-deck/postman-to-k6. The project is active and maintained on this fork.