Help with 'plugin' option

[RPM1984]

Hi @aliostad

Firstly, awesome library...love it! 👏

I'm trying to get the -p plugin option working, and not having much luck. I'm just wanting to send a bunch of POST requests with different JSON payloads.

What i did

1. Created a new .NET 4.7 class library
2. Installed this NuGet package
3. Created a class like this:

public class SuperbenchmarkerPlugin : IValueProvider
    {
        public IDictionary<string, object> GetValues(int index)
        {
            return new Dictionary<string, object>
            {
                {
                    "hi",
                    new
                    {
                        someProp = "hi"
                    }
                }
            };
        }
    }

4. Built the class library
5. Opened up command line, went to the bin directory of the above project
6. Ran the following sb command, and got an error looking for a CommandLine package:
   

Can you let me know what i'm doing wrong? Any chance you can update the README/examples section with a bit more info on the -p option?

Also, can you please provide more info on how to correctly implement the GetValues method? What do the string and object dictionary keys represent?

Thanks! 😃

[aliostad]

Hi, thanks for the kind words!

First of all, plugin library will implement the interface which will - as you done - return a dictionary of objects (mainly primitive types such as DateTime, string, int, etc) with string keys to replace placeholders.

Let's say you want to send this request to your API:

sb -c 1 -n 1000 -u "http://myserver/api/resource?foo={{{fooValue}}}&bar={{{barValue}}}"

where fooValue is an integer and varValue is a double. You would implement:

public class SuperbenchmarkerPlugin : IValueProvider
    {
        private Random _random = new Random();
        public IDictionary<string, object> GetValues(int index)
        {
            return new Dictionary<string, object>
            {
                {
                    "fooValue",
                    _random.Next()
                },
                {
                    "barValue",
                    _random.NextDouble()
                }
            };
        }
    }

and you would use sb with -p option. As you can see, the .ToString() of the objects (here int and double) will be used to replace the placeholders.

Now going back to your issue. The exception probably has little to do with sb itself. My guess is you are using sb from cloning and building and not using chocolatey or from Download folder of the project which already ilmerge all dependencies into the sb.exe.

Alternatively you are using a different version of CommandLine library which conflicts with the version ilmerged. If none of above is true, please send me a repro project and I will try it out.

Also this could be .NET 4.7. Can you please try with plain .NET 4.5.2?

[aliostad]

Oh, I think I got what you mean. The sb from nuget package is meant just as the library not the runner. You need to get the runner from chocolatey or download from nuget from the Download folder of this project - that has ilmerge of everything and is 4MB.

[RPM1984]

Hi @aliostad , sorry for radio silence...getting slammed at work 😞

Ah, i see the problem...i was running the sb.exe from the folder in my plugin bin directory, whereas i should have been running it from the global level, and absolute referencing the plugin.dll. I got it now... (e.g that error doesn't occur when running from a global cmd level)

So now my next question, clarifying what you have above: if i understand correctly will substitute the fooValue and barValue querystring params, but what i want is to actually send through a POST body as JSON.

e.g

POST: /myapi
Content-Type: "application/json"
{
   "ids": [1,2,3]
}

then another request with:

POST: /myapi
Content-Type: "application/json"
{
   "ids": [4,5]
}

So basically, i'm altering the JSON payload each request (not touching the URI/querystring)

How can i do that with an IValueProvider implementation?

Thanks again ✌️

[aliostad]

Parameterising body is exactly the same, just remember to include -b parameter so sb replaces your placeholders in the body too.

So your template file would look like:

Content-Type: application/json

{
   "ids": [{{{IDS}}}]
}

And in your class:

public class SuperbenchmarkerPlugin : IValueProvider
    {
        private Random _random = new Random();
        public IDictionary<string, object> GetValues(int index)
        {
            return new Dictionary<string, object>
            {
                {
                    "IDS",
                    string.join(",", Enumerable.Range(0, 2).Select(x => _random.Next()));
                },
            };
        }
    }

You can randomise the number of items to send too but you get the idea.

And then run

sb -u "http://myserver/myapi" -t "c:\mytemplate.txt" -b -p mydll.dll

[RPM1984]

Awesome, i got it...just had to make sure i was referencing the plugin DLL via absolute.

Thanks again @aliostad ! 😃

[RPM1984]

Hey @aliostad sorry one final question.. 😊

How can i use the -f (CSV) option in conjuction with the template file?

I see you've got an example on the README:

-u http://localhost/api/myApi/{{{ID}}} -f values.txt -m POST -t template.txt (values file is CSV and has a column for ID, also for all placeholders within the template file)

Does the -f option only apply to the URL?

What i'm wondering, is what if i wanted to test say 3 sets of endpoints, using the plugin technique above? From the looks of it, i would have to create 3 seperate plugin DLL's, and 3 seperate sb.exe commands? From my understanding, the template file applies to all requests.. and i can't just dump multiple varying payloads in there..

Totally workable in the meantime, but in my particular example i have a few endpoints (all POST, with different JSON payloads), that i want to all hit with a single sb.exe test.

[aliostad]

Yes you can, -f applies to URL, headers and body. But do not forget to include -b to parameterise body.

in my particular example i have a few endpoints (all POST, with different JSON payloads), that i want to all hit with a single sb.exe test.

Providing multiple templates not supported but nothing stops you to run it like this http://localhost/api/myApi/{{{URL_PATH}}}:

public class SuperbenchmarkerPlugin : IValueProvider
    {
        private Random _random = new Random();
        public IDictionary<string, object> GetValues(int index)
        
            return new Dictionary<string, object>
            {
                {
                    "URL_PATH",
                    "My_path"
                },
                {
                    "Payload",
                    "{blah blah}"
                },
            };
        }
    }

and then alternate between patterns.

[RPM1984]

Ah got it... thanks!