Locust test (#257)

* Add: Locust test with arbitrary jac code.

* Add: Functioning locust test file.

* Add: Try to login before creating users.

* Modify: MVP Locust Test.

* Add: MVP locust test containerized

* Add: Text-segmenter action loaded

* Modify: Readme

* Modify: separate user create and the usual test program.

* Modify: Separated the test program configuration into json files.

* Modify: Simplify the output.

* Modify: working sample code with a not working local action load now.

* Fix: the biencoder test program.

* Modify: the overall folder structre, deleted useless files.

* Modify: Further clarify the structure.

* Format: the locust test.

* Add: Minimal Viable Dockerfile

* Add: a simple python to setup the docker containers.

The python is not yet ready.

* Add: Multiple concurrent tests.

* Add: Retrieved all the necessary data.

* Modify: separate test and sentinel

* Modify: Combined the prepare with the addjac.py

* Modify: Set the on exit kill function

Add: A test with action load.

* format

* Modify: README

* Remove: Useless files

* Typo: 4 Locust CSV

* Some updates and improvements

Co-authored-by: Yiping Kang <ypkang@umich.edu>

support/locust/.gitignore

@@ -0,0 +1,2 @@
+__pycache__
+results/

support/locust/Dockerfile

@@ -0,0 +1,8 @@
+FROM python:latest
+WORKDIR /locust
+EXPOSE 8089
+COPY . /locust/src
+COPY sample_code /locust/sample_code
+RUN pip3 install locust
+RUN mkdir /locust/csv
+CMD locust -f /locust/src/addjac.py --headless -u $LOCUST_USER_NUMBER -r $LOCUST_SPAWN_RATE -H $LOCUST_HOST --run-time $LOCUST_DURATION --csv=csv/data

support/locust/README.md

@@ -0,0 +1,76 @@
+# Locust Load Test for JASECI
+Locust is an easy-to-use, distributed, user load testing tool. It is intended for load-testing web sites (or other systems) and figuring out how many concurrent users a system can handle.
+
+## Run Locust natively
+
+### Install Locust
+```console
+pip install locust
+```
+### Configure the test
+Create test users. This following script will prompt you for the jaseci server URL and number of test users you wish to create.
+```bash
+python create_users.py
+```
+Then create a folder in `sample_code/`. Set up a file `config.json` in the folder. Here is an example:
+```json
+{
+    "walkers" : ["init"], 
+    "src" : "walker.jac",
+    "remote_actions" : ["http://flair-ner:80/"],
+    "local_actions" : []
+}
+```
+
+`walkers` is a list of walkers that you want to call (in sequence). `src` the name of the file that contains your code. `remote_actions` should contain a list of URLs of your remote services. `local_actions` should contain a list of names of your modules.
+
+### Run the test
+The program reads the environment variable `LOCUST_TEST_SRC` for the location of the test configuration and `LOCUST_HOST` for the jaseci server URL.
+```bash
+LOCUST_HOST='JASECI_URL' LOCUST_TEST_SRC='sample_code/<YOUR TEST>' locust -f run_jac.py
+```
+
+Go to the link specified in console, e.x http://0.0.0.0:8089 and specify the desired number of users for the load test and initiate the test. You can also change the server URL in the web UI.
+
+## Run Locust with docker
+
+### Set up the environment
+Install docker
+```bash
+pip install docker
+```
+Build the custom docker image
+```bash
+docker build -t locust-jac-test .
+```
+**Note** If you are testing a localhost jaseci, please make sure that your Jaseci service is exposed to `0.0.0.0` since we are going to access the service from docker, not local. To achieve that, run
+```bash
+kubectl port-forward <JASECI POD NAME> 8888:80 --address="0.0.0.0"
+```
+### Configure the tests
+
+**Note** Please make sure that you have configured the tests properly as we did in the previous section.
+
+Since we are not going to open a Web UI this time, we need some more information. Please give all the information in `test.json`. Here is an example
+```json
+{
+    "hostName": "http://172.17.0.1:8888",
+    "testName": "simple",
+    "testSRC": "sample_code/simple",
+    "userNum": 5,
+    "spawnRate": 1,
+    "duration": "10s"
+}
+```
+`hostName` gives the URL of the host. Note that `localhost` on the host computer is mapped to `172.17.0.1` inside docker containers. `testName` is a simple name for the test. It will be included in the name of the container and the result that we retrieve. `testSRC` specifies the path to the specific test configuration. `userNum` specifies the number of users that we need to spawn in this test. `spawnRate` specfies the speed that we spawn the users (How many users created in one second). `duration` sets the time length of the test.
+
+### Run the test
+To run the test
+```bash
+python start_docker.py
+```
+All the tests will be created inside a separate docker container. The containers are named `Locust_<TESTNAME>`. All the tests should be run in parallel. When all the tests are done, the python script automatically removes and kills all the containers.
+
+### Retrieve the test data
+
+All available data are retrieved after you ran the script. They should be available under `results/<testName>/`. `logs.txt` is the log of the test. `data.tar` file should contain four CSV files. They are directly from locust. 

support/locust/create_users.py

@@ -0,0 +1,20 @@
+import requests
+import utils
+
+targetServer = input("Target Server: ")
+UserNum = int(input("Number of Users: "))
+
+# Create all the users
+for i in range(UserNum):
+    username = utils.gen_username(i)
+    password = utils.gen_password(i)
+    data = {
+        "email": username,
+        "password": password,
+        "is_activated": True,
+        "is_superuser": True,
+    }
+    URL = targetServer + "/user/create/"
+    print(data)
+    response = requests.post(URL, data=data)
+    print(response.text)

support/locust/prepare.py

@@ -0,0 +1,60 @@
+import requests
+import utils
+import os
+
+
+TEST_PATH = os.environ.get("LOCUST_TEST_SRC", "")
+HOST = os.environ.get("LOCUST_HOST", "http://localhost:8888")
+
+# Log in as user 0, return the token
+def login():
+    userName = utils.gen_username(0)
+    password = utils.gen_password(0)
+    response = requests.post(
+        HOST + "/user/token/", json={"email": userName, "password": password}
+    )
+    return response.json()["token"]
+
+
+# register sentinel, return the jid
+def registerSentinel(token: str):
+    req = {
+        "name": "jac_prog",
+        "code": utils.get_code(utils.load_config(TEST_PATH)["src"]),
+    }
+    response = requests.post(
+        HOST + "/js/sentinel_register",
+        headers={"authorization": f"Token {token}"},
+        json=req,
+    )
+    return response.json()[0]["jid"]
+
+
+def setSentinelGlobal(token: str, snt: str):
+    response = requests.post(
+        HOST + "/js_admin/global_sentinel_set",
+        headers={"authorization": f"Token {token}"},
+        json={"snt": snt},
+    )
+
+
+def load_actions(token: str):
+    for action in utils.load_config(TEST_PATH)["remote_actions"]:
+        response = requests.post(
+            HOST + "/js_admin/actions_load_remote",
+            headers={"authorization": f"Token {token}"},
+            json={"url": action},
+        )
+    for action in utils.load_config(TEST_PATH)["local_actions"]:
+        response = requests.post(
+            HOST + "/js_admin/actions_load_local",
+            headers={"authorization": f"Token {token}"},
+            json={"file": action},
+        )
+
+    response = requests.post(
+        HOST + "/js_admin/actions_list",
+        headers={"authorization": f"Token {token}"},
+    )
+
+    # print(response.text)

support/locust/run_jac.py

@@ -0,0 +1,73 @@
+from locust import task, HttpUser, SequentialTaskSet, constant, HttpUser
+import os
+from utils import gen_username, gen_password, load_config
+import prepare
+
+TEST_PATH = os.environ.get("LOCUST_TEST_SRC", "")
+SNT = ""
+
+
+def format_output(userName: str, output: str):
+    print(f"{userName}: {output}")
+
+
+UserID = 0
+actionLoaded = False
+
+
+def print_response(response):
+    failure = False
+    if response.status_code != 200 and response.status_code != 201:
+        failure = True
+    success = response.json().get("success", False)
+    if not success:
+        failure = True
+    if failure:
+        print(response.text)
+
+
+class SeqTask(SequentialTaskSet):
+    def on_start(self):
+        self.userName = "Not_exist"  # Initialize
+        self.password = "Not_exist"
+        self.zsb_token = "None"
+        self.zsb_jid = "None"
+        global UserID
+        self.userName, self.password = gen_username(UserID), gen_password(UserID)
+        self.userID = UserID
+        UserID += 1
+
+    @task
+    def generate_userToken(self):
+        response = self.client.post(
+            "/user/token/", json={"email": self.userName, "password": self.password}
+        )
+        format_output(self.userName, response.text)
+        json_var = response.json()
+        self.user_token = json_var["token"]
+
+    @task
+    def walker_run(self):
+        for walkerName in load_config(TEST_PATH)["walkers"]:
+            req = {"name": walkerName, "snt": SNT}
+            # print(f"Walker {walkerName} running.")
+            response = self.client.post(
+                "/js/walker_run",
+                headers={"authorization": f"Token {self.user_token}"},
+                json=req,
+            )
+            print(f"Walker {walkerName} finished. {response.text}")
+            # print(f"Walker {walkerName} Output: {response.json()}")
+
+
+class addJac(HttpUser):
+    host = "http://127.0.0.1:8888"
+    tasks = [SeqTask]
+    wait_time = constant(2)
+
+
+token = prepare.login()
+SNT = prepare.registerSentinel(token)
+prepare.load_actions(token)
+prepare.setSentinelGlobal(token, SNT)
+print(SNT)

support/locust/sample_code/bi_enc_test/config.json

@@ -0,0 +1,6 @@
+{
+    "walkers" : ["init"],
+    "src" : "walker.jac",
+    "remote_actions" : ["http://bi-enc:80/"],
+    "local_actions" : []
+}

support/locust/sample_code/bi_enc_test/walker.jac

@@ -0,0 +1,6 @@
+walker init {
+    can bi_enc.infer;
+    root {
+        report bi_enc.infer(contexts = ["I like the restaurant near my house!"], candidates = ["University", "restaurant", "school", "computer"], context_type = "text", candidate_type = "text");
+    }
+}

support/locust/sample_code/flair_test/config.json

@@ -0,0 +1,7 @@
+
+{
+    "walkers" : ["init"],
+    "src" : "walker.jac",
+    "remote_actions" : ["http://flair-ner:80/"],
+    "local_actions" : []
+}

support/locust/sample_code/flair_test/walker.jac

@@ -0,0 +1,6 @@
+walker init {
+    can ent_ext.entity_detection;
+    root {
+        report ent_ext.entity_detection(text = "Hello, I am from the University of Michigan!", ner_labels = ["University", "detection", "US", "Jaseci"])['entities'];
+    }
+}

support/locust/sample_code/flair_test_local/config.json

@@ -0,0 +1,7 @@
+
+{
+    "walkers" : ["init"],
+    "src" : "walker.jac",
+    "remote_actions" : [],
+    "local_actions" : ["/jaseci/jaseci_kit/jaseci_kit/modules/entity_extraction/entity_extraction.py"]
+}

support/locust/sample_code/flair_test_local/walker.jac

@@ -0,0 +1,6 @@
+walker init {
+    can ent_ext.entity_detection;
+    root {
+        report ent_ext.entity_detection(text = "Hello, I am from the University of Michigan!", ner_labels = ["University", "detection", "US", "Jaseci"])['entities'];
+    }
+}

support/locust/sample_code/simple/config.json

@@ -0,0 +1,6 @@
+{
+    "walkers" : ["init"],
+    "remote_actions" : [],
+    "local_actions" : [],
+    "src" : "walker.jac"
+}

support/locust/sample_code/simple/walker.jac

@@ -0,0 +1,6 @@
+walker init {
+    root {
+        std.out("Hello, this is from the root");
+        report "Test SUCCESS";
+    }
+}