Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add response.ok support to FastHttpUser's FastResponse, ala HttpUser/requests #2520

Closed
2 tasks done
petarmaric opened this issue Dec 20, 2023 · 4 comments
Closed
2 tasks done

Add response.ok support to FastHttpUser's FastResponse, ala HttpUser/requests #2520

petarmaric opened this issue Dec 20, 2023 · 4 comments
Labels
feature request

Comments

@petarmaric
Copy link

Prerequisites

Description

This will allow FastHttpUser to be more of a drop-in replacement for HttpUser in auth-or-die scenarios, such as this:

import signal

from locust import HttpUser, task


class HelloWorldUser(HttpUser):
    def _quit(self):
        # HACK: Raise a *nix signal to quit on errors as the usual way of
        # `self.environment.runner.quit()` no longer works reliably when the
        # locust's 'run-time' option is used. It does quit the locust's runner,
        # but not the locust's *nix process itself - which remains alive until
        # the entire 'run-time' has expired.
        signal.raise_signal(signal.SIGTERM)

    def on_start(self):
        with self.client.post(
            url="/v1/token",
            json={
                "username": "super-secret-username",
                "password": "super-secret-password",
            },
            catch_response=True,
        ) as response:
            if not response.ok:
                response.failure(
                    f"Login failed, please check the provided username/password. HTTP {response.status_code}: {response.text}"
                )
                self._quit()

    @task
    def hello_world(self):
        self.client.get("/hello")
        self.client.get("/world")

Example locust run:

$ locust --users=50 --autostart --autoquit=1 --run-time=2m --locustfile=HttpUser.py --host=https://example.org
[2023-12-20 13:39:17,193] tuxedo/INFO/locust.main: Starting web interface at http://0.0.0.0:8089
[2023-12-20 13:39:17,199] tuxedo/INFO/locust.main: Starting Locust 2.20.0
[2023-12-20 13:39:17,200] tuxedo/INFO/locust.main: Run time limit set to 120 seconds
[2023-12-20 13:39:17,203] tuxedo/INFO/locust.runners: Ramping to 50 users at a rate of 1.00 per second
[2023-12-20 13:39:17,816] tuxedo/INFO/locust.main: Got SIGTERM signal
[2023-12-20 13:39:17,816] tuxedo/INFO/locust.main: Shutting down (exit code 1)
Type     Name                                                                # reqs      # fails |    Avg     Min     Max    Med |   req/s  failures/s
--------|------------------------------------------------------------------|-------|-------------|-------|-------|-------|-------|--------|-----------
POST     /v1/token                                                                1   1(100.00%) |    611     611     611    611 |    1.63        1.63
--------|------------------------------------------------------------------|-------|-------------|-------|-------|-------|-------|--------|-----------
         Aggregated                                                               1   1(100.00%) |    611     611     611    611 |    1.63        1.63

Response time percentiles (approximated)
Type     Name                                                                        50%    66%    75%    80%    90%    95%    98%    99%  99.9% 99.99%   100% # reqs
--------|----------------------------------------------------------------------|--------|------|------|------|------|------|------|------|------|------|------|------
POST     /v1/token                                                                   610    610    610    610    610    610    610    610    610    610    610      1
--------|----------------------------------------------------------------------|--------|------|------|------|------|------|------|------|------|------|------|------
         Aggregated                                                                  610    610    610    610    610    610    610    610    610    610    610      1

Error report
# occurrences      Error                                                                                               
------------------|-----------------------------------------------------------------------------------------------------------------------------------
1                  POST /v1/token: CatchResponseError('Login failed, please check the provided username/password. HTTP 404: <?xml version="1.0" encoding="iso-8859-1"?>\n<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"\n         "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">\n<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">\n\t<head>\n\t\t<title>404 - Not Found</title>\n\t</head>\n\t<body>\n\t\t<h1>404 - Not Found</h1>\n\t\t<script type="text/javascript" src="//obj.ac.bcon.ecdns.net/ec_tpm_bcon.js"></script>\n\t</body>\n</html>\n')
------------------|-----------------------------------------------------------------------------------------------------------------------------------
@petarmaric
Copy link
Author

The naive FastHttpUser drop-in will not work:

diff --git a/HttpUser.py b/FastHttpUser.py
index db0ad8b..df53521 100644
--- a/HttpUser.py
+++ b/FastHttpUser.py
@@ -1,9 +1,9 @@
 import signal
 
-from locust import HttpUser, task
+from locust import FastHttpUser, task
 
 
-class HelloWorldUser(HttpUser):
+class HelloWorldUser(FastHttpUser):
     def _quit(self):
         # HACK: Raise a *nix signal to quit on errors as the usual way of
         # `self.environment.runner.quit()` no longer works reliably when the

... as seen in this example locust run:

$ locust --users=50 --autostart --autoquit=1 --run-time=2m --locustfile=FastHttpUser.py --host=https://example.org
[2023-12-20 13:44:03,924] tuxedo/INFO/locust.main: Starting web interface at http://0.0.0.0:8089
[2023-12-20 13:44:03,931] tuxedo/INFO/locust.main: Starting Locust 2.20.0
[2023-12-20 13:44:03,931] tuxedo/INFO/locust.main: Run time limit set to 120 seconds
[2023-12-20 13:44:03,935] tuxedo/INFO/locust.runners: Ramping to 50 users at a rate of 1.00 per second
[2023-12-20 13:44:04,670] tuxedo/ERROR/locust.user.users: 'ResponseContextManager' object has no attribute 'ok'
Traceback (most recent call last):
  File "/home/petar/.virtualenvs/tmp-d6872755a32613/lib/python3.10/site-packages/locust/user/users.py", line 149, in run
    self.on_start()
  File "/tmp/FastHttpUser.py", line 24, in on_start
    if not response.ok:
AttributeError: 'ResponseContextManager' object has no attribute 'ok'

Traceback (most recent call last):
  File "src/gevent/greenlet.py", line 908, in gevent._gevent_cgreenlet.Greenlet.run
  File "/home/petar/.virtualenvs/tmp-d6872755a32613/lib/python3.10/site-packages/locust/user/users.py", line 187, in run_user
    user.run()
  File "/home/petar/.virtualenvs/tmp-d6872755a32613/lib/python3.10/site-packages/locust/user/users.py", line 149, in run
    self.on_start()
  File "/tmp/FastHttpUser.py", line 24, in on_start
    if not response.ok:
AttributeError: 'ResponseContextManager' object has no attribute 'ok'
2023-12-20T12:44:04Z <Greenlet at 0x7f40140e3920: run_user(<FastHttpUser.HelloWorldUser object at 0x7f4013fee)> failed with AttributeError

KeyboardInterrupt
2023-12-20T12:44:04Z
[2023-12-20 13:44:04,769] tuxedo/INFO/locust.main: Shutting down (exit code 0)
Type     Name                                                                # reqs      # fails |    Avg     Min     Max    Med |   req/s  failures/s
--------|------------------------------------------------------------------|-------|-------------|-------|-------|-------|-------|--------|-----------
--------|------------------------------------------------------------------|-------|-------------|-------|-------|-------|-------|--------|-----------
         Aggregated                                                               0     0(0.00%) |      0       0       0      0 |    0.00        0.00

Response time percentiles (approximated)
Type     Name                                                                        50%    66%    75%    80%    90%    95%    98%    99%  99.9% 99.99%   100% # reqs
--------|----------------------------------------------------------------------|--------|------|------|------|------|------|------|------|------|------|------|------
--------|----------------------------------------------------------------------|--------|------|------|------|------|------|------|------|------|------|------|------

@petarmaric
Copy link
Author

FWIW, this is my current (albeit quite hacky) workaround:

import signal

from locust import FastHttpUser, task


class HelloWorldUser(FastHttpUser):
    def _quit(self):
        # HACK: Raise a *nix signal to quit on errors as the usual way of
        # `self.environment.runner.quit()` no longer works reliably when the
        # locust's 'run-time' option is used. It does quit the locust's runner,
        # but not the locust's *nix process itself - which remains alive until
        # the entire 'run-time' has expired.
        signal.raise_signal(signal.SIGTERM)

    def on_start(self):
        with self.client.post(
            url="/v1/token",
            json={
                "username": "super-secret-username",
                "password": "super-secret-password",
            },
            catch_response=True,
        ) as response:
            # HACK: Locust `FastHttpUser` doesn't have `response.ok` (unlike
            # `HttpUser`) so we're employing a hack here to quit on errors.
            # Same principle as used by the `FastHttpUser` internals, until
            # https://github.com/locustio/locust/issues/2520 is resolved :shrug:
            if getattr(response, "error", None):
                if response.text is None:
                    response.failure(response.error)
                else:
                    response.failure(
                        f"Login failed, please check the provided username/password. HTTP {response.status_code}: {response.text}"
                    )

                self._quit()

    @task
    def hello_world(self):
        self.client.get("/hello")
        self.client.get("/world")

... with its example locust run:

$ locust --users=50 --autostart --autoquit=1 --run-time=2m --locustfile=FastHttpUserHACK.py --host=https://example.org
[2023-12-20 14:34:56,139] tuxedo/INFO/locust.main: Starting web interface at http://0.0.0.0:8089
[2023-12-20 14:34:56,146] tuxedo/INFO/locust.main: Starting Locust 2.20.0
[2023-12-20 14:34:56,146] tuxedo/INFO/locust.main: Run time limit set to 120 seconds
[2023-12-20 14:34:56,150] tuxedo/INFO/locust.runners: Ramping to 50 users at a rate of 1.00 per second
[2023-12-20 14:34:56,877] tuxedo/INFO/locust.main: Got SIGTERM signal
[2023-12-20 14:34:56,877] tuxedo/INFO/locust.main: Shutting down (exit code 1)
Type     Name                                                                # reqs      # fails |    Avg     Min     Max    Med |   req/s  failures/s
--------|------------------------------------------------------------------|-------|-------------|-------|-------|-------|-------|--------|-----------
POST     /v1/token                                                                1   1(100.00%) |    726     726     726    726 |    1.38        1.38
--------|------------------------------------------------------------------|-------|-------------|-------|-------|-------|-------|--------|-----------
         Aggregated                                                               1   1(100.00%) |    726     726     726    726 |    1.38        1.38

Response time percentiles (approximated)
Type     Name                                                                        50%    66%    75%    80%    90%    95%    98%    99%  99.9% 99.99%   100% # reqs
--------|----------------------------------------------------------------------|--------|------|------|------|------|------|------|------|------|------|------|------
POST     /v1/token                                                                   730    730    730    730    730    730    730    730    730    730    730      1
--------|----------------------------------------------------------------------|--------|------|------|------|------|------|------|------|------|------|------|------
         Aggregated                                                                  730    730    730    730    730    730    730    730    730    730    730      1

Error report
# occurrences      Error                                                                                               
------------------|-----------------------------------------------------------------------------------------------------------------------------------
1                  POST /v1/token: CatchResponseError('Login failed, please check the provided username/password. HTTP 404: <?xml version="1.0" encoding="iso-8859-1"?>\n<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"\n         "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">\n<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">\n\t<head>\n\t\t<title>404 - Not Found</title>\n\t</head>\n\t<body>\n\t\t<h1>404 - Not Found</h1>\n\t\t<script type="text/javascript" src="//obj.ac.bcon.ecdns.net/ec_tpm_bcon.js"></script>\n\t</body>\n</html>\n')
------------------|-----------------------------------------------------------------------------------------------------------------------------------

@cyberw
Copy link
Collaborator

cyberw commented Dec 20, 2023

👍 PR welcome.

@cyberw
Copy link
Collaborator

cyberw commented Jan 6, 2024

fixed in #2535

@cyberw cyberw closed this as completed Jan 6, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
feature request
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@petarmaric @cyberw