datasette-gunicorn plugin

[simonw]

Based on I'm interested to see how Datasette would work if run using Gunicorn with multiple workers.

datasette gunicorn ... options here ...

[simonw]

https://docs.gunicorn.org/en/latest/custom.html is relevant.

Used GitHub Code Search for gunicorn.app.base.BaseApplication to find this example:

https://github.com/apache/airflow/blob/5e542e7532521665dadc4d428c53cbc8e7e57c66/airflow/utils/serve_logs.py#L117

[simonw]

Even better example: https://github.com/cetanu/sovereign/blob/69f91c7ea7519d658beb74822941cf375bdb2f19/src/sovereign/server.py#L8

class StandaloneApplication(gunicorn.app.base.BaseApplication):  # type: ignore
    def __init__(
        self, application: FastAPI, options: Optional[Dict[str, Any]] = None
    ) -> None:
        self.options = options or {}
        self.application = application
        super().__init__()

    def load_config(self) -> None:
        for key, value in self.options.items():
            self.cfg.set(key.lower(), value)

    def load(self) -> FastAPI:
        return self.application


def main() -> None:
    asgi = StandaloneApplication(
        application=app, options=asgi_config.as_gunicorn_conf()
    )
    asgi.run()

[simonw]

Datasette's CLI serve() function has an undocumented return_instance=False argument I can use to get back the Datasette instance:

https://github.com/simonw/datasette/blob/fdf9891c3f0313af9244778574c7ebaac9c3a438/datasette/cli.py#L499

[simonw]

Got it working, tried with 4 workers, got 66.42 requests/second.

Type     Name                                                                          # reqs      # fails |    Avg     Min     Max    Med |   req/s  failures/s
--------|----------------------------------------------------------------------------|-------|-------------|-------|-------|-------|-------|--------|-----------
GET      /fixtures/sortable                                                              1922     8(0.42%) |   1357      37    3232   1200 |   66.42        0.28
--------|----------------------------------------------------------------------------|-------|-------------|-------|-------|-------|-------|--------|-----------
         Aggregated                                                                      1922     8(0.42%) |   1357      37    3232   1200 |   66.42        0.28

Response time percentiles (approximated)
Type     Name                                                                                  50%    66%    75%    80%    90%    95%    98%    99%  99.9% 99.99%   100% # reqs
--------|--------------------------------------------------------------------------------|--------|------|------|------|------|------|------|------|------|------|------|------
GET      /fixtures/sortable                                                                   1200   1300   2100   2400   2500   2600   2700   2800   3200   3200   3200   1922
--------|--------------------------------------------------------------------------------|--------|------|------|------|------|------|------|------|------|------|------|------
         Aggregated                                                                           1200   1300   2100   2400   2500   2600   2700   2800   3200   3200   3200   1922

Error report
# occurrences      Error                                                                                               
------------------|---------------------------------------------------------------------------------------------------------------------------------------------
8                  GET /fixtures/sortable: RemoteDisconnected('Remote end closed connection without response')         
------------------|---------------------------------------------------------------------------------------------------------------------------------------------

With one worker 26.78 requests/second.

Type     Name                                                                          # reqs      # fails |    Avg     Min     Max    Med |   req/s  failures/s
--------|----------------------------------------------------------------------------|-------|-------------|-------|-------|-------|-------|--------|-----------
GET      /fixtures/sortable                                                               768     0(0.00%) |   3183     536    4548   3500 |   26.78        0.00
--------|----------------------------------------------------------------------------|-------|-------------|-------|-------|-------|-------|--------|-----------
         Aggregated                                                                       768     0(0.00%) |   3183     536    4548   3500 |   26.78        0.00

Response time percentiles (approximated)
Type     Name                                                                                  50%    66%    75%    80%    90%    95%    98%    99%  99.9% 99.99%   100% # reqs
--------|--------------------------------------------------------------------------------|--------|------|------|------|------|------|------|------|------|------|------|------
GET      /fixtures/sortable                                                                   3500   3500   3500   3600   3600   3900   4000   4200   4500   4500   4500    768
--------|--------------------------------------------------------------------------------|--------|------|------|------|------|------|------|------|------|------|------|------
         Aggregated                                                                           3500   3500   3500   3600   3600   3900   4000   4200   4500   4500   4500    768

That was against /fixtures/sortable not in WAL mode.

[simonw]

Switched it to WAL mode (which took a few goes, not sure why - kept getting errors about not being able to open the database but they eventually went away after I tried a few times with fresh copies).

With 1 worker, 21 requests/second:

Type     Name                                                                          # reqs      # fails |    Avg     Min     Max    Med |   req/s  failures/s
--------|----------------------------------------------------------------------------|-------|-------------|-------|-------|-------|-------|--------|-----------
GET      /fixtures/sortable                                                               714     0(0.00%) |   4166     686    5938   4600 |   21.33        0.00
--------|----------------------------------------------------------------------------|-------|-------------|-------|-------|-------|-------|--------|-----------
         Aggregated                                                                       714     0(0.00%) |   4166     686    5938   4600 |   21.33        0.00

Response time percentiles (approximated)
Type     Name                                                                                  50%    66%    75%    80%    90%    95%    98%    99%  99.9% 99.99%   100% # reqs
--------|--------------------------------------------------------------------------------|--------|------|------|------|------|------|------|------|------|------|------|------
GET      /fixtures/sortable                                                                   4600   4600   4700   4700   4700   4800   5200   5500   5900   5900   5900    714
--------|--------------------------------------------------------------------------------|--------|------|------|------|------|------|------|------|------|------|------|------
         Aggregated                                                                           4600   4600   4700   4700   4700   4800   5200   5500   5900   5900   5900    714

With 4 workers, 75 requests/second:

Type     Name                                                                          # reqs      # fails |    Avg     Min     Max    Med |   req/s  failures/s
--------|----------------------------------------------------------------------------|-------|-------------|-------|-------|-------|-------|--------|-----------
GET      /fixtures/sortable                                                              2024     0(0.00%) |   1176      50    3009   1100 |   75.96        0.00
--------|----------------------------------------------------------------------------|-------|-------------|-------|-------|-------|-------|--------|-----------
         Aggregated                                                                      2024     0(0.00%) |   1176      50    3009   1100 |   75.96        0.00

Response time percentiles (approximated)
Type     Name                                                                                  50%    66%    75%    80%    90%    95%    98%    99%  99.9% 99.99%   100% # reqs
--------|--------------------------------------------------------------------------------|--------|------|------|------|------|------|------|------|------|------|------|------
GET      /fixtures/sortable                                                                   1100   1300   1400   2000   2300   2400   2400   2700   2900   3000   3000   2024
--------|--------------------------------------------------------------------------------|--------|------|------|------|------|------|------|------|------|------|------|------
         Aggregated                                                                           1100   1300   1400   2000   2300   2400   2400   2700   2900   3000   3000   2024

This is with 100 concurrent users:

locust --headless --users 100 --spawn-rate 20 -H http://127.0.0.1:8001

Gonna try with 4 concurrent users instead:

locust --headless --users 4 --spawn-rate 4 -H http://127.0.0.1:8001

Type     Name                                                                          # reqs      # fails |    Avg     Min     Max    Med |   req/s  failures/s
--------|----------------------------------------------------------------------------|-------|-------------|-------|-------|-------|-------|--------|-----------
GET      /fixtures/sortable                                                              1418     0(0.00%) |     52      34     124     39 |   76.03        0.00
--------|----------------------------------------------------------------------------|-------|-------------|-------|-------|-------|-------|--------|-----------
         Aggregated                                                                      1418     0(0.00%) |     52      34     124     39 |   76.03        0.00

Response time percentiles (approximated)
Type     Name                                                                                  50%    66%    75%    80%    90%    95%    98%    99%  99.9% 99.99%   100% # reqs
--------|--------------------------------------------------------------------------------|--------|------|------|------|------|------|------|------|------|------|------|------
GET      /fixtures/sortable                                                                     39     58     75     76     80     85     95     99    120    120    120   1418
--------|--------------------------------------------------------------------------------|--------|------|------|------|------|------|------|------|------|------|------|------
         Aggregated                                                                             39     58     75     76     80     85     95     99    120    120    120   1418

That got me 76 requests/second but a much better latency (all within 120ms).

With only a single worker:

datasette gunicorn fixtures.db -w 1

Locust:

locust --headless --users 4 --spawn-rate 4 -H http://127.0.0.1:8001

Type     Name                                                                          # reqs      # fails |    Avg     Min     Max    Med |   req/s  failures/s
--------|----------------------------------------------------------------------------|-------|-------------|-------|-------|-------|-------|--------|-----------
GET      /fixtures/sortable                                                               475     0(0.00%) |    169     110     483    170 |   23.58        0.00
--------|----------------------------------------------------------------------------|-------|-------------|-------|-------|-------|-------|--------|-----------
         Aggregated                                                                       475     0(0.00%) |    169     110     483    170 |   23.58        0.00

Response time percentiles (approximated)
Type     Name                                                                                  50%    66%    75%    80%    90%    95%    98%    99%  99.9% 99.99%   100% # reqs
--------|--------------------------------------------------------------------------------|--------|------|------|------|------|------|------|------|------|------|------|------
GET      /fixtures/sortable                                                                    170    170    180    180    190    200    210    250    480    480    480    475
--------|--------------------------------------------------------------------------------|--------|------|------|------|------|------|------|------|------|------|------|------
         Aggregated                                                                            170    170    180    180    190    200    210    250    480    480    480    475

All under 475ms, 24 requests/second.

Trying one worker, one user:

Type     Name                                                                          # reqs      # fails |    Avg     Min     Max    Med |   req/s  failures/s
--------|----------------------------------------------------------------------------|-------|-------------|-------|-------|-------|-------|--------|-----------
GET      /fixtures/sortable                                                               497     0(0.00%) |     38      34     184     37 |   26.05        0.00
--------|----------------------------------------------------------------------------|-------|-------------|-------|-------|-------|-------|--------|-----------
         Aggregated                                                                       497     0(0.00%) |     38      34     184     37 |   26.05        0.00

Response time percentiles (approximated)
Type     Name                                                                                  50%    66%    75%    80%    90%    95%    98%    99%  99.9% 99.99%   100% # reqs
--------|--------------------------------------------------------------------------------|--------|------|------|------|------|------|------|------|------|------|------|------
GET      /fixtures/sortable                                                                     37     37     38     38     41     43     56     57    180    180    180    497
--------|--------------------------------------------------------------------------------|--------|------|------|------|------|------|------|------|------|------|------|------
         Aggregated                                                                             37     37     38     38     41     43     56     57    180    180    180    497

26 requests/second, most under 43 ms, 497ms max.

[simonw]

It doesn't really have a test yet.

Also some of the options don't work - datasette gunicorn --get /.json for example doesn't do what --get is supposed to do, it just starts the server.

[simonw]

I want a test that actually starts a server, runs an HTTP request through it and then shuts the server down again.

https://github.com/simonw/datasette/blob/0.62/tests/test_cli_serve_server.py has some examples, using this fixture: https://github.com/simonw/datasette/blob/0.62/tests/conftest.py#L104-L119