Experimental logs route

[irevoire]

Feature name	Enable when launching	Enable at runtime	Description
Get logs through a route	./meilisearch --experimental-enable-logs-route	curl \ -X PATCH 'http://localhost:7700/experimental-features/' \ -H 'Content-Type: application/json' \ --data-binary '{ "logsRoute": true }'	Create a new route under `/logs/stream` that streams logs to the client

Meilisearch v1.7.0 (released on March 11, 2024) introduces a new route to get logs.

Experimental feature abstract

Enabling the experimental feature will make a new /logs/stream route available in Meilisearch that lets you configure which logs you want and their format.
Analyzing those logs can give great information on what's happening inside the engine and how the Meilisearch team can improve the indexing performances.
It also makes a new /logs/stderr route available in Meilisearch that lets you configure the logs you want on your console at runtime.

How to use the feature?

1. Enable the feature through the CLI parameter --experimental-enable-logs-route or call the PATCH /experimental-features passing "logsRoute": true.
2. Call the /logs/stream route with your parameters:

curl \
	-X POST http://localhost:7700/logs/stream \
	-H 'Content-Type: application/json' \
	--data-binary '{
	    "mode": "human",
            "target": "index_scheduler=trace"
    }'

3. When you stop listening on the route, Meilisearch should drop the connection automatically at some point, but that may take a while. If you want to stop the connection straight away, you can call the /logs/stream route again but with the DELETE verb:

curl \
        -X DELETE http://localhost:7700/logs/stream

---

Or you can simply update the log level of the logs outputted to stderr by sending:

curl \
	-X POST http://localhost:7700/logs/stderr \
	-H 'Content-Type: application/json' \
	--data-binary '{
            "target": "index_scheduler=trace"
    }'

---

The available modes are:

• human: Those are the same logs you see in the console.
• profile: This mode profiles the time spent in each instrumented function of meilisearch. It can help to understand performance issues

The target parameter lets you specify which parts of the code you want to inspect and is formatted like that: code_part=log_level,code_part=log_level. See the following valid examples:

• milli=trace,index_scheduler=info,actix_web=off
• milli is a valid shortcut to enable the info logs for milli
• trace is another shortcut to enable the trace level for everything
• If you don’t specify anything, info is used as the default value
• If you specify an empty string, a reminder of the format is returned in an error message

💡 More information here

Why is this feature not stable yet?

The main purpose of this route right now is to help people debug issues in production without restarting their instance and profile Meilisearch.
We expect the route to evolve over time as we discover issues and information we're missing.

🗣️ You are welcome to give feedback or ask any question on its usage; we are eager to collect feedback on the feature

---

A few notes on the feature:

• You can’t have two clients listening at the same time on the route of the logs
• To access the route, you need an API-key with at least the metrics.get action
• I noticed that streaming logs didn’t work with httpie. The client should support HTTP streams.

[sanders41]

I was experimenting with this and I think I had the same issue httpie has. It kind of works. With httpie if you run it with the --stream flag it looks frozen, but it is working. If you kill the stream with curl -X DELETE http://localhost:7700/logs/stream while httpie is running the logs will print after the stream stops. It seems to wait for the stream to end to print it all at once.

I'm not sure if any of this helps and I don't know anything you can do about it, but thought I'd pass on what I have found 😄

[sanders41]

I doubt this is helpful, and it for sure won't help with httpie, but in the interest of providing all the information I have this is the best work around I have come up with so far. It only prints the results after the sleep, but at least you get the results.

import asyncio

from httpx import AsyncClient, StreamError


async def sample_logs() -> None:
    async def process_stream(client: AsyncClient) -> None:
        payload = {"mode": "human", "target": "index_scheduler=info"}
        async with client.stream(
            "post",
            "logs/stream",
            json=payload,
            headers={"Content-Type": "application/json"},
        ) as response:
            if response.status_code != 200:
                a = await response.aread()
                raise StreamError(a.decode())
            async for line in response.aiter_lines():
                print(line)

    async with AsyncClient(
        base_url="http://127.0.0.1:7700",
        headers={"Authorization": "Bearer masterKey"},
        timeout=None,
    ) as client:
        task = asyncio.create_task(process_stream(client))
        await asyncio.sleep(10)
        await client.delete("logs/stream")
        await task


async def main() -> int:
    await sample_logs()

    return 0


if __name__ == "__main__":
    raise SystemExit(asyncio.run(main()))

[irevoire]

Oh yes, nice to know, thanks!

[macraig]

Update: v1.7 has been released 🐇