Every time I use -f I get the RuntimeError: Query returns more than 10000 log entries. Use offsets to query in chunks.

[dlamblin]

This RuntimeError occurs not for the first printed output, but the second after a 1s wait.

Obviously its generated at this line.

bonfire/bonfire/graylog_api.py

Line 141 in d0af9ca

raise RuntimeError("Query returns more than 10000 log entries. Use offsets to query in chunks.")

Given that a -f can't specify a limit, what should I be doing to use it correctly?

[valentin-krasontovitsch]

yeah the follow mode is a bit problematic - i think right now it's just not taking into consideration that there might be more than 10000 results in one second!?

i would love to change the way it queries in follow mode (so instead of querying all matching logs from the next second interval, query say the next 100 matching log), but i'm not sure that it's possible.

in the meantime, a solution for querying huge dataloads in followmode might be to decrease the interval from one second to a tenth of a second, for instance?

On the other hand, is it really reasonable to query in follow-mode when you have more than 10000 logs per second? that's gotta be hard to follow on screen... perhaps you could adjust your query to get the results you really want to see?

[valentin-krasontovitsch]

perhaps this is an x y problem - closing for now, feel free to reopen with a detailed use-case that does not permit a work-around by just using the tool differently.