Add CFR and WTF diagnostics programs #88
Conversation
Codecov ReportAttention:
Additional details and impacted files@@ Coverage Diff @@
## main #88 +/- ##
==========================================
- Coverage 84.93% 81.48% -3.45%
==========================================
Files 48 58 +10
Lines 1805 2285 +480
==========================================
+ Hits 1533 1862 +329
- Misses 272 423 +151
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. |
amotl
changed the title
cratedb-wtf diagnostics programcratedb-wtf diagnostics program
cratedb_toolkit/wtf/library.py
Outdated
| class Logs: | ||
| # TODO: Implement `tail` in one way or another. -- https://stackoverflow.com/q/4714975 | ||
| # SELECT * FROM sys.jobs_log OFFSET -10; | ||
| # SELECT * FROM sys.jobs_log OFFSET (SELECT count(*) FROM sys.jobs_log)-10; | ||
| # https://cratedb.com/docs/crate/reference/en/latest/general/builtins/scalar-functions.html#to-char-expression-format-string | ||
| # https://cratedb.com/docs/crate/reference/en/latest/general/builtins/scalar-functions.html#date-format-format-string-timezone-timestamp | ||
| user_queries = """ | ||
| SELECT | ||
| DATE_FORMAT('%Y-%m-%dT%H:%i:%s.%f', started) AS started, | ||
| DATE_FORMAT('%Y-%m-%dT%H:%i:%s.%f', ended) AS ended, | ||
| classification, stmt, username, node | ||
| FROM | ||
| sys.jobs_log | ||
| WHERE | ||
| stmt NOT LIKE '%sys.%' AND | ||
| stmt NOT LIKE '%information_schema.%' | ||
| """ |
There was a problem hiding this comment.
Dear @hlcianfagna, @hammerhead, and @WalBeh,
I would like to implement tailing the sys.jobs_log table in one way or another. Actually, tail --follow. Did you manage to do that yet, using any kind of OFFSET/LIMIT magic?
SELECT * FROM sys.jobs_log OFFSET -10;
SELECT * FROM sys.jobs_log OFFSET (SELECT count(*) FROM sys.jobs_log)-10;Those statements outlined above failed for me. Is there any way to get the number of total records out of the subselect into the OFFSET parameter? Most probably, I am only too silly to make it work, so I am humbly asking for your support.
Cheers,
Andreas.
There was a problem hiding this comment.
SELECT CURRENT_TIMESTAMP AS last_timestamp,
(ended / 10000) * 10000 + 5000 AS ended_time,
COUNT(*) / 10.0 AS qps,
TRUNC(AVG(ended::bigint - started::bigint), 2) AS duration,
UPPER(regexp_matches(stmt, '^\s*(\w+).*') [1]) AS query_type
FROM sys.jobs_log
WHERE ended > now() - ('15 minutes')::interval
GROUP BY 1,
2,
5
ORDER BY ended_time ASCThis is the query used in the panel to show the latest 15 minutes QPS, perhaps it can serve as inspiration :P
There was a problem hiding this comment.
Thanks. So this is synthesizing the limiting by timestamp, compared to what is stored within the ended column? Hmm. Naturally I'd favor a more generic solution, but in this case, it could make an acceptable workaround. Sweet.
What's stored in ended when, well, the query has not finished yet? For emulating a tail -f, I'd probably better rely on started instead?
There was a problem hiding this comment.
It seems empty when it doesn't have a value.
Something like
Query ran at 2023-12-02T11:02:47.288Z on CrateDB 5.5.0
SELECT
DATE_FORMAT(started)
FROM
"sys"."jobs_log"
ORDER BY
started DESC
LIMIT
10QUERY OK, 10 record(s) returned in 0.0045s
| date_format(started) |
|---|
| "2023-12-02T11:02:40.890000Z" |
| "2023-12-02T11:02:40.825000Z" |
| "2023-12-02T11:02:40.822000Z" |
| "2023-12-02T11:02:40.819000Z" |
| "2023-12-02T11:02:40.819000Z" |
| "2023-12-02T11:02:40.819000Z" |
| "2023-12-02T11:02:40.819000Z" |
| "2023-12-02T11:02:35.892000Z" |
| "2023-12-02T11:02:35.821000Z" |
| "2023-12-02T11:02:35.821000Z" |
Seem to work just fine as tail -f
There was a problem hiding this comment.
To follow what is new I would keep track of the last started timestamp observed and query what is new
amotl
force-pushed
the
cratedb-wtf
branch
2 times, most recently
from
0e3ca31
to
002a255
Compare
amotl
force-pushed
the
cratedb-wtf
branch
3 times, most recently
from
3f53a92
to
2b2e8ff
Compare
|
Backlog: Check if the patch includes relevant details from here. |
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #88 +/- ##
===========================================
+ Coverage 62.11% 77.01% +14.89%
===========================================
Files 56 68 +12
Lines 1969 2684 +715
===========================================
+ Hits 1223 2067 +844
+ Misses 746 617 -129
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. |
amotl
force-pushed
the
cratedb-wtf
branch
6 times, most recently
from
2f96848
to
f0902b2
Compare
amotl
changed the title
cratedb-wtf diagnostics program
amotl
force-pushed
the
cratedb-wtf
branch
2 times, most recently
from
5f62d1a
to
c3222b3
Compare
amotl
force-pushed
the
cratedb-wtf
branch
2 times, most recently
from
4d3b461
to
0b3a47c
Compare
Add basic implementation for `sys-export` and `sys-import` subcommands. It is about exporting system tables of CrateDB into SQL DDL and JSONL files, and re-importing them for later analysis.
About
There are many smart queries to leverage information from CrateDB's internal
sys.*tables. This subsystem aims to bundle and collect them, in order to unlock easy access from CLI and HTTP interfaces, and to be able to use them as building blocks for other software components.Status
It is still a work in progress, and needs more attention and love. What can be done right now is outlined within the preview section. Any kind of help to expand this is much appreciated.
Documentation
Rendered in preview mode.
Both documents include directives to outline how operations work using Docker. However, the program can also be installed natively, also while not on PyPI yet. Please use one of those commands to acquire the software. When using Docker, make sure to use the
cratedb-toolkit:pr-88image on GHCR for preview purposes.Help
Backlog