Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add support for multiquery in ts with limit #2970

Merged
merged 26 commits into from
Jul 16, 2023
Merged

Conversation

nityanandagohain
Copy link
Member

@nityanandagohain nityanandagohain commented Jun 23, 2023

Fixes #2845

  • two queries are created pre and main
  • during execution if main is found then pre is executed and the result is added to main query and then main query is executed.

test curl

curl --location 'localhost:8080/api/v3/query_range' \
--header 'Content-Type: application/json' \
--data '{
    "start": 1680712088000000000,
    "end": 1690771611000000000,
    "step": 60,
    "dataSource": "logs",
    "compositeQuery": {
        "queryType": "builder",
        "panelType": "graph",
        "limit": 1,
        "builderQueries": {
            "A": {
                "stepInterval": 60,
                "queryName": "A",
                "dataSource": "logs",
                "aggregateOperator": "count",
                "filters": {},
                "limit": 3,
                "groupBy": [
                    {
                        "key": "method",
                        "isColumn": true
                    }
                ],
                "expression": "A",
                "disabled": false
            }
        }
    }
}'

@github-actions github-actions bot added the enhancement New feature or request label Jun 23, 2023
@nityanandagohain nityanandagohain marked this pull request as ready for review July 7, 2023 11:06
@nityanandagohain
Copy link
Member Author

Have added support for multiple group by

@srikanthccv
Copy link
Member

https://clickhouse.com/docs/en/sql-reference/operators/in

The left side of the operator is either a single column or a tuple.

My suggestion for the growing complexity of this is to use a subquery to get the tuple of columns you want to filter on and then use that in the outer query. You will get the same benefit as the two-query approach but without the complexity. You don't have to worry about the subquery affecting performance because it will only be executed once. And don't need to worry about the distributed setup (GLOBAL IN) either because IN doesn't involve a separate table; instead, it is a tuple of columns. If anything, it should be better than the two-query because we eliminate the query-service from the equation.

The one query equivalent of the two-query approach is:

SELECT
    toStartOfInterval(fromUnixTimestamp64Nano(timestamp), toIntervalSecond(60)) AS ts,
    method AS method,
    severity_number AS severity_number,
    toFloat64(count(*)) AS value
FROM signoz_logs.distributed_logs
WHERE timestamp >= 1680712088000000000 AND timestamp <= 1690771611000000000 AND (method, severity_number) IN (
    SELECT
        method,
        severity_number
    FROM
    (
        SELECT
            method AS method,
            severity_number AS severity_number,
            toFloat64(count(*)) AS value
        FROM signoz_logs.distributed_logs
        WHERE (timestamp >= 1680712088000000000) AND (timestamp <= 1690771611000000000)
        GROUP BY
            method,
            severity_number
        ORDER BY value ASC
    )
    LIMIT 1
)
GROUP BY
    method,
    severity_number,
    ts
ORDER BY
    method ASC,
    severity_number ASC,
    ts ASC

@srikanthccv
Copy link
Member

You will still use the part of work you worked on already; instead of running queries and the preparing addition part of where you can replace it with a subquery.

@nityanandagohain
Copy link
Member Author

Have added new changes where if there is no orderBy provided

  • For table and TS -> add value DESC as orderBy
  • For listview -> add timestamp DESC as orderBy

If an order by key is provided.

  • For table, ts and listview -> use key provided for ordering.

@nityanandagohain nityanandagohain merged commit 7f9ba6c into develop Jul 16, 2023
7 of 8 checks passed
@nityanandagohain nityanandagohain deleted the issue_2845 branch July 16, 2023 17:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add support for limit on Time series
3 participants