⚠️ [klogs] Use maps instead of columns for custom fields #437
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
| // to render the distribution chart, which shows how many documents/rows are available within a bucket. | ||
| sqlQueryBuckets := fmt.Sprintf(`SELECT toStartOfInterval(timestamp, INTERVAL %d second) AS interval_data, count(*) AS count_data FROM %s.logs WHERE timestamp >= FROM_UNIXTIME(%d) AND timestamp <= FROM_UNIXTIME(%d) %s GROUP BY interval_data ORDER BY interval_data WITH FILL FROM toStartOfInterval(FROM_UNIXTIME(%d), INTERVAL %d second) TO toStartOfInterval(FROM_UNIXTIME(%d), INTERVAL %d second) STEP %d SETTINGS skip_unavailable_shards = 1`, interval, i.database, timeStart, timeEnd, conditions, timeStart, interval, timeEnd, interval, interval) | ||
| log.Debug(ctx, "SQL query buckets", zap.String("query", sqlQueryBuckets)) | ||
| rowsBuckets, err := i.client.QueryContext(ctx, sqlQueryBuckets) |
Check failure
Code scanning / CodeQL
Database query built from user-controlled sources
| // timestamp field and limiting the results / using a offset for pagination. | ||
| sqlQueryRawLogs := fmt.Sprintf("SELECT %s FROM %s.logs WHERE (%s) %s ORDER BY %s LIMIT %d SETTINGS skip_unavailable_shards = 1", defaultColumns, i.database, timeConditions, conditions, parsedOrder, limit) | ||
| log.Debug(ctx, "SQL query raw logs", zap.String("query", sqlQueryRawLogs)) | ||
| rowsRawLogs, err := i.client.QueryContext(ctx, sqlQueryRawLogs) |
Check failure
Code scanning / CodeQL
Database query built from user-controlled sources
ricoberger
force-pushed
the
klogs-use-maps-instead-of-arrays-for-custom-fields
branch
2 times, most recently
from
ebef360
to
a128579
Compare
ricoberger
changed the title
This is a follow up for kobsio/klogs#34 were we changed the database schema for klogs to use maps instead of arrays for custom fields. To be able to query the data ingested by the klogs plugin we had to adjust the queries to also use maps instead of columns for these fields.
ricoberger
force-pushed
the
klogs-use-maps-instead-of-arrays-for-custom-fields
branch
from
a128579
to
3c0377b
Compare
ricoberger
deleted the
klogs-use-maps-instead-of-arrays-for-custom-fields
branch
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This is a follow up for kobsio/klogs#34 were we changed the database schema for klogs to use maps instead of arrays for custom fields.
To be able to query the data ingested by the klogs plugin we had to adjust the queries to also use maps instead of columns for these fields.
docs/plugins/sql.mdplugins/plugin-istio/pkg/instance/instance.go