Conversation
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## main #3213 +/- ##
==========================================
+ Coverage 78.37% 78.43% +0.06%
==========================================
Files 675 680 +5
Lines 55600 55978 +378
Branches 728 728
==========================================
+ Hits 43574 43909 +335
- Misses 11948 11991 +43
Partials 78 78
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
clayandgen
left a comment
clayandgen
left a comment
There was a problem hiding this comment.
The eslint checks look real, would be good to address!
Move raw-log reingest from Bucket Explorer into the Data Gap Analysis dialog as a per-row Repair action, running async via ReingestJob. The new DecomCommon module lets reingest skip check_limits on historical data, groups files by embedded target hash, and falls back to 'current' when the historical archive is missing. Metrics tab gains a configurable data-gap sample interval. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
clayandgen
left a comment
clayandgen
left a comment
There was a problem hiding this comment.
Thanks for splitting up the Vue files! Some further comments as the PR has grown...
- reingest_job: validate and inline target extraction from path; raise ReingestJobError when the path doesn't encode a target so the job marks Crashed with a clear message instead of ingesting against stale global System state and silently marking Complete - storage_controller: reject non-identifier scope/target in repair_candidates so the S3 key prefix can't be reshaped by path separators or control characters (admin-only, defense in depth) - tsdb tabs: extract the shared execSql helper into tsdbApi.js; TsdbMetrics and TsdbQueries import it instead of duplicating the retry logic 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
![aikido-pr-checks[bot]](https://avatars.githubusercontent.com/in/898896?s=60&v=4){kind=small}
| bucket_client.get_object(bucket: bucket_name, key: key, path: temp_file) | ||
| if File.extname(filename) == '.gz' | ||
| decompressed = BucketUtilities.uncompress_file(temp_file) | ||
| File.delete(temp_file) |
There was a problem hiding this comment.
Path traversal attack possible - medium severity
A malicious actor could control the location of this file, which may allow them to retrieve, write or delete files outside of the intended folder.
Show fix
Remediation: To address this, ensure that user-controlled variables in file paths are sanitized at least to not contain '..' or forward slashes.
Reply @AikidoSec ignore: [REASON] to ignore this issue.
More info
|
To test you can run for a while and then disconnect INST or INST2. Then go into the TSDB Queries tab and issue the following (this is easier in the full QuestDB console if you enable that): Determine a good time range based on the above result and issue the following after changing the time range: Reconnect INST and INST2 and graph INST HEALTH_STATUS TEMP1 with 1hr history. You'll see a 10min gap (using the values above). 
Now you can go into the TSDB Metrics and perform a data gap analysis. You'll see the gap dialog and you can restore with the wrench. 
It will prompt which raw files to re-import to cover the time period: 
After processing and waiting for the DEDUP cooldown: 
Clean graph! 
|
clayandgen
left a comment
clayandgen
left a comment
There was a problem hiding this comment.
This is awesome, and works great when I tried it! One small comment might be that the last dialog closes easily (clicking away from it). It could be good for that last dialog (waiting for dedup cooldown) to not close so easily so users can monitor it
|
3 participants
Calculate metrics across the entire TSDB:
Queries remains:
closes #3211