importinto: add SHOW RAW IMPORT JOB(S) for machine-readable stats#66502
importinto: add SHOW RAW IMPORT JOB(S) for machine-readable stats#66502GMHDBJD wants to merge 11 commits intopingcap:masterfrom
Conversation
# Conflicts: # pkg/parser/parser.go
ti-chi-bot
bot
added
the
release-note
|
Review Complete Findings: 2 issues |
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: The full list of commands accepted by this bot can be found here. DetailsNeeds approval from an approver in each of these files:Approvers can indicate their approval by writing |
ti-chi-bot
bot
added
size/XXL
|
Hi @GMHDBJD. Thanks for your PR. PRs from untrusted users cannot be marked as trusted with I understand the commands that are listed here. DetailsInstructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. |
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## master #66502 +/- ##
================================================
+ Coverage 77.6746% 78.2425% +0.5678%
================================================
Files 2006 1936 -70
Lines 548722 536906 -11816
================================================
- Hits 426218 420089 -6129
+ Misses 120844 116376 -4468
+ Partials 1660 441 -1219
Flags with carried forward coverage won't be shown. Click here to find out more.
🚀 New features to boost your workflow:
|
![pantheon-ai[bot]](https://avatars.githubusercontent.com/in/2265341?s=60&v=4){kind=small}
pkg/executor/show.go
Outdated
| return err | ||
| } | ||
|
|
||
| loc := sctx.GetSessionVars().Location() |
There was a problem hiding this comment.
[P2] Unix timestamps can be incorrect/unstable across sessions due to timezone mismatch
Why: SHOW RAW IMPORT JOB(S) computes *_time_unix fields using the caller session's time_zone, but the underlying TIMESTAMP(6) data is loaded via an internal session that may have a different timezone. This causes GoTime(location).Unix() to produce incorrect Unix seconds that vary across sessions with different time_zone settings, breaking the machine-readable stats contract.
Evidence: pkg/executor/show.go:2878
loc := sctx.GetSessionVars().Location()
// ... later calls BuildRawImportJobStats(loc, info, runInfo)The internal session (taskManager.WithNewSession) doesn't sync timezone with the caller, and types.Time.GoTime(loc) interprets the stored timestamp in the provided loc, causing Unix second shifts.
Suggested fix: Compute Unix timestamps in SQL using UNIX_TIMESTAMP(create_time) or consistently convert using the internal session's timezone, not the caller's.
| } | ||
| // JobStatus is a machine-friendly contract returned by SHOW RAW IMPORT JOB(S). | ||
| // It is a type alias to keep the contract centralized in one place. | ||
| type JobStatus = importer.RawImportJobStats |
There was a problem hiding this comment.
[P2] Breaking change to public SDK API - importsdk.JobStatus type alias
Why: importsdk.JobStatus was changed from a standalone struct to a type alias of importer.RawImportJobStats, which removes/renames public fields (ResultMessage, SourceFileSize, time.Time timestamps). This breaks compilation for external SDK consumers relying on these fields.
Evidence: pkg/importsdk/model.go:79
type JobStatus = importer.RawImportJobStatsPreviously (commit 0f706ff), JobStatus was a struct with fields like SourceFileSize string, ResultMessage string, and time.Time timestamps. The new RawImportJobStats has different field names/types (e.g., SourceFileSizeBytes, ErrorMessage, unix timestamps).
Suggested fix: Keep the old JobStatus struct for backward compatibility, or introduce a new type (e.g., RawJobStatus) to avoid breaking downstream consumers.
There was a problem hiding this comment.
I think downstream updates should modify the corresponding code based on this update.
GMHDBJD
force-pushed
the
rawImportInto
branch
from
f13d405 to
128c41b
Compare
|
@GMHDBJD: The following tests failed, say
Full PR test history. Your PR dashboard. DetailsInstructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here. |
1 participant
What problem does this PR solve?
Issue Number: close #66461
Problem Summary:
Import SDK / Lightning import-into backend needs a stable, machine-friendly contract to poll IMPORT INTO job status/progress without parsing
SHOW IMPORT JOBSoutput.What changed and how does it work?
SHOW RAW IMPORT JOB <job_id>andSHOW RAW IMPORT JOBS [WHERE group_key = ...].Raw_Statsas a JSON contract (RawImportJobStats) with timestamps (unix seconds) and current-step progress/ETA.Check List
Tests
Side effects
Documentation
Release note
Please refer to Release Notes Language Style Guide to write a quality release note.