[DB] Align run start_time column and body field
#1607
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Hedingber
changed the title
start_time column and body field
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Fixes: https://jira.iguazeng.com/browse/ML-145
When storing run the logic was that on creation the start time column is either coming from the body or
datetime.now()(by that precedence), and on update, nothing happens with it.When running a job the first the triggering client is storing the run, it's the first store, so it's creation, no start time is in the body, therefore the time is just now. but then, when the job starting, on the
MLClientCtxinitialization, the start time is being set to now, and then when the job is finishing, the context store the run again, this time causing an update, but now the column doesn't change, while the body is being changed to have a different time (it takes some time for the pod to come up and actually start running the code)So bottomline the
start_timecolumn and the field inside the body were inconsistent.After consulting with @yaronha we agreed that it does make more sense to have the time from when the code execution actually started (without considering the time to pull the image and such) as the start time, therefore we needed to make sure that on update, if something is in the body, it applies to the column as well (we do keep defaulting to now on creation as some runtimes won't have any MLRun context used in them)
In a followup PR I'll add a data migration that will go over all runs and align the column to have the body field if it exists