Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add ability to associate run with git commit or comment #58

Closed
aslotte opened this issue Jun 7, 2020 · 8 comments · Fixed by #119
Closed

Add ability to associate run with git commit or comment #58

aslotte opened this issue Jun 7, 2020 · 8 comments · Fixed by #119
Assignees
Labels
enhancement New feature or request help wanted Extra attention is needed

Comments

@aslotte
Copy link
Owner

aslotte commented Jun 7, 2020

Is your feature request related to a problem? Please describe.
While out running (that's when my ideas come) I thought of Sammy's question on how we would know which code was used to train a specific model. Given that we currently would run an e.g. a GitHub Action on a training run, or when running locally, have no GitHub commit at all, I can see that we easily would lose track of which run/artifact belonged to what code commit, e.g. model pipeline.

Describe the solution you'd like
A simple solution for us to start with is to add two columns on the run entity, both optional.

  • GitHub commit hash
  • Comments field

This would allow us to add the GitHub commit hash to the run, and if running locally a comment on what e.g. what the changes are that one is trying. Ideally we would then be able to link the web client with the repo in which the model was trained so that you could click on the run and it would automatically bring up the Git commit (e.g. PR) for which this code was trained on.

Describe alternatives you've considered
Open for suggestions.

@aslotte aslotte added enhancement New feature or request help wanted Extra attention is needed labels Jun 7, 2020
@aslotte
Copy link
Owner Author

aslotte commented Jun 7, 2020

cc: @dcostea @lqdev @AnoojNair - would love your input on this one. Sammy, I can't tag you yet but I'll tag you in the associated issue for your awareness.

@ssa3512
Copy link
Contributor

ssa3512 commented Jun 7, 2020

Once #50 is merged assemby you should be able to use something along these lines to get the git hash using something like this:

typeof(MLOps.NET.MLLifeCycleManager).Assembly
    .GetCustomAttribute<AssemblyInformationalVersionAttribute>()
    .InformationalVersion

This returns 0.0.13-alpha+d3d9d32478 on the assembly from one of the packages generated from my testing setting up the git versioning

@aslotte
Copy link
Owner Author

aslotte commented Jun 7, 2020

That's super helpful, thanks for the heads-up @ssa3512

@lqdev
Copy link
Collaborator

lqdev commented Jun 7, 2020

Would this value (git hash/ build version) be surfaced up in the user program so that they can then log it in the metadata store?

@aslotte
Copy link
Owner Author

aslotte commented Jun 7, 2020

Ideally it would be set automatically on the run and they can provide a comment is they like

@AnoojNair
Copy link
Collaborator

AnoojNair commented Jun 7, 2020

Another way to achieve this would be to use tags. We can tag the repo whenever we get the best run? Then we don't store anything git specific in the dB.

This should be possible from GitHub actions? I am not sure

@AnoojNair AnoojNair reopened this Jun 7, 2020
@aslotte
Copy link
Owner Author

aslotte commented Jun 7, 2020

@AnoojNair I think we would need to store some kind of traceable artifact, since down the line we want to be able to look back from a model to see what data and code that was used. I'm thinking that we in the UI would be able to click a model and be taken straight to the repo and the comimt which contains the code used :)

@AnoojNair
Copy link
Collaborator

@AnoojNair I think we would need to store some kind of traceable artifact, since down the line we want to be able to look back from a model to see what data and code that was used. I'm thinking that we in the UI would be able to click a model and be taken straight to the repo and the comimt which contains the code used :)

Nice, that would be a good feature

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request help wanted Extra attention is needed
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants