MLEM Release blog post #3575

aguschin · 2022-05-19T14:18:58Z

No description provided.

gatsby-cloud · 2022-05-19T14:23:41Z

Gatsby Cloud Build Report

dvc.org

🎉 Your build was successful! See the Deploy preview here.

Build Details

View the build logs here.

🕐 Build time: 1m

Performance

Lighthouse report

Metric	Score
Performance	🔶 60
Accessibility	💚 98
Best Practices	🔶 83
SEO	💚 93

🔗 View full report

aguschin · 2022-05-19T17:54:34Z

content/blog/2022-05-24-MLEM-release.md

+ML model registries give your team key capabilities:
+
+- Collect and organize model [versions] from different sources effectively,
+  preserving their data provenance and lineage information.
+- Share metadata including [metrics and plots][mp] to help use and evaluate
+  models.
+- A standard interface to access all your ML artifacts, from early-stage
+  [experiments] to production-ready models.
+- Deploy specific models on different environments (dev, shadow, prod, etc.)
+  without touching the applications that consume them.
+- For security, control who can manage models, and audit their usage trails.
+
+Many of these benefits are built into DVC: Your [modeling process] and
+[performance data][mp] become **codified** in Git-based <abbr>DVC
+repositories</abbr>, making it possible to reproduce and manage models with
+standard Git workflows (along with code). Large model files are stored
+separately and efficiently, and can be pushed to [remote storage] -- a scalable
+access point for [sharing].
+
+To make a Git-native registry (on top of DVC or not), one option is to use [GTO]
+(Git Tag Ops). It tags ML model releases and promotions, and links them to
+artifacts in the repo using versioned annotations. This creates abstractions for
+your models, which lets you **manage their lifecycle** freely and directly from
+Git.


This part I took from @jorgeorpinel's PR: #3333

If you want to reuse explanations from other places that's fine but rephrase them in your own words (the way you understand it). Blog posts should have a consistent author's voice IMO.

OK to have very small sections (ad admonition, a sentence or 2) copy/pasted between blog and docs.

content/blog/2022-05-24-MLEM-release.md

aguschin · 2022-05-19T18:01:06Z

@jendefig @jorgeorpinel @jurv11 would be glad to get some comments. I added my part of the text very quickly and this is WIP, so not sure you need to provide very detailed feedback for this iteration. Does the structure work? Do some examples seem irrelevant? Did I miss to demonstrate some big ideas down the road? Thanks!

aguschin · 2022-05-19T18:02:37Z

content/blog/2022-05-24-MLEM-release.md

+  We’re excited to announce the launch of our latest open source offering,
+  [MLEM](https://mlem.ai)! MLEM is a tool that automatically extracts meta
+  information like environment and frameworks from models and standardizes that
+  information into a human-readable format within Git. ML teams can then use the
+  model information for deployment into downstream production apps and services.
+  MLEM easily connects to solutions like Heroku to dramatically decrease model
+  deployment time.
+picture: 2022-05-24/mlem-rocket.png
+author: aguschin
+# commentsUrl: TODO
+tags:
+  - Machine Learning
+  - Deployment
+  - Model Registry
+  - MLOps
+---
+
+We built MLEM to address issues that MLOps teams have around managing model
+information as they move them from training and development to production and,
+ultimately, retirement. MLEM is meant to help teams automate the collection of
+information around how the model was trained, what the model is for, and
+operational requirements around deployment.
+
+Just like all our [other](https://dvc.org) [tools](https://cml.dev), MLEM uses
+your Git service to store model information and connects with CI/CD solutions
+for deployment (like Heroku). This Git-based model
+([one of our core philosophies](https://iterative.ai/why-iterative/)) aligns
+model operations and deployment with software development teams – information
+and automation is all based on familiar DevOps tools – so that deploying any
+model into production is that much faster.
+
+With MLEM, ML teams get:
+
+- Human-readable information about a model for search and documentation
+- One-step automated deployment across any cloud
+- Fast model registry setup based on Git


This part I took from @jurv11 doc

content/blog/2022-05-24-MLEM-release.md

jorgeorpinel · 2022-05-19T21:14:35Z

What's the concept behind the image? Dog in rocket "to the moon" looks a bit like some cryptocurrency meme. Maybe it's just my bias but that could be misleading.

jendefig · 2022-05-19T23:58:33Z

What's the concept behind the image?

MLEM takes the different models (on the mlem rocket ship with the mlem dog) and deliver to deploying to the different stars in space. It's too late to change image.

jendefig · 2022-05-20T00:00:07Z

~~Why has this not been deployed? @julieg18~~
Nevermind. I see it. Why is it at the top and not the bottom?

jorgeorpinel · 2022-05-20T00:06:59Z

@aguschin the structure makes sense

Intro: feel free to provide even more background and motivation if you want. You can add a link to jump to TL;DR ("With MLEM, ML teams get") if it gets long.
+ State that MLEM is a Python-specific tool (currently not explicit anywhere).
Model metadata codification: Not sure we should emphasize "human-readable" (really that depends on whether you're familiar with YAML). I think that the key aspect here is the special "magic" (ML framework integrations) to automatically capture all the relevant modeling context.
Run models anywhere: Good catchy phrase. Should we also use buzz word "productionize" though? That would include packaging & distributing, running in batch (ETL), containerize/cloud deploy, or serve directly -- all MLEM features. I don't think "to deploy" captures all of that.
Git-native model registry: This section doesn't really talk about MLEM.
The core registry features would be provided by GTO right? So should this be about making Git-based model catalogs (link to GTO+DVC use case) deployable? Although that could be redundant with 2....

Maybe this can be reduced to a single paragraph somewhere and wait for the GTO release post (is there one planned?) to go into details.

jorgeorpinel · 2022-05-20T00:14:52Z

Git-native model registry: This section doesn't really talk about MLEM.

p.s. I think I know what the issue is: we mention Git in the abstract and intro but never explain (in the codification section) that you can version .mlem files with Git, bringing you to GitOps. That context is a missing piece of the puzzle rn.

jendefig

This is looking really good! Added some thoughts/changes/questions/comments

content/blog/2022-05-24-MLEM-release.md

jendefig · 2022-05-20T00:10:16Z

content/blog/2022-05-24-MLEM-release.md

+information around how the model was trained, what the model is for, and
+operational requirements around deployment.
+
+Just like all our [other](https://dvc.org) [tools](https://cml.dev), MLEM uses


Instead of doing these two links, maybe we should send to... Ok nevermind. I thought we have a product page at iterative.ai, but it's just a drop - down. cc: @jurv11 @julieg18, we should add this to the website list if we don't have it on there yet. There's the pricing page which shows all the tools, but that's not where we would want to send people in this case.

content/blog/2022-05-24-MLEM-release.md

jendefig · 2022-05-20T00:27:45Z

content/blog/2022-05-24-MLEM-release.md

+[gitops]: https://www.gitops.tech/
+
+MLEM is a core building block for a Git-based ML model registry, together with
+other Iterative tools, like GTO and DVC.


GTO - other than at ODSC East and those that have found repo, we haven't really exposed GTO. We probably need more links/explanation/docs/repo pointing here.

ALso I'm realizing we need to address that in the image for Twitter....

Yes, I think we need something. I'm also thinking about a technical page that explains how to set up MLEM + GTO + DVC together.

content/blog/2022-05-24-MLEM-release.md

shcheklein

Nice work @aguschin !

Some comments:

Intro is very long - the whole screen of text that goes into explanations about Git, etc, etc w/o giving me fist even idea what the tool is about. My 2cs - start simpler with "With MLEM, ML teams get:", then some before / after side by side then some deployment magic. Explanations can go in the middle. A bit extreme, probably the best format is something in between :)
... codification - not sure this is the best, codification is still niche, probably better to avoid it, be more explicit or use that + explanation
DVC pipelines - I think if we want to include it - let's do a separate section at the end. Describe storage and pipelines. Otherwise it makes text too complicated, we can't expect people to know DVC, etc, etc
The main goal of MLEM is to provide you a single tool that enables any kind of model productionization scenarios. - why don't we mention this in the very beginning of the blog post?
Git-native - on the fence here on using it in the title 🤔
What's next - need to put an image, make it more actionable? Start - can be an emoji, etc ... can we make some competition or some viral thingy on Twitter here cc @jendefig ?

content/blog/2022-05-24-MLEM-release.md

shcheklein · 2022-05-20T16:56:17Z

content/blog/2022-05-24-MLEM-release.md

+
+With MLEM, ML teams get:
+
+- **Model metadata codification**: Human-readable information about a model for


it looks like codification is only for search and docs, but this undersells it. This meta-information is needed to deploy things in the first place, to build clients faster, etc? This is main purpose.

Ideally we can converge this into a single value prop - packaging models to deploy, everything else comes as a benefit on top?

otherwise we start with some philosophy, then we go into codification ... and only after we go into deployment ... and only after into model registry ... it feels it should be presenting things other way around - high level solution / value prop first, then goes into impl details and ... or at least they should come really close to each other

I hope it makes sense :) happy to brainstorm on this more if needed ...

If "codification" is to niche or technical maybe speak of the user benefits like "reliable, standard metadata".

Thank you for your review, @shcheklein!

It seems I can't distribute the first section paragraphs ("We built MLEM to address..." and "Just like all our..." and "Capturing model-specific...") anywhere except for the second section "Model metadata codification". At least in the current form. So I can try to rewrite those and move them to the rest of the document. But after addressing your other comments that may be not needed anymore. Please let me know WDYT.

I think we need to use "codify". It sounds great and explains what MLEM does with metainformation in a single word - that's good for the quicker explanations later. I've provided some description about codification right after the first word occurence. Do you think it's enough?

Removed DVC code examples.

I think this is addressed now.

"Productionize your models with MLEM in a Git-native way" maybe?

I put a picture with a dog asking for the stars for now :)

content/blog/2022-05-24-MLEM-release.md

jendefig · 2022-05-23T19:10:16Z

Is this ready for release @aguschin ?

aguschin · 2022-05-24T09:54:08Z

Yes, unless @shcheklein or @dmpetrov wants to provide some feedback. If you need this ASAP, I think it's ok to take it as is.

jendefig

Found some grammar/typos

content/blog/2022-05-24-MLEM-release.md

shcheklein · 2022-06-01T04:37:19Z

content/blog/2022-05-24-MLEM-release.md

+Docker Image, or export it as some special format (like `.onnx` which is coming
+soon).
+
+```shell


@iterative/websites do we have syntax highlighters ready for MLEM?

We have a Gatsby Cloud issue that is preventing us from merging #3396. It's already available on other websites.

@yathomasi, we can't use cli highlighter here also yet, right?

initial version

6a2c728

aguschin added 2 commits May 19, 2022 19:32

replace image with new version

e402293

adding code snippets

2c7f7ab

aguschin commented May 19, 2022

View reviewed changes

content/blog/2022-05-24-MLEM-release.md Outdated Show resolved Hide resolved

aguschin commented May 19, 2022

View reviewed changes

content/blog/2022-05-24-MLEM-release.md Outdated Show resolved Hide resolved

aguschin commented May 19, 2022

View reviewed changes

content/blog/2022-05-24-MLEM-release.md Outdated Show resolved Hide resolved

Update content/blog/2022-05-24-MLEM-release.md

b1b349e

restyled-io bot mentioned this pull request May 19, 2022

Restyle MLEM Release blog post #3576

Closed

fix

d13d00b

jendefig approved these changes May 20, 2022

View reviewed changes

aguschin self-assigned this May 20, 2022

aguschin added 2 commits May 20, 2022 17:13

working on the feedback

aa82450

some fixes

d6890ed

shcheklein reviewed May 20, 2022

View reviewed changes

content/blog/2022-05-24-MLEM-release.md Outdated Show resolved Hide resolved

shcheklein reviewed May 20, 2022

View reviewed changes

jendefig added the C: blog TEMPORARY Content of /blog label May 20, 2022

jendefig reviewed May 20, 2022

View reviewed changes

content/blog/2022-05-24-MLEM-release.md Outdated Show resolved Hide resolved

aguschin added 3 commits May 21, 2022 19:30

adding feedback

fd35882

trying to merge three items into a single value prop

2f1fa5c

update

e600882

change title and address renamings

5080bb9

shcheklein added A: docs Area: user documentation (gatsby-theme-iterative) p1-important Active priorities to deal within next sprints labels May 25, 2022

jendefig approved these changes May 31, 2022

View reviewed changes

jendefig reviewed May 31, 2022

View reviewed changes

content/blog/2022-05-24-MLEM-release.md Outdated Show resolved Hide resolved

jendefig reviewed May 31, 2022

View reviewed changes

content/blog/2022-05-24-MLEM-release.md Outdated Show resolved Hide resolved

shcheklein reviewed Jun 1, 2022

View reviewed changes

aguschin added 2 commits June 1, 2022 11:25

Jeny's and Ivan's feedback

123e278

typo

f8c008b

aguschin merged commit 1579a65 into master Jun 1, 2022

aguschin deleted the blog-mlem-release branch June 1, 2022 13:03

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

MLEM Release blog post #3575

MLEM Release blog post #3575

aguschin commented May 19, 2022

gatsby-cloud bot commented May 19, 2022 •

edited

Loading

aguschin May 19, 2022

jorgeorpinel May 19, 2022 •

edited

Loading

aguschin commented May 19, 2022

aguschin May 19, 2022

jorgeorpinel commented May 19, 2022 •

edited

Loading

jendefig commented May 19, 2022 •

edited by jorgeorpinel

Loading

jendefig commented May 20, 2022 •

edited

Loading

jorgeorpinel commented May 20, 2022 •

edited

Loading

jorgeorpinel commented May 20, 2022

jendefig left a comment

jendefig May 20, 2022

jendefig May 20, 2022

jendefig May 20, 2022

aguschin May 20, 2022

shcheklein left a comment

shcheklein May 20, 2022

jorgeorpinel May 21, 2022

aguschin May 22, 2022

jendefig commented May 23, 2022

aguschin commented May 24, 2022

jendefig left a comment

shcheklein Jun 1, 2022

yathomasi Jun 1, 2022

aguschin Jun 1, 2022


		With MLEM, ML teams get:

		- Model metadata codification: Human-readable information about a model for

MLEM Release blog post #3575

MLEM Release blog post #3575

Conversation

aguschin commented May 19, 2022

gatsby-cloud bot commented May 19, 2022 • edited Loading

Gatsby Cloud Build Report

Build Details

Performance

Lighthouse report

Choose a reason for hiding this comment

jorgeorpinel May 19, 2022 • edited Loading

Choose a reason for hiding this comment

aguschin commented May 19, 2022

Choose a reason for hiding this comment

jorgeorpinel commented May 19, 2022 • edited Loading

jendefig commented May 19, 2022 • edited by jorgeorpinel Loading

jendefig commented May 20, 2022 • edited Loading

jorgeorpinel commented May 20, 2022 • edited Loading

jorgeorpinel commented May 20, 2022

jendefig left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

shcheklein left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jendefig commented May 23, 2022

aguschin commented May 24, 2022

jendefig left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

gatsby-cloud bot commented May 19, 2022 •

edited

Loading

jorgeorpinel May 19, 2022 •

edited

Loading

jorgeorpinel commented May 19, 2022 •

edited

Loading

jendefig commented May 19, 2022 •

edited by jorgeorpinel

Loading

jendefig commented May 20, 2022 •

edited

Loading

jorgeorpinel commented May 20, 2022 •

edited

Loading