Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

ZenServer #879

Merged
merged 791 commits into from Oct 2, 2022
Merged

ZenServer #879

merged 791 commits into from Oct 2, 2022

Conversation

fa9r
Copy link
Contributor

@fa9r fa9r commented Sep 7, 2022 •
edited

Describe changes

  • ZenML can now run as a server that can be accessed via REST API and comes with a visual user interface. This server can be deployed in arbitrary environments (local, on-prem, via Docker, on AWS / GCP / Azure / ...) and supports user management, project scoping, and more.
  • All metadata is now stored, tracked, and managed by ZenML itself.
  • Stack components can now be registered without having the required integrations installed. As part of this change, we split all existing stack component definitions into three classes: An implementation class that defines the logic of the stack component, a config class that defines the attributes and performs input validations, and a flavor class that links implementation and config classes together. See component flavor models #895 for more details.
  • To further improve reproducibility, pipelines themselves are now tracked in ZenML and exposed as first-level citizens. Each pipeline clearly defines what steps are used in the pipeline and in what order the steps are executed. By default, pipeline runs are now scoped by pipeline.

As part of these changes, the following concepts have been removed:

  • Metadata Stores,
  • Profiles,
  • Local YAML ZenStores

New CLI commands:

  • zenml server connect / disconnect / down / up / explain / list / logs / status
  • zenml config describe / explain / set
  • zenml pipeline list / runs / delete
  • zenml profile list / migrate

List of other PRs that have become part of this one:

Pre-requisites

Please ensure you have done the following:

  • I have read the CONTRIBUTING.md document.
  • If my change requires a change to docs, I have updated the documentation accordingly.
  • If I have added an integration, I have updated the integrations table and the corresponding website section.
  • I have added tests to cover my changes.

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)
  • Other (add details above)

@review-notebook-app
Copy link

Check out this pull request on  ReviewNB

See visual diffs & provide feedback on Jupyter Notebooks.

Powered by ReviewNB

@github-actions
Copy link
Contributor

github-actions bot commented Sep 8, 2022

@schustmi, @bcdurak
1.25 (one and twenty-five hundredths) business days have passed since the review started. Pretty please review the PR.

16 similar comments
@github-actions
Copy link
Contributor

github-actions bot commented Sep 9, 2022

@schustmi, @bcdurak
1.25 (one and twenty-five hundredths) business days have passed since the review started. Pretty please review the PR.

@github-actions
Copy link
Contributor

github-actions bot commented Sep 9, 2022

@schustmi, @bcdurak
1.25 (one and twenty-five hundredths) business days have passed since the review started. Pretty please review the PR.

@github-actions
Copy link
Contributor

@schustmi, @bcdurak
1.25 (one and twenty-five hundredths) business days have passed since the review started. Pretty please review the PR.

@github-actions
Copy link
Contributor

@schustmi, @bcdurak
1.25 (one and twenty-five hundredths) business days have passed since the review started. Pretty please review the PR.

@github-actions
Copy link
Contributor

@schustmi, @bcdurak
1.25 (one and twenty-five hundredths) business days have passed since the review started. Pretty please review the PR.

@github-actions
Copy link
Contributor

@schustmi, @bcdurak
1.25 (one and twenty-five hundredths) business days have passed since the review started. Pretty please review the PR.

@github-actions
Copy link
Contributor

@schustmi, @bcdurak
1.25 (one and twenty-five hundredths) business days have passed since the review started. Pretty please review the PR.

@github-actions
Copy link
Contributor

@schustmi, @bcdurak
1.25 (one and twenty-five hundredths) business days have passed since the review started. Pretty please review the PR.

@github-actions
Copy link
Contributor

@schustmi, @bcdurak
1.25 (one and twenty-five hundredths) business days have passed since the review started. Pretty please review the PR.

@github-actions
Copy link
Contributor

@schustmi, @bcdurak
1.25 (one and twenty-five hundredths) business days have passed since the review started. Pretty please review the PR.

@github-actions
Copy link
Contributor

@schustmi, @bcdurak
1.25 (one and twenty-five hundredths) business days have passed since the review started. Pretty please review the PR.

@github-actions
Copy link
Contributor

@schustmi, @bcdurak
1.25 (one and twenty-five hundredths) business days have passed since the review started. Pretty please review the PR.

@github-actions
Copy link
Contributor

@schustmi, @bcdurak
1.25 (one and twenty-five hundredths) business days have passed since the review started. Pretty please review the PR.

@github-actions
Copy link
Contributor

@schustmi, @bcdurak
1.25 (one and twenty-five hundredths) business days have passed since the review started. Pretty please review the PR.

@github-actions
Copy link
Contributor

@schustmi, @bcdurak
1.25 (one and twenty-five hundredths) business days have passed since the review started. Pretty please review the PR.

@github-actions
Copy link
Contributor

@schustmi, @bcdurak
1.25 (one and twenty-five hundredths) business days have passed since the review started. Pretty please review the PR.

@github-actions
Copy link
Contributor

@schustmi, @bcdurak
1.25 (one and twenty-five hundredths) business days have passed since the review started. Pretty please review the PR.

1 similar comment
@github-actions
Copy link
Contributor

@schustmi, @bcdurak
1.25 (one and twenty-five hundredths) business days have passed since the review started. Pretty please review the PR.

@fa9r fa9r marked this pull request as ready for review September 21, 2022 19:39
@fa9r fa9r marked this pull request as draft September 21, 2022 19:40
@fa9r fa9r marked this pull request as ready for review September 21, 2022 19:40
@github-actions
Copy link
Contributor

@schustmi, @bcdurak
1.25 (one and twenty-five hundredths) business days have passed since the review started. Pretty please review the PR.

3 similar comments
@github-actions
Copy link
Contributor

@schustmi, @bcdurak
1.25 (one and twenty-five hundredths) business days have passed since the review started. Pretty please review the PR.

@github-actions
Copy link
Contributor

@schustmi, @bcdurak
1.25 (one and twenty-five hundredths) business days have passed since the review started. Pretty please review the PR.

@github-actions
Copy link
Contributor

@schustmi, @bcdurak
1.25 (one and twenty-five hundredths) business days have passed since the review started. Pretty please review the PR.

htahir1 and others added 24 commits October 1, 2022 23:11
@htahir1
Merge branch 'misc/edit-README' into doc/new-docs-post-zenserver
289965a
@stefannica
Fix minor issues with stack registration and profile migration
a5eea37
@htahir1
Merge remote-tracking branch 'origin/feature/ENG-1103-merging-metadat…
7178fe5 
…a-stores-and-zen-server' into doc/new-docs-post-zenserver
@htahir1
Overview
a9d9cf2
@htahir1
Merge branch 'doc/new-docs-post-zenserver' into doc/post-zenserver-de…
d089547 
…ployment-guide
@htahir1
Moving deployment
b209699
@htahir1
Removing deployment
e12fc81
@htahir1
Fixed link
bd398cb
@htahir1
intro
c44fef3
@htahir1
Intro reworked
88a8551
@htahir1
Reworking etting started
74090a3
@stefannica
Add dashboard installation script and removed dashboard files from repo
88eef22
@stefannica
Add dashboard install to release workflow and updated zenml-server do…
3cac93c 
…ckerfile
@htahir1
Intro finished
054e5c7
@htahir1
took out one paragraph
6636422
@htahir1
Picture moveD
da3d4db
@htahir1
Core concepts done
23cae22
@htahir1
Added guidelines and misc section
9d2f588
@htahir1
Migration story added
45eda59
@htahir1
Fixed roken links
13064b1
@stefannica
Fix the docker local dashboard deployment
f29e0f2
@htahir1
Added installation guide
5b2db6d
@stefannica
Allow the zen server to function without dashboard static files
52d027a
@htahir1
Merge remote-tracking branch 'origin/feature/ENG-1103-merging-metadat…
d211063 
…a-stores-and-zen-server' into doc/new-docs-post-zenserver
@htahir1 htahir1 merged commit ded2c0f into develop Oct 2, 2022
@htahir1 htahir1 deleted the feature/ENG-1103-merging-metadata-stores-and-zen-server branch October 2, 2022 15:01
@stefannica stefannica restored the feature/ENG-1103-merging-metadata-stores-and-zen-server branch October 2, 2022 22:04
@fa9r fa9r mentioned this pull request Oct 5, 2022
Merged
8 tasks
@fa9r fa9r deleted the feature/ENG-1103-merging-metadata-stores-and-zen-server branch October 13, 2022 08:15
@fa9r fa9r mentioned this pull request Sep 11, 2023
Merged
8 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@schustmi schustmi Awaiting requested review from schustmi

@bcdurak bcdurak Awaiting requested review from bcdurak

Assignees
No one assigned
Labels
breaking-change
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
8 participants
@fa9r @htahir1 @wjayesh @AlexejPenner @schustmi @stefannica @strickvl @bcdurak