Skip to content

Pending Changes for Active Stack #3372

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 8 commits into
base: develop
Choose a base branch
from

Conversation

preyasshah9
Copy link

@preyasshah9 preyasshah9 commented Feb 24, 2025

Describe changes

I implemented changes that were left-over as a part of #2370.

Pre-requisites

Please ensure you have done the following:

  • I have read the CONTRIBUTING.md document.
  • I have added tests to cover my changes.
  • I have based my new branch on develop and the open PR is targeting develop. If your branch wasn't based on develop read Contribution guide on rebasing branch to develop.
  • IMPORTANT: I made sure that my changes are reflected properly in the following resources:
    • ZenML Docs
    • Dashboard: Needs to be communicated to the frontend team.
    • Templates: Might need adjustments (that are not reflected in the template tests) in case of non-breaking changes and deprecations.
    • Projects: Depending on the version dependencies, different projects might get affected.

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)
  • Other (add details above)

@CLAassistant
Copy link

CLAassistant commented Feb 24, 2025

CLA assistant check
All committers have signed the CLA.

Copy link
Contributor

coderabbitai bot commented Feb 24, 2025

Important

Review skipped

Auto reviews are disabled on this repository.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share
🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>, please review it.
    • Explain this complex logic.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai explain this code block.
    • @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
    • @coderabbitai read src/utils.ts and explain its main purpose.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
    • @coderabbitai help me debug CodeRabbit configuration file.

Support

Need help? Create a ticket on our support page for assistance with any issues or questions.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (Invoked using PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai full review to do a full review from scratch and review all the files again.
  • @coderabbitai summary to regenerate the summary of the PR.
  • @coderabbitai generate docstrings to generate docstrings for this PR.
  • @coderabbitai generate sequence diagram to generate a sequence diagram of the changes in this PR.
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
  • @coderabbitai help to get help.

Other keywords and placeholders

  • Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
  • Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
  • Add @coderabbitai anywhere in the PR title to generate the title automatically.

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

Copy link
Contributor

@schustmi schustmi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is one major part missing here still: When running/building a pipeline for which this config attribute is set, we actually need to use the temporary_active_stack context manager to switch the stack for the run/build.

This will be in PipelineDefinion.build(...) and PipelineDefinition._run(...).

@@ -46,6 +47,7 @@ class PipelineRunConfiguration(
steps: Dict[str, StepConfigurationUpdate] = {}
settings: Dict[str, SerializeAsAny[BaseSettings]] = {}
tags: Optional[List[str]] = None
stack: Optional[Stack] = None
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should be an Optional[str] instead. The Stack object is not something that can be defined in a yaml file, which is usually how users specify attributes of this run configuration class. The value for stack will be the name or ID of a stack, which will then be passed to temporary_active_stack.

pipeline_id=pipeline_id,
code_repository=code_repository,
)
with temporary_active_stack(stack_name_or_id=run_config.stack):
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Read the value of stack from the PipelineRunConfiguration and if available use that within the temporary_active_stack context manager. Same below.

@preyasshah9 preyasshah9 requested a review from schustmi March 2, 2025 20:40
@preyasshah9
Copy link
Author

@schustmi Hi Michael,

Thank you for the feedback on PR. I have processed through code and tried to address your comment to best of my understanding. Let me know if this sounds good!
Let me know if this sounds good. If you have any pointers on how to test this change specifically, I would really appreciate this.

@preyasshah9 preyasshah9 force-pushed the active_stack_config_yaml branch from 25a491e to 0d059d3 Compare March 2, 2025 20:50
Copy link
Contributor

@schustmi schustmi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One thing I forgot: Could you also add this logic to temporarily set a different stack on the PipelineDefinition.create_run_template method?

As for testing this:

  • The test should first register a second stack (should probably just be default local orchestrator/artifact store)
  • The active stack should be the default one
  • We then configure any pipeline (just a single-step, do nothing pipeline should be fine) to run on a different stack.
  • The above bullet point can be done either via a config file, or by passing it directly to test_pipeline.with_options(stack=...). These should probably be two different test cases.
  • It should then run the pipeline run = test_pipeline(), and check whether the run ran on the configured stack. assert run.stack.id == ...
  • We can then also check that the stack gets correctly reset to the previously active stack: assert Client().active_stack.id == ...

I think examples for most of these can be found in tests/unit/pipelines/test_base_pipeline.py (e.g. running a pipeline, configuring from a file). Let me know if I can help in any other way

local_repo = code_repository_utils.find_active_code_repository()
code_repository = build_utils.verify_local_repository_context(
deployment=deployment, local_repo_context=local_repo
_from_config_file = self._parse_config_file(
Copy link
Contributor

@schustmi schustmi Mar 7, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not a huge fan of you having to do this manually before the compilation step in many places, but I also don't see any easy way around this. I'll refactor this (the entire config file logic is very messy right now) after this PR is merged.

@preyasshah9
Copy link
Author

PipelineDefinition.create_run_template

When create_run_template function is called, it is already within the context manager of temporary_active_stack:

template = pipeline_instance.create_run_template(name=name)

I can also add a context manager but I believe this case is already handled.

@preyasshah9 preyasshah9 marked this pull request as ready for review March 10, 2025 02:43
@schustmi
Copy link
Contributor

PipelineDefinition.create_run_template

When create_run_template function is called, it is already within the context manager of temporary_active_stack:

template = pipeline_instance.create_run_template(name=name)

I can also add a context manager but I believe this case is already handled.

Yep, but that is only available when using the CLI to create a run template, but not when doing this from purely in code. I think the better solution would be to remove the context manager from the CLI function, and instead pass the --stack CLI option via the newly introduced PipelineRunConfiguration.stack attribute if set.

@preyasshah9 preyasshah9 requested a review from schustmi March 10, 2025 22:55
@preyasshah9
Copy link
Author

@schustmi Added some unit tests as per your comment. When I ran locally, I did not see issues related to the tests I added. I don't think unit tests are auto-executed by CI here. I executed tests on my branch using: bash scripts/test-coverage-xml.sh

@preyasshah9
Copy link
Author

preyasshah9 commented Mar 17, 2025

PipelineDefinition.create_run_template

When create_run_template function is called, it is already within the context manager of temporary_active_stack:

template = pipeline_instance.create_run_template(name=name)

I can also add a context manager but I believe this case is already handled.

Yep, but that is only available when using the CLI to create a run template, but not when doing this from purely in code. I think the better solution would be to remove the context manager from the CLI function, and instead pass the --stack CLI option via the newly introduced PipelineRunConfiguration.stack attribute if set.

One last question. I did pass the stack option to the pipeline_instance.with_options. However, passing it to the create_run_template, there are a couple of calls this function is making:

deployment = self._create_deployment(**self._run_args, skip_schedule_registration=True)
return Client().create_run_template(name=name, deployment_id=deployment.id, **kwargs)

Here the create_run_template, but it is not possible (or required) to patch the stack. Let me know if this needs to be updated.

However, _create_deployment seems an important place that uses stack option. This calls the _compile function which initializes the PipelineRunConfiguration. But there is no easier way to pass it outside of the configuration file. The options passed from parent functions are hidden, and the options from config file are not overridden.

self._reconfigure_from_file_with_overrides(config_path=config_path)
.

Any suggestions?

@preyasshah9
Copy link
Author

@schustmi Friendly ping for the clarification :)

@preyasshah9
Copy link
Author

@schustmi It's been a while! Any feedback on the comment above?

* Update issue template configurations and URLs

* Auto-update of LLM Finetuning template

* Update .github/ISSUE_TEMPLATE/feature_report.yml

Co-authored-by: Alex Strick van Linschoten <strickvl@users.noreply.github.com>

* Update .github/ISSUE_TEMPLATE/feature_report.yml

Co-authored-by: Alex Strick van Linschoten <strickvl@users.noreply.github.com>

---------

Co-authored-by: GitHub Actions <actions@github.com>
Co-authored-by: Alex Strick van Linschoten <strickvl@users.noreply.github.com>
(cherry picked from commit b098798)
@schustmi
Copy link
Contributor

@preyasshah9 Sorry for the late comment, I lost track of this among so many ongoing things.

The code in the pipeline_definition.py file on develop has changed a bit, can you resolve those merge conflicts before we have another look please.

In general, the create_run_template method when being called from a python file by the user (not using the CLI function) will still not respect the stack. So yes you're right we might need to implement this generically in the _create_deployment method somehow.

htahir1 and others added 7 commits July 3, 2025 10:29
…zenml-io#3796)

* Created service accounts with least privilege permissions

* Update Vertex AI and AzureML permissions for security

* Add new GCP permission for accessing GCS artifacts

* Add GetLogEvents permission to custom SageMaker policy

(cherry picked from commit d2068df)
* Update best practices for upgrading ZenML servers

* Apply suggestions from code review

Co-authored-by: Alex Strick van Linschoten <strickvl@users.noreply.github.com>

* Update best practices for ZenML server upgrades

* Update run templates in production for ZenML Pro

* Add rescheduling pipelines for all users in UAT phase

* Recreate run templates in production using code patterns

* Update docs/book/how-to/manage-zenml-server/best-practices-upgrading-zenml.md

Co-authored-by: Alexej Penner <thealexejpenner@gmail.com>

* Update docs/book/how-to/manage-zenml-server/best-practices-upgrading-zenml.md

Co-authored-by: Alexej Penner <thealexejpenner@gmail.com>

* Add server migration steps for upgrading ZenML

* Update best practices for upgrading ZenML server

* Update ZenML upgrade best practices for smooth team workflow

* Added assets

* Update diagrams and images for best practices

* Optimised images with calibre/image-actions

* Added assets

* Optimised images with calibre/image-actions

* Update best practices steps with "Step" instead of "Rule".

* Update image placeholder in best practices guide

---------

Co-authored-by: Alex Strick van Linschoten <strickvl@users.noreply.github.com>
Co-authored-by: Alexej Penner <thealexejpenner@gmail.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
(cherry picked from commit 470cbba)
* initial

* more

* interim

* Update README.md to enhance clarity and detail on ZenML's capabilities

- Revised section titles for improved readability, changing "Stack-agnostic: keep your code the same" to "Write Code Once, Run Anywhere" and "Built for platform teams" to "Automatic Versioning for MLOps and LLMOps".
- Expanded descriptions to better articulate ZenML's functionality, including examples of code snippets for pipelines and steps.
- Added visual ideas to illustrate key features and workflows, enhancing the documentation's engagement and usability.
- Included new sections on integrating tools and building production-grade AI workflows, showcasing ZenML's versatility in MLOps and LLMOps contexts.

* Revise README.md to clarify ZenML's role in production AI

- Updated the section title from "The Hidden Complexity of Production AI" to "Beyond the Demo: The Reality of Production AI" to better reflect the content.
- Enhanced the description of challenges faced when transitioning from POC to production, emphasising the complexities of evaluation pipelines, document processing, cost tracking, and compliance.
- Added specific target audiences for ZenML, highlighting its benefits for platform teams, enterprises, and organisations in regulated industries.
- Removed outdated sections to streamline the document and improve focus on ZenML's orchestration capabilities.

* Update README.md to enhance clarity and functionality of ZenML pipelines

- Modified the `my_training_pipeline` function to return the trained model, improving the pipeline's utility.
- Updated the description of automatic versioning to include code formatting for `scikit-learn`, enhancing readability.
- Replaced the `Client().active_step_run.log_metadata` call with `log_metadata` for logging evaluation costs, streamlining the logging process.
- Renamed the `training_pipeline_with_integrations` to `llm_finetuning_pipeline` for better alignment with its purpose.
- Added an `on_failure` hook to the `llm_trainer` step, improving error handling in the pipeline.
- Ensured consistent formatting and clarity throughout the README for better user understanding.

* update README(s)

* More README updates

* README improvements

* remove zenml go

* Small update for LLM support

* update header

* use other version's line

* improve secrets section

* Remove excess

* Remove decision maker section

* Footer changes

* Update README

* Optimised images with calibre/image-actions

* Update the use case example so that it all connects

* Update README with improved MLOps messaging

* Add version 0.83.1 release notes to README

* Update latest release link in README.md

* Update README with new content about AI agent development

* Delete compliance, RAG pipeline, architecture, and orchestrator images

* Add images for problem and development lifecycle

* Optimised images with calibre/image-actions

---------

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: Hamza Tahir <htahir111@gmail.com>
Co-authored-by: Hamza Tahir <hamza@zenml.io>
(cherry picked from commit c049143)
@preyasshah9 preyasshah9 force-pushed the active_stack_config_yaml branch from eee6818 to e14dc69 Compare July 11, 2025 02:23
@preyasshah9 preyasshah9 changed the base branch from develop to main July 11, 2025 02:24
@preyasshah9 preyasshah9 changed the base branch from main to develop July 11, 2025 02:27
@preyasshah9 preyasshah9 changed the base branch from develop to main July 11, 2025 02:27
@preyasshah9
Copy link
Author

The merge conflicts have been resolved!

@preyasshah9 preyasshah9 changed the base branch from main to develop July 12, 2025 16:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants