Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[DEVX-210]: YAML based workflow creation #175

Merged
merged 16 commits into from
Sep 22, 2023

Conversation

sainivedh
Copy link
Contributor

@sainivedh sainivedh commented Sep 20, 2023

Why

  • Enable users to create workflow with a simple YAML config script.
  • Also helps user to export existing workflow as a YAML

How

Flow Chart of Node Creation

image

Workflow Create

The Workflow Create function allows you to create a new workflow, specified by a yaml config file.

# Note: CLARIFAI_PAT must be set as env variable.
from clarifai.client.app import App
app = App(app_id="app_id", user_id="user_id")
workflow = app.create_workflow(config_filepath="config.yml")

Workflow Export

The Workflow Export function allows you to export an existing workflow from Clarifai and save it to a local file for future use.

# Note: CLARIFAI_PAT must be set as env variable.
from clarifai.client.workflow import Workflow
workflow = Workflow("https://clarifai.com/clarifai/main/workflows/Demographics")
workflow.export('demographics_workflow.yml')

What

  • Added Workflow Validate, Export, Create functionalities
  • Added list module versions
  • Added display of workflow graph
workflow:
  id: test-wf-prompter
  nodes:
    - id: prompter
      model:
          model_id: prompter
          model_type_id: prompter
          description: 'Sentiment Analysis'
          output_info:
            params:
              prompt_template: 'Classify sentiment between postive and negative {data.text.raw}'

    - id: llm
      model:
          user_id: openai
          model_id: GPT-3_5-turbo
          app_id: chat-completion

      node_inputs:
        - node_id: prompter

image

Tests

  • Reused the same test cases as in yaml2workflow for export, create, validate
  • Create covers multiple scenarios as multi branch/multi node/custom model/public model...

@sainivedh sainivedh marked this pull request as ready for review September 20, 2023 15:08
README.md Outdated Show resolved Hide resolved
README.md Outdated Show resolved Hide resolved
clarifai/workflows/validate.py Show resolved Hide resolved
tests/workflow/test_export.py Outdated Show resolved Hide resolved
tests/workflow/test_create.py Outdated Show resolved Hide resolved
tests/workflow/test_create.py Outdated Show resolved Hide resolved
clarifai/client/app.py Outdated Show resolved Hide resolved
clarifai/utils/logging.py Show resolved Hide resolved
clarifai/client/app.py Outdated Show resolved Hide resolved
clarifai/client/app.py Outdated Show resolved Hide resolved
Copy link
Contributor

@sanjaychelliah sanjaychelliah left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@ghost ghost self-requested a review September 22, 2023 09:16
Copy link

@ghost ghost left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, nice work!

@sainivedh sainivedh merged commit 2a662e7 into master Sep 22, 2023
4 checks passed
@sainivedh sainivedh deleted the DEVX-210-YAML-Workflow-Creation branch September 22, 2023 09:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.

2 participants