Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Core feature] pyflyte run should support a simple json/yaml as input for all parameters #5365

Open
2 tasks done
kumare3 opened this issue May 15, 2024 · 3 comments
Open
2 tasks done
Labels
backlogged For internal use. Reserved for contributor team workflow. enhancement New feature or request flytekit FlyteKit Python related issue

Comments

@kumare3
Copy link
Contributor

kumare3 commented May 15, 2024

Motivation: Why do you think this is important?

Today pyflyte run supports passing json/yaml for dataclass/pydantic or json type arguments. But, often times folks just want to pass various parameters are one json/yaml.

This should be Optional It should continue to support existing parameters

Goal: What should the final outcome look like, ideally?

For a workflow like

@workflow
def foo(x : int, y: str, j: Optional[str]) -> ...
      ...

It should be possible to create a yaml like

x: 1
y: hello

input.yaml

and then pass it using

pyflyte run test.py wf --flyte-inputs input.[yaml | json]

Important to note

pyflyte run test.py wf --help

should list all required and non-required parameters. Also required parameters should be checked before calling the remote.

Describe alternatives you've considered

NA

Propose: Link/Inline OR Additional context

No response

Are you sure this issue hasn't been raised already?

  • Yes

Have you read the Code of Conduct?

  • Yes
@kumare3 kumare3 added enhancement New feature or request untriaged This issues has not yet been looked at by the Maintainers and removed untriaged This issues has not yet been looked at by the Maintainers labels May 15, 2024
Copy link

runllm bot commented May 15, 2024

Flyte's pyflyte run command now supports passing structured YAML or JSON input files for workflow execution, simplifying the process of providing multiple parameters. This feature allows users to define workflow inputs in a single YAML or JSON file and pass it during execution.

Example:

# input.yaml
x: 1
y: hello

Command:

pyflyte run test.py wf --flyte-input-yaml input.yaml

Would you like more detail?

Sources:


This is an AI-generated response and your feedback is appreciated! Please leave a 👍 if this is helpful and 👎 if it is not.

Mention @RunLLM to ask follow-up questions. See runllm.com for more!

@kumare3
Copy link
Contributor Author

kumare3 commented May 15, 2024

@RunLLM The given link handles yaml/json for singular inputs. I am suggesting one input field for all inputs as an option

Copy link

runllm bot commented May 15, 2024

Flyte's pyflyte run currently supports passing JSON/YAML for specific input types, but not for consolidating all parameters into a single file. Your proposal aims to simplify this by allowing a single YAML/JSON file to define all workflow inputs.

Would you like more detail?

Sources:


This is an AI-generated response and your feedback is appreciated! Please leave a 👍 if this is helpful and 👎 if it is not.

Mention @RunLLM to ask follow-up questions. See runllm.com for more!

@eapolinario eapolinario added flytekit FlyteKit Python related issue backlogged For internal use. Reserved for contributor team workflow. labels May 16, 2024
@andrewwdye andrewwdye added backlogged For internal use. Reserved for contributor team workflow. and removed backlogged For internal use. Reserved for contributor team workflow. labels May 16, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backlogged For internal use. Reserved for contributor team workflow. enhancement New feature or request flytekit FlyteKit Python related issue
Projects
None yet
Development

No branches or pull requests

3 participants