Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Replace Agent State with Agent Phase #2123

Merged
merged 15 commits into from
Feb 5, 2024

Conversation

Future-Outlier
Copy link
Member

@Future-Outlier Future-Outlier commented Jan 21, 2024

Tracking issue

Fixes flyteorg/flyte#3936

Why are the changes needed?

  1. We want to deprecate State in webapi and use core.TaskExecution.Phase instead to make users less confused.
  2. We want to print the agent's metadata when using local execution continuously.

What changes were proposed in this pull request?

  • Change variables and functions below.
  1. cur_state -> cur_phase
  2. return Resource(state=...) -> return Resource(phase=...)
  3. admin state -> propeller phase
  4. mmcloud_status_to_flyte_state -> mmcloud_status_to_flyte_phase
  5. convert_to_flyte_state -> convert_to_flyte_phase

How was this patch tested?

  • local agent task execution
  • remote agent task execution
  • unit test

Setup process

  1. start agent server by flytekit
  2. start flyte components with --dev
  3. compile and start the flyte cluster
  4. run a remote task with sensor agent
pyflyte serve agent
flytectl demo start --dev
cd flyte
make compile;
flyte start --config ../flyte-single-binary-local-dev.yaml
pyflyte run --remote sensor_example.py wf
tasks:
  task-plugins:
    enabled-plugins:
      - agent-service

plugins:
  agent-service:
    supportedTaskTypes:
      - sensor
    defaultAgent:
      endpoint: "dns:///localhost:8000"
      insecure: true
      timeouts:
        GetTask: 100s
      defaultTimeout: 100s
from flytekit.sensor.file_sensor import FileSensor
from flytekit import ImageSpec, task, workflow
sensor = FileSensor(name="test_sensor")

@task()
def t1():
    return

@workflow()
def wf():
    sensor(path="/tmp") >> t1()

if __name__ == "__main__":
    wf()

Screenshots

  • Remote Execution
    image

  • Local Execution
    image

Check all the applicable boxes

  • I updated the documentation accordingly.
  • All new and existing tests passed.
  • All commits are signed-off.

Related PRs

flyteorg/flyte#4738

Signed-off-by: Future-Outlier <eric901201@gmail.com>
Signed-off-by: Future-Outlier <eric901201@gmail.com>
Copy link

codecov bot commented Jan 21, 2024

Codecov Report

Attention: 10 lines in your changes are missing coverage. Please review.

Comparison is base (b230f3e) 85.79% compared to head (de4675e) 85.57%.
Report is 6 commits behind head on master.

Files Patch % Lines
flytekit/extend/backend/base_agent.py 73.33% 2 Missing and 2 partials ⚠️
.../flytekit-airflow/flytekitplugins/airflow/agent.py 50.00% 4 Missing ⚠️
...lytekit-bigquery/flytekitplugins/bigquery/agent.py 83.33% 1 Missing ⚠️
...tekit-snowflake/flytekitplugins/snowflake/agent.py 83.33% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master    #2123      +/-   ##
==========================================
- Coverage   85.79%   85.57%   -0.23%     
==========================================
  Files         313      308       -5     
  Lines       23628    23113     -515     
  Branches     3541     3540       -1     
==========================================
- Hits        20272    19779     -493     
+ Misses       2744     2722      -22     
  Partials      612      612              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Signed-off-by: Future-Outlier <eric901201@gmail.com>
@Future-Outlier
Copy link
Member Author

image

Signed-off-by: Future-Outlier <eric901201@gmail.com>
Signed-off-by: Future-Outlier <eric901201@gmail.com>
@Future-Outlier Future-Outlier marked this pull request as ready for review January 27, 2024 07:01
pingsutw
pingsutw previously approved these changes Jan 31, 2024
phase = res.resource.phase

progress.print(f"Task phase: {TaskExecution.Phase.Name(phase)}, Phase message: {res.resource.message}")
if hasattr(res.resource, "log_links"):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do we really new hasattr here?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do we really new hasattr here?

will remove it, thank you

Signed-off-by: Future-Outlier <eric901201@gmail.com>
Future-Outlier and others added 3 commits January 31, 2024 21:05
Signed-off-by: Future-Outlier <eric901201@gmail.com>
Signed-off-by: Kevin Su <pingsutw@gmail.com>
Signed-off-by: Kevin Su <pingsutw@gmail.com>
@pingsutw pingsutw changed the title Deprecated Agent State to Agent Phase Replace Agent State to Agent Phase Feb 1, 2024
@pingsutw pingsutw changed the title Replace Agent State to Agent Phase Replace Agent State with Agent Phase Feb 1, 2024
Signed-off-by: Kevin Su <pingsutw@gmail.com>
Signed-off-by: Kevin Su <pingsutw@gmail.com>
Signed-off-by: Kevin Su <pingsutw@gmail.com>
Signed-off-by: Kevin Su <pingsutw@gmail.com>
Signed-off-by: Kevin Su <pingsutw@gmail.com>
@pingsutw pingsutw merged commit e7accce into master Feb 5, 2024
79 of 84 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
2 participants