[BUG] Launch Plan Registered Twice

[jasonlai1218]

Describe the bug

I write a launch plan

from dev_python_dependency_deck import py_deps_deck_wf
from flytekit import LaunchPlan, CronSchedule


py_deps_deck_lp = LaunchPlan.get_or_create(
    py_deps_deck_wf,
    name="py_deps_deck_lp",
    schedule=CronSchedule(schedule="06 00 * * *"),
)

and I execute cmd: FLYTE_SDK_LOGGING_LEVEL=20 pyflyte register --non-fast -p flytesnacks --domain development dev_register_lp.py --version t1

then I found that the result was...

{"asctime": "2024-03-19 09:02:19,831", "name": "flytekit", "levelname": "INFO", "message": "Registering Sensor for task type: sensor_v0"}
Running pyflyte register from /Users/al02027975/Documents/jasonlai1218/flytekit with images ImageConfig(default_image=Image(name='default', fqn='cr.flyte.org/flyteorg/flytekit', tag='py3.11-latest'), images=[Image(name='default', fqn='cr.flyte.org/flyteorg/flytekit', tag='py3.11-latest')]) and image destination folder /root on 1 package(s) ('/Users/al02027975/Documents/jasonlai1218/flytekit/dev_register_lp.py',)
Registering against localhost:30080
Detected Root /Users/al02027975/Documents/jasonlai1218/flytekit, using this to create deployable package...
Loading packages ['dev_register_lp'] under source root /Users/al02027975/Documents/jasonlai1218/flytekit
Image jasonlai1218/flytekit:Lfx23vy2B9huZasN473ypQ not found. building...
Run command: envd build --path /var/folders/r7/cgsr7w_n4v148k8qbryjc6ym0000gp/T/flyte75ltk3sy/control_plane_metadata/local_flytekit/4b55674ab034d8c89e440cef577d0ca1 --platform linux/amd64 --output type=image,name=jasonlai1218/flytekit:Lfx23vy2B9huZasN473ypQ,push=true
#1 local://build-context
#1 DONE 0.0s
#2 [internal] setting pip cache mount permissions
#2 DONE 0.0s
#3 docker-image://cr.flyte.org/flyteorg/flytekit:py3.11-latest
...
...
...
#14 pushing layers
#14 pushing layers 16.9s done
#14 pushing manifest for docker.io/jasonlai1218/flytekit:Lfx23vy2B9huZasN473ypQ@sha256:54ef656aaf12784fdcaacd600079e0c024271e39ab81b4f925d35e785c0e1164
#14 pushing manifest for docker.io/jasonlai1218/flytekit:Lfx23vy2B9huZasN473ypQ@sha256:54ef656aaf12784fdcaacd600079e0c024271e39ab81b4f925d35e785c0e1164 0.7s done
#14 DONE 23.3s
Successfully serialized 4 flyte objects
[✔] Registration dev_python_dependency_deck.python_dependency_deck type TASK successful with version t1
[✔] Registration dev_python_dependency_deck.py_deps_deck_wf type WORKFLOW successful with version t1
[✔] Registration dev_python_dependency_deck.py_deps_deck_wf type LAUNCH_PLAN successful with version t1
[✔] Registration py_deps_deck_lp type LAUNCH_PLAN successful with version t1
Successfully registered 4 entities

I don't know why there are two LAUNCH_PLAN

Expected behavior

I think only one is correct

[✔] Registration py_deps_deck_lp type LAUNCH_PLAN successful with version t1

Additional context to reproduce

1. write a workflow and task

import flytekit
from flytekit import task, workflow, ImageSpec
from flytekit.deck.deck import PythonDependencyDeck

flytekit_version = "git+https://github.com/jasonlai1218/flytekit.git@add-py-deps-deck#commit=341a7dab611e31023ae82628f5aad5792b7c5d51"
deck_plugin_version = "git+https://github.com/jasonlai1218/flytekit.git@add-py-deps-deck#commit=341a7dab611e31023ae82628f5aad5792b7c5d51#subdirectory=plugins/flytekit-deck-standard"

image_spec = ImageSpec(
    packages=[flytekit_version, deck_plugin_version],
    apt_packages=["git"],
    registry="jasonlai1218",
    env={"FLYTE_SDK_LOGGING_LEVEL": "20"},
)


@task(container_image=image_spec, enable_deck=True)
def python＿dependency_deck() -> None:
    ctx = flytekit.FlyteContextManager.current_context()
    ctx.add_deck(PythonDependencyDeck())


@workflow
def py_deps_deck_wf() -> None:
    python＿dependency_deck()


if __name__ == "__main__":
    py_deps_deck_wf()

2. write a launch plan

from dev_python_dependency_deck import py_deps_deck_wf
from flytekit import LaunchPlan, CronSchedule


py_deps_deck_lp = LaunchPlan.get_or_create(
    py_deps_deck_wf,
    name="py_deps_deck_lp",
    schedule=CronSchedule(schedule="06 00 * * *"),
)

3. execute cmd: FLYTE_SDK_LOGGING_LEVEL=20 pyflyte register --non-fast -p flytesnacks --domain development dev_register_lp.py --version t1

Screenshots

Are you sure this issue hasn't been raised already?

• Yes

Have you read the Code of Conduct?

• Yes

[dosubot]

Hey @jasonlai1218, great to see you diving deep into Flyte again! Hope all's well on your end. 😊

Based on the logs you've provided, it seems like two launch plans are being registered because the workflow itself is being registered as a launch plan, and then the explicit launch plan you've defined is also being registered.

This is evident from these two lines in your logs:

[✔] Registration dev_python_dependency_deck.py_deps_deck_wf type LAUNCH_PLAN successful with version t1
[✔] Registration py_deps_deck_lp type LAUNCH_PLAN successful with version t1

The first line is registering the workflow dev_python_dependency_deck.py_deps_deck_wf as a launch plan, and the second line is registering the explicit launch plan py_deps_deck_lp you've defined.

This behavior is expected in Flyte. When you register a workflow, Flyte automatically creates a default launch plan for it. This default launch plan has the same name as the workflow and allows you to execute the workflow without having to define a separate launch plan.

Then, when you define an explicit launch plan using LaunchPlan.get_or_create(), this is registered as a separate launch plan. This allows you to have multiple launch plans for the same workflow, each with different configurations (e.g., schedules, default inputs).

If you only want to register the explicit launch plan and not the default one, you can register the tasks and workflows separately using the pyflyte register command with the --tasks-only and --workflows-only options, and then register the launch plans. Here's how you can do it:

1. Register tasks:

FLYTE_SDK_LOGGING_LEVEL=20 pyflyte register --non-fast -p flytesnacks --domain development dev_register_lp.py --version t1 --tasks-only

2. Register workflows:

FLYTE_SDK_LOGGING_LEVEL=20 pyflyte register --non-fast -p flytesnacks --domain development dev_register_lp.py --version t1 --workflows-only

3. Register launch plans:

FLYTE_SDK_LOGGING_LEVEL=20 pyflyte register --non-fast -p flytesnacks --domain development dev_register_lp.py --version t1 --launchplans-only

This way, only the explicit launch plan you've defined will be registered, and not the default one.

I hope this helps! Let me know if you have any other questions.

Sources

• flyteadmin/tests/launch_plan_test.go
• flyteadmin/pkg/manager/impl/util/single_task_execution.go
• flyteidl/gen/pb_python/flyteidl/service/flyteadmin/flyteadmin/api/admin_service_api.py

About Dosu

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.

[wild-endeavor]

The dosubot is almost correct

The first line is registering the workflow dev_python_dependency_deck.py_deps_deck_wf as a launch plan, and the second line is registering the explicit launch plan py_deps_deck_lp you've defined.

There is a default launch plan that is always registered for every workflow, which is why there are two.