Add Flyte 1 → 2 migration skills + eval coverage#22
Open
cosmicBboy wants to merge 1 commit into
Open
Conversation
cosmicBboy
force-pushed
the
evals-harness
branch
from
July 23, 2026 21:44
fca6917 to
bcb5ae5
Compare
Convert the official Flyte 1->2 migration guide
(union.ai/docs/v2/flyte/user-guide/migration/flyte-2) into 6 agent skills:
- flyte-migrate start-here orchestrator (concept mapping, the two
mechanical changes, strategy, hybrid v1/v2, gotchas)
- flyte-migrate-tasks-workflows @task/@workflow/@dynamic -> @env.task
- flyte-migrate-config images/resources/cache/secrets/scheduling + CLI
- flyte-migrate-control-flow conditional/@dynamic/on_failure/map_task
- flyte-migrate-data-io FlyteFile/StructuredDataset -> flyte.io.*
- flyte-migrate-ml ML workloads + new-in-v2 patterns
Each skill carries verbatim before/after code, concept/parameter mapping
tables, and anti-patterns.
Eval harness: add static + trajectory scenarios for all 6 skills (12 total).
Trajectory scenarios feed the agent real v1 flytekit code and assert the
migrated output uses v2 APIs with no leftover v1 constructs. Update both
READMEs with a Migration section.
Verified: static lint 21/21 skills passing, 29 unit tests passing,
select.py correctly scopes migration-skill changes.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
cosmicBboy
force-pushed
the
migration-skills
branch
from
July 23, 2026 21:48
ebf5951 to
1060086
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Converts the official Flyte 1 → 2 migration guide (11 pages) into 6 agent skills and wires them into the eval harness.
New skills
flyte-migrateflytekit→flyteshift, terminology/concept mapping, the two mechanical changes, incremental strategy, hybrid v1/v2 pipelines, gotchas — routes to the restflyte-migrate-tasks-workflows@task/@workflow/@dynamic→ a single@env.taskon aTaskEnvironment; ordering without>>; subworkflows as tasksflyte-migrate-configImageSpec→flyte.Image), resources/GPUs, caching, secrets, scheduling (LaunchPlan/CronSchedule→Trigger/Cron), andpyflyte→flyteCLI/configflyte-migrate-control-flowconditional()→if/else,@dynamic→plain loops,on_failure→try/except,map_task→flyte.map/asyncio.gatherflyte-migrate-data-ioFlyteFile/FlyteDirectory→flyte.io.File/Dir,StructuredDataset→flyte.io.DataFrame, dataclasses/Pydantic I/Oflyte-migrate-mlEach skill carries verbatim before/after
Flyte 1/Flyte 2code, concept/parameter mapping tables, a Grounding References table, and Anti-Patterns.Eval coverage
staticlint scenario per skill + 6trajectoryscenarios that feed the agent real v1flytekitcode and assert the migrated output importsflyte, usesTaskEnvironment/@env.task, and has no leftover v1 constructs (import flytekit,@workflow,map_task,conditional(,FlyteFile,ImageSpec, …), plus an LLM-judge rubric.Verification
select.pycorrectly scopes migration-skill changesNot yet run: the trajectory/judge scenarios were verified structurally (load + select + lint) but not executed against a live agent+GLM run — that needs the auth-gated GLM key (same open spike as the base harness).
🤖 Generated with Claude Code