[Monorepo] Bring flytectl #5301

eapolinario · 2024-04-30T20:59:26Z

Tracking issue

NA

Why are the changes needed?

Having flytectl in the monorepo helps in the development and rollout of changes that require cross-component changes (including flyteidl).

What changes were proposed in this pull request?

Similar to #4017, this PR only brings in flytectl at flyteorg/flytectl@67cae23. Subsequent PRs will hook up the CI checks and necessary changes to release flytectl on Flyte releases.

How was this patch tested?

Setup process

Screenshots

Check all the applicable boxes

I updated the documentation accordingly.
All new and existing tests passed.
All commits are signed-off.

Related PRs

Docs link

Signed-off-by: Flyte-Bot <admin@flyte.org> Co-authored-by: evalsocket <evalsocket@users.noreply.github.com>

* fix sandbox start if sandbox exist Signed-off-by: Yuvraj <code@evalsocket.dev>

* Added node execution data to show inputs and outputs Signed-off-by: Prafulla Mahindrakar <prafulla.mahindrakar@gmail.com> * Added closure class Signed-off-by: Prafulla Mahindrakar <prafulla.mahindrakar@gmail.com> * Removed the old wrapper classes Signed-off-by: Prafulla Mahindrakar <prafulla.mahindrakar@gmail.com> * Uncommented the task exec closure Signed-off-by: Prafulla Mahindrakar <prafulla.mahindrakar@gmail.com> * Fixed unit tests for the closure classes Signed-off-by: Prafulla Mahindrakar <prafulla.mahindrakar@gmail.com> * go mod tidy Signed-off-by: Prafulla Mahindrakar <prafulla.mahindrakar@gmail.com> * Adding .gitattribute file to contain rule for .rst files Signed-off-by: Prafulla Mahindrakar <prafulla.mahindrakar@gmail.com> * Adding gen path to .gitattr Signed-off-by: Prafulla Mahindrakar <prafulla.mahindrakar@gmail.com> * Added comments to new structs Signed-off-by: Prafulla Mahindrakar <prafulla.mahindrakar@gmail.com>

Signed-off-by: Sean Lin <sean@union.ai>

* fix gcp parameters... s3 -> google Signed-off-by: Bruce Arctor <5032356+brucearctor@users.noreply.github.com> * change s3 to google for gcs docs Signed-off-by: Bruce Arctor <5032356+brucearctor@users.noreply.github.com>

Signed-off-by: Flyte-Bot <admin@flyte.org> Co-authored-by: kumare3 <kumare3@users.noreply.github.com>

* Added setup flytedir before config init Signed-off-by: Yuvraj <code@evalsocket.dev>

Signed-off-by: Yuvraj <code@evalsocket.dev>

* code-of-conduct Signed-off-by: Samhita Alla <aallasamhita@gmail.com> * boilerplate Signed-off-by: Samhita Alla <aallasamhita@gmail.com>

Signed-off-by: cosmicBboy <niels.bantilan@gmail.com>

…un individually (#155) Signed-off-by: Sean Lin <sean@union.ai>

Signed-off-by: Sean Lin <sean@union.ai>

* Added k8s check in sandbox * Added upgrade command Signed-off-by: Yuvraj <code@evalsocket.dev>

Signed-off-by: Flyte-Bot <admin@flyte.org> Co-authored-by: pmahindrakar-oss <pmahindrakar-oss@users.noreply.github.com>

* Added version compare check and bug fix in sandbox start Signed-off-by: Yuvraj <code@evalsocket.dev>

Signed-off-by: Arnaud Alies <arnaudalies.py@gmail.com>

Signed-off-by: Flyte-Bot <admin@flyte.org> Co-authored-by: evalsocket <evalsocket@users.noreply.github.com>

Signed-off-by: Yuvraj <code@evalsocket.dev>

Signed-off-by: Ketan Umare <ketan.umare@gmail.com>

Update current video to a shorter version specifically edited to be embedded in the docs.

…#168) `flytectl get task/workflow/launchplan -o table` panics if resource has no expected output. We encountered `FormatVariableDescriptions at pkg/printer/printer.go:173` to panic when using the get commands on a resource without expected output. This happened with both explicit table output as well as default. Replicated on lastest master, workflow demo-1.no-output has no expected output: ``` $ go run ./main.go get launchplans -p demo-1 -d production --latest demo-1.no-output panic: assignment to entry in nil map goroutine 1 [running]: github.com/flyteorg/flytectl/pkg/printer.FormatVariableDescriptions(0x0) /workspace/flyteorg/flytectl/pkg/printer/printer.go:202 +0x307 github.com/flyteorg/flytectl/cmd/get.LaunchplanToTableProtoMessages({0xc000a30938, 0x1, 0x7ffeefbff977}) /workspace/flyteorg/flytectl/cmd/get/launch_plan.go:136 +0xee github.com/flyteorg/flytectl/cmd/get.getLaunchPlanFunc({0x2dfbf98, 0xc0000580b8}, {0xc0002fb8c0, 0x1, 0x11519b4}, {{0x2e55850, 0xc000a30120}, {0x2e472c0, 0xc0009a3110}, {0x2ddff60, ...}, ...}) /workspace/flyteorg/flytectl/cmd/get/launch_plan.go:158 +0x307 github.com/flyteorg/flytectl/cmd/core.generateCommandFunc.func1(0xc000481680, {0xc0002fb8c0, 0x1, 0x6}) /workspace/flyteorg/flytectl/cmd/core/cmd.go:69 +0x47d github.com/spf13/cobra.(*Command).execute(0xc000481680, {0xc0002fb860, 0x6, 0x6}) /workspace/go/pkg/mod/github.com/spf13/cobra@v1.1.3/command.go:852 +0x60e github.com/spf13/cobra.(*Command).ExecuteC(0xc000480000) /workspace/go/pkg/mod/github.com/spf13/cobra@v1.1.3/command.go:960 +0x3ad github.com/spf13/cobra.(*Command).Execute(...) /workspace/go/pkg/mod/github.com/spf13/cobra@v1.1.3/command.go:897 github.com/flyteorg/flytectl/cmd.ExecuteCmd() /workspace/flyteorg/flytectl/cmd/root.go:135 +0x1e main.main() /workspace/flyteorg/flytectl/main.go:12 +0x1d exit status 2 ``` Simple fix: nil checks exists for container but not on the map itself prior to calling, and nil input causes the panic. Added nil check on map parameter prior to calling in all places found. Signed-off-by: Viktor Gerdin <viktorg@spotify.com>

Signed-off-by: Yuvraj <code@evalsocket.dev>

* Added launchplan update command and moved namedentity Signed-off-by: Prafulla Mahindrakar <prafulla.mahindrakar@gmail.com>

Signed-off-by: Katrina Rogan <katroganGH@gmail.com>

* Rename --archive to --deactivate in update launchplan Signed-off-by: asoundarya96 <sumisoundarya96@gmail.com> * Rename --archive to --deactivate in update launchplan Signed-off-by: asoundarya96 <sumisoundarya96@gmail.com> * Keep --archive as deprecated flag Signed-off-by: asoundarya96 <sumisoundarya96@gmail.com> * make generate Signed-off-by: Eduardo Apolinario <eapolinario@users.noreply.github.com> --------- Signed-off-by: asoundarya96 <sumisoundarya96@gmail.com> Signed-off-by: Eduardo Apolinario <eapolinario@users.noreply.github.com> Co-authored-by: Eduardo Apolinario <eapolinario@users.noreply.github.com>

Signed-off-by: Niels Bantilan <niels.bantilan@gmail.com>

Signed-off-by: Kevin Su <pingsutw@gmail.com>

* project-archive-with-yaml Signed-off-by: Peeter Piegaze <1153481+ppiegaze@users.noreply.github.com> * fix syntax Signed-off-by: Peeter Piegaze <1153481+ppiegaze@users.noreply.github.com> * fix list Signed-off-by: Peeter Piegaze <1153481+ppiegaze@users.noreply.github.com> * update generated Signed-off-by: Peeter Piegaze <1153481+ppiegaze@users.noreply.github.com> --------- Signed-off-by: Peeter Piegaze <1153481+ppiegaze@users.noreply.github.com>

Signed-off-by: Niels Bantilan <niels.bantilan@gmail.com>

…st (#450) Signed-off-by: Guy Rapaport <guy4261@gmail.com>

Signed-off-by: Jeongwon Song <jeongwon412@gmail.com>

Signed-off-by: nikki everett <nikki@union.ai>

Signed-off-by: troychiu <y.troychiu@gmail.com>

Signed-off-by: Yee Hing Tong <wild-endeavor@users.noreply.github.com>

Signed-off-by: cosmicBboy <niels.bantilan@gmail.com>

* feat: add pagination for get execution (draft) Signed-off-by: zychen5186 <brianchen5197@gmail.com> fix: catches existing commands in os.Args Signed-off-by: zychen5186 <brianchen5197@gmail.com> fix: restore neccessary codes Signed-off-by: zychen5186 <brianchen5197@gmail.com> * feat: use -i to trigger bubbletea, add pagination for get execution Signed-off-by: zychen5186 <brianchen5197@gmail.com> * change dot to arabic paging format Signed-off-by: zychen5186 <brianchen5197@gmail.com> change dot to arabic paging format Signed-off-by: zychen5186 <brianchen5197@gmail.com> change var names Signed-off-by: zychen5186 <brianchen5197@gmail.com> fix: lint Signed-off-by: zychen5186 <brianchen5197@gmail.com> * reuse JSONToTable Signed-off-by: zychen5186 <brianchen5197@gmail.com> * reuse JSONToTable Signed-off-by: zychen5186 <brianchen5197@gmail.com> change := to var Signed-off-by: zychen5186 <brianchen5197@gmail.com> * keep original format when not using bubbletea Signed-off-by: zychen5186 <brianchen5197@gmail.com> * improve readability and no functionality is changed Signed-off-by: zychen5186 <brianchen5197@gmail.com> --------- Signed-off-by: zychen5186 <brianchen5197@gmail.com>

Signed-off-by: Eduardo Apolinario <eapolinario@users.noreply.github.com> Co-authored-by: Eduardo Apolinario <eapolinario@users.noreply.github.com>

…pare-monorepo--flytectl Signed-off-by: Eduardo Apolinario <eapolinario@users.noreply.github.com>

codecov · 2024-04-30T21:05:19Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 60.19%. Comparing base (40dccab) to head (8cad4ed).

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #5301      +/-   ##
==========================================
+ Coverage   59.68%   60.19%   +0.50%     
==========================================
  Files         568      646      +78     
  Lines       41729    45654    +3925     
==========================================
+ Hits        24906    27480    +2574     
- Misses      14410    15582    +1172     
- Partials     2413     2592     +179

Flag	Coverage Δ
unittests-datacatalog	`69.31% <ø> (ø)`
unittests-flyteadmin	`58.78% <ø> (-0.05%)`	⬇️
unittests-flytecopilot	`17.79% <ø> (ø)`
unittests-flyteidl	`79.30% <ø> (ø)`
unittests-flyteplugins	`61.94% <ø> (ø)`
unittests-flytepropeller	`57.32% <ø> (ø)`
unittests-flytestdlib	`65.73% <ø> (?)`

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Signed-off-by: Eduardo Apolinario <eapolinario@users.noreply.github.com>

katrogan

amazing, thank you!

katrogan · 2024-04-30T22:33:41Z

flytectl/go.mod

+)
+
+replace (
+	github.com/flyteorg/flyte/flyteidl => github.com/flyteorg/flyte/flyteidl v1.9.12


should this be relative now?

yeah, module renaming is next.

katrogan · 2024-04-30T22:34:41Z

flytectl/pull_request_template.md

still necessary?

nope, will drop in the next PR.

katrogan · 2024-04-30T22:35:20Z

flytectl/CODE_OF_CONDUCT.md

I guess this can be handled in a follow-up, but do we still need all the package-level boilerplate with this move?

yeah, these all will be removed in separate PRs.

ddl-ebrown · 2024-04-30T22:47:09Z

Oh cool... will hold off on any work for flyteorg/flytectl#472 and reevaluate for vulns after the move is complete

eapolinario · 2024-04-30T22:48:30Z

thank you, @ddl-ebrown!

troychiu · 2024-04-30T23:08:07Z

cc @zychen5186

flyte-bot and others added 30 commits July 13, 2021 00:24

Update documentation (#142)

5695606

Signed-off-by: Flyte-Bot <admin@flyte.org> Co-authored-by: evalsocket <evalsocket@users.noreply.github.com>

Bug fix in sandbox start when sandbox exist (#145)

50ee307

* fix sandbox start if sandbox exist Signed-off-by: Yuvraj <code@evalsocket.dev>

Adding support for workflow execution config (#144)

9b05df0

Signed-off-by: Sean Lin <sean@union.ai>

fix gcp parameters... s3 -> google (#147)

5392ada

* fix gcp parameters... s3 -> google Signed-off-by: Bruce Arctor <5032356+brucearctor@users.noreply.github.com> * change s3 to google for gcs docs Signed-off-by: Bruce Arctor <5032356+brucearctor@users.noreply.github.com>

Update documentation (#146)

0510aaa

Signed-off-by: Flyte-Bot <admin@flyte.org> Co-authored-by: kumare3 <kumare3@users.noreply.github.com>

Add ability to recover executions (#149)

2d65956

Fix config init command fails if no flyte dir doesn't exist (#150)

a66e669

* Added setup flytedir before config init Signed-off-by: Yuvraj <code@evalsocket.dev>

#none add go mod tidy in flyteidl release workflow (#151)

5da69f5

Signed-off-by: Yuvraj <code@evalsocket.dev>

Update code of conduct (#148)

3d86194

* code-of-conduct Signed-off-by: Samhita Alla <aallasamhita@gmail.com> * boilerplate Signed-off-by: Samhita Alla <aallasamhita@gmail.com>

update doc requirements with sphinx v4 (#153)

2a9a827

Signed-off-by: cosmicBboy <niels.bantilan@gmail.com>

Update Flyteidl version (#134)

2fd5e56

Render descriptions in flytectl (#154)

eaeb998

Reset default filter in the task_test so that the unit tests can be r…

f2d8560

…un individually (#155) Signed-off-by: Sean Lin <sean@union.ai>

Add additional nil check (#156)

3e4323a

Signed-off-by: Sean Lin <sean@union.ai>

Remove input output columns for list workflows (#157)

2fae848

Signed-off-by: Sean Lin <sean@union.ai>

Add upgrade command with some refactor (#152)

89cb374

* Added k8s check in sandbox * Added upgrade command Signed-off-by: Yuvraj <code@evalsocket.dev>

Update Flyteidl version (#160)

c98c23c

Update documentation (#159)

5e45344

Signed-off-by: Flyte-Bot <admin@flyte.org> Co-authored-by: pmahindrakar-oss <pmahindrakar-oss@users.noreply.github.com>

Bug fix in sandbix start (#161)

3a48b4a

* Added version compare check and bug fix in sandbox start Signed-off-by: Yuvraj <code@evalsocket.dev>

add --dryRun flag to flytectl subcommands (#163)

ce15cf0

Update upgrade.go (#164)

c664440

Signed-off-by: Arnaud Alies <arnaudalies.py@gmail.com>

Update documentation (#162)

e3c8caf

Signed-off-by: Flyte-Bot <admin@flyte.org> Co-authored-by: evalsocket <evalsocket@users.noreply.github.com>

bind sandbox port to host system (#165)

6618d9e

Signed-off-by: Yuvraj <code@evalsocket.dev>

youtube video embedding for flytectl with walkthrough (#166)

3b266d7

Signed-off-by: Ketan Umare <ketan.umare@gmail.com>

Point Tutorial Youtube video to the correct one (#167)

1cf79c1

Update current video to a shorter version specifically edited to be embedded in the docs.

hotfix fast register (#171)

ac2f7c8

Signed-off-by: Yuvraj <code@evalsocket.dev>

Added launchplan update command and moved namedentity (#170)

0a23db3

* Added launchplan update command and moved namedentity Signed-off-by: Prafulla Mahindrakar <prafulla.mahindrakar@gmail.com>

Fix panic in rendering LP input descriptions (#174)

dcc2c09

Signed-off-by: Katrina Rogan <katroganGH@gmail.com>

asoundarya96 and others added 19 commits December 26, 2023 11:49

add monodocs redirect banner (#452)

9322c06

Signed-off-by: Niels Bantilan <niels.bantilan@gmail.com>

add flytectl docs redirects to monodocs (#454)

06765aa

Signed-off-by: Niels Bantilan <niels.bantilan@gmail.com>

add monodocs index page in flytectl (#453)

d30de34

Signed-off-by: Niels Bantilan <niels.bantilan@gmail.com>

fix dependencies (#455)

f5473de

Signed-off-by: Niels Bantilan <niels.bantilan@gmail.com>

Hydrate failure node (#456)

dda6460

Signed-off-by: Kevin Su <pingsutw@gmail.com>

Remove dependency on unsupported k8s.io/kubernetes library (#457)

71e61f1

install latest flyteidl with monodocs build (#459)

b725a65

Signed-off-by: Niels Bantilan <niels.bantilan@gmail.com>

Update typo nodID -> nodeID in execution.go, flytectl_get_execution.r…

cff5a3d

…st (#450) Signed-off-by: Guy Rapaport <guy4261@gmail.com>

Fix: Check for git authentication when GITHUB_TOKEN exists (#463)

8362431

Signed-off-by: Jeongwon Song <jeongwon412@gmail.com>

update links for monodocs (#465)

007a4b1

Signed-off-by: nikki everett <nikki@union.ai>

Update admin endpoint to use port 30080 in config.yaml (#467)

44c55f6

Signed-off-by: troychiu <y.troychiu@gmail.com>

Fix get task resource attribute comment (#469)

b03e86d

Signed-off-by: Yee Hing Tong <wild-endeavor@users.noreply.github.com>

use sphinx-design directives instead of sphinx-panels (#471)

de61bd0

Signed-off-by: cosmicBboy <niels.bantilan@gmail.com>

Do not upload codecoverage data from boilerplate (#478)

ce0fd45

Signed-off-by: Eduardo Apolinario <eapolinario@users.noreply.github.com> Co-authored-by: Eduardo Apolinario <eapolinario@users.noreply.github.com>

Merge remote-tracking branch 'flytectl/prepare-for-monorepo' into pre…

defacc1

…pare-monorepo--flytectl Signed-off-by: Eduardo Apolinario <eapolinario@users.noreply.github.com>

eapolinario changed the title ~~Prepare monorepo flytectl~~ [Monorepo] Bring flytectl Apr 30, 2024

Fix typos

8cad4ed

Signed-off-by: Eduardo Apolinario <eapolinario@users.noreply.github.com>

katrogan approved these changes Apr 30, 2024

View reviewed changes

eapolinario merged commit 79a1992 into master Apr 30, 2024
48 checks passed

eapolinario deleted the prepare-monorepo--flytectl branch April 30, 2024 22:37

ddl-ebrown mentioned this pull request May 14, 2024

Upgrade go 1.19 -> 1.21 / resolve vulns flyteorg/flytectl#472

Closed

8 tasks

eapolinario mentioned this pull request Jul 3, 2024

Document how to release flytectl #5537

Merged

3 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Monorepo] Bring flytectl #5301

[Monorepo] Bring flytectl #5301

eapolinario commented Apr 30, 2024

codecov bot commented Apr 30, 2024 •

edited

Loading

katrogan left a comment

katrogan Apr 30, 2024

eapolinario Apr 30, 2024

katrogan Apr 30, 2024

eapolinario Apr 30, 2024

katrogan Apr 30, 2024

eapolinario Apr 30, 2024

ddl-ebrown commented Apr 30, 2024

eapolinario commented Apr 30, 2024

troychiu commented Apr 30, 2024

[Monorepo] Bring flytectl #5301

[Monorepo] Bring flytectl #5301

Conversation

eapolinario commented Apr 30, 2024

Tracking issue

Why are the changes needed?

What changes were proposed in this pull request?

How was this patch tested?

Setup process

Screenshots

Check all the applicable boxes

Related PRs

Docs link

codecov bot commented Apr 30, 2024 • edited Loading

Codecov Report

katrogan left a comment

Choose a reason for hiding this comment

katrogan Apr 30, 2024

Choose a reason for hiding this comment

eapolinario Apr 30, 2024

Choose a reason for hiding this comment

katrogan Apr 30, 2024

Choose a reason for hiding this comment

eapolinario Apr 30, 2024

Choose a reason for hiding this comment

katrogan Apr 30, 2024

Choose a reason for hiding this comment

eapolinario Apr 30, 2024

Choose a reason for hiding this comment

ddl-ebrown commented Apr 30, 2024

eapolinario commented Apr 30, 2024

troychiu commented Apr 30, 2024

codecov bot commented Apr 30, 2024 •

edited

Loading