Skip to content
This repository has been archived by the owner on Oct 23, 2023. It is now read-only.

Add the scalar to the operand #427

Merged
merged 4 commits into from Aug 14, 2023
Merged

Add the scalar to the operand #427

merged 4 commits into from Aug 14, 2023

Conversation

pingsutw
Copy link
Member

TL;DR

Adding the scalar to the operand gives Propeller the ability to

  1. Check if the input value is None
  2. Compare two structured Dataset schemas
  3. Compare two file paths

Type

  • Bug Fix
  • Feature
  • Plugin

Are all requirements met?

  • Code completed
  • Smoke tested
  • Unit tests added
  • Code documentation added
  • Any pending items have an associated Issue

Complete description

^^^

Tracking Issue

flyteorg/flyte#3514

Follow-up issue

NA

Signed-off-by: Kevin Su <pingsutw@apache.org>
@pingsutw pingsutw requested a review from hamersaw July 21, 2023 17:12
Signed-off-by: Kevin Su <pingsutw@apache.org>
@codecov
Copy link

codecov bot commented Jul 21, 2023

Codecov Report

Merging #427 (70bdca5) into master (0dabef7) will increase coverage by 2.55%.
The diff coverage is n/a.

❗ Current head 70bdca5 differs from pull request most recent head 9550df1. Consider uploading reports for the commit 9550df1 to get more accurate results

@@            Coverage Diff             @@
##           master     #427      +/-   ##
==========================================
+ Coverage   75.92%   78.48%   +2.55%     
==========================================
  Files          18       18              
  Lines        1458     1250     -208     
==========================================
- Hits         1107      981     -126     
+ Misses        294      212      -82     
  Partials       57       57              
Flag Coverage Δ
unittests ?

Flags with carried forward coverage won't be shown. Click here to find out more.

see 18 files with indirect coverage changes

@hamersaw
Copy link
Contributor

@pingsutw the biggest concern here is opening up required support for union, map, collection, etc that aren't in primitive, but are in scalar. Is the plan for flytekit logic to have very specific errors saying "Unsupported type 'X' in branch conditional .." or something?

@pingsutw
Copy link
Member Author

Is the plan for flytekit logic to have very specific errors saying "Unsupported type 'X' in branch conditional .." or something?

Yes, in this PR, flytekit will return an error if people use collection or map. Now, we only support primitive and none.

Signed-off-by: Kevin Su <pingsutw@apache.org>
@pingsutw pingsutw merged commit 5198c9b into master Aug 14, 2023
17 checks passed
@pingsutw pingsutw deleted the operand-1 branch August 14, 2023 22:19
eapolinario pushed a commit that referenced this pull request Sep 8, 2023
Signed-off-by: Kevin Su <pingsutw@apache.org>
jeevb pushed a commit that referenced this pull request Sep 18, 2023
Signed-off-by: Kevin Su <pingsutw@apache.org>
Signed-off-by: Jeev B <jeevb@users.noreply.github.com>
jeevb added a commit that referenced this pull request Sep 20, 2023
* add field

Signed-off-by: Yee Hing Tong <wild-endeavor@users.noreply.github.com>
Signed-off-by: Jeev B <jeevb@users.noreply.github.com>

* Pass task execution metadata from agent (#422)

* Pass task execution metadata from agent

Signed-off-by: Hongxin Liang <honnix@users.noreply.github.com>

* Add doc

Signed-off-by: Hongxin Liang <honnix@users.noreply.github.com>

* Update protos/flyteidl/admin/agent.proto

Co-authored-by: Kevin Su <pingsutw@gmail.com>
Signed-off-by: Honnix <honnix@users.noreply.github.com>

* Regenerate

---------

Signed-off-by: Hongxin Liang <honnix@users.noreply.github.com>
Signed-off-by: Honnix <honnix@users.noreply.github.com>
Co-authored-by: Kevin Su <pingsutw@gmail.com>
Signed-off-by: Jeev B <jeevb@users.noreply.github.com>

* Add tags to execution spec (#414)

* add tags to execution spec

Signed-off-by: Kevin Su <pingsutw@apache.org>

* add tags to execution spec

Signed-off-by: Kevin Su <pingsutw@apache.org>

* add comment

Signed-off-by: Kevin Su <pingsutw@apache.org>

---------

Signed-off-by: Kevin Su <pingsutw@apache.org>
Signed-off-by: Jeev B <jeevb@users.noreply.github.com>

* Correct comment for array job max parallelism (#431)

Signed-off-by: Katrina Rogan <katroganGH@gmail.com>
Signed-off-by: Jeev B <jeevb@users.noreply.github.com>

* Add the scalar to the operand (#427)

Signed-off-by: Kevin Su <pingsutw@apache.org>
Signed-off-by: Jeev B <jeevb@users.noreply.github.com>

* add selector

Signed-off-by: Yee Hing Tong <wild-endeavor@users.noreply.github.com>
Signed-off-by: Jeev B <jeevb@users.noreply.github.com>

* move selectors from container to task metadata

Signed-off-by: Yee Hing Tong <wild-endeavor@users.noreply.github.com>
Signed-off-by: Jeev B <jeevb@users.noreply.github.com>

* drop only_preferred

Signed-off-by: Jeev B <jeevb@users.noreply.github.com>

* Updating boilerplate to lock golangci-lint version (#435)

Signed-off-by: Daniel Rammer <daniel@union.ai>
Signed-off-by: Jeev B <jeevb@users.noreply.github.com>

* add unpartitioned selector

Signed-off-by: Jeev B <jeevb@users.noreply.github.com>

* refactor

Signed-off-by: Jeev B <jeevb@users.noreply.github.com>

* refactor

Signed-off-by: Jeev B <jeevb@users.noreply.github.com>

* fix oneof names

Signed-off-by: Jeev B <jeevb@users.noreply.github.com>

* add build.os for read the docs

Signed-off-by: Jeev B <jeevb@users.noreply.github.com>

---------

Signed-off-by: Yee Hing Tong <wild-endeavor@users.noreply.github.com>
Signed-off-by: Jeev B <jeevb@users.noreply.github.com>
Signed-off-by: Hongxin Liang <honnix@users.noreply.github.com>
Signed-off-by: Honnix <honnix@users.noreply.github.com>
Signed-off-by: Kevin Su <pingsutw@apache.org>
Signed-off-by: Katrina Rogan <katroganGH@gmail.com>
Signed-off-by: Daniel Rammer <daniel@union.ai>
Co-authored-by: Honnix <honnix@users.noreply.github.com>
Co-authored-by: Kevin Su <pingsutw@gmail.com>
Co-authored-by: Kevin Su <pingsutw@apache.org>
Co-authored-by: Katrina Rogan <katroganGH@gmail.com>
Co-authored-by: Jeev B <jeevb@users.noreply.github.com>
Co-authored-by: Dan Rammer <daniel@union.ai>
eapolinario pushed a commit that referenced this pull request Sep 28, 2023
* add field

Signed-off-by: Yee Hing Tong <wild-endeavor@users.noreply.github.com>
Signed-off-by: Jeev B <jeevb@users.noreply.github.com>

* Pass task execution metadata from agent (#422)

* Pass task execution metadata from agent

Signed-off-by: Hongxin Liang <honnix@users.noreply.github.com>

* Add doc

Signed-off-by: Hongxin Liang <honnix@users.noreply.github.com>

* Update protos/flyteidl/admin/agent.proto

Co-authored-by: Kevin Su <pingsutw@gmail.com>
Signed-off-by: Honnix <honnix@users.noreply.github.com>

* Regenerate

---------

Signed-off-by: Hongxin Liang <honnix@users.noreply.github.com>
Signed-off-by: Honnix <honnix@users.noreply.github.com>
Co-authored-by: Kevin Su <pingsutw@gmail.com>
Signed-off-by: Jeev B <jeevb@users.noreply.github.com>

* Add tags to execution spec (#414)

* add tags to execution spec

Signed-off-by: Kevin Su <pingsutw@apache.org>

* add tags to execution spec

Signed-off-by: Kevin Su <pingsutw@apache.org>

* add comment

Signed-off-by: Kevin Su <pingsutw@apache.org>

---------

Signed-off-by: Kevin Su <pingsutw@apache.org>
Signed-off-by: Jeev B <jeevb@users.noreply.github.com>

* Correct comment for array job max parallelism (#431)

Signed-off-by: Katrina Rogan <katroganGH@gmail.com>
Signed-off-by: Jeev B <jeevb@users.noreply.github.com>

* Add the scalar to the operand (#427)

Signed-off-by: Kevin Su <pingsutw@apache.org>
Signed-off-by: Jeev B <jeevb@users.noreply.github.com>

* add selector

Signed-off-by: Yee Hing Tong <wild-endeavor@users.noreply.github.com>
Signed-off-by: Jeev B <jeevb@users.noreply.github.com>

* move selectors from container to task metadata

Signed-off-by: Yee Hing Tong <wild-endeavor@users.noreply.github.com>
Signed-off-by: Jeev B <jeevb@users.noreply.github.com>

* drop only_preferred

Signed-off-by: Jeev B <jeevb@users.noreply.github.com>

* Updating boilerplate to lock golangci-lint version (#435)

Signed-off-by: Daniel Rammer <daniel@union.ai>
Signed-off-by: Jeev B <jeevb@users.noreply.github.com>

* add unpartitioned selector

Signed-off-by: Jeev B <jeevb@users.noreply.github.com>

* refactor

Signed-off-by: Jeev B <jeevb@users.noreply.github.com>

* refactor

Signed-off-by: Jeev B <jeevb@users.noreply.github.com>

* fix oneof names

Signed-off-by: Jeev B <jeevb@users.noreply.github.com>

* add build.os for read the docs

Signed-off-by: Jeev B <jeevb@users.noreply.github.com>

---------

Signed-off-by: Yee Hing Tong <wild-endeavor@users.noreply.github.com>
Signed-off-by: Jeev B <jeevb@users.noreply.github.com>
Signed-off-by: Hongxin Liang <honnix@users.noreply.github.com>
Signed-off-by: Honnix <honnix@users.noreply.github.com>
Signed-off-by: Kevin Su <pingsutw@apache.org>
Signed-off-by: Katrina Rogan <katroganGH@gmail.com>
Signed-off-by: Daniel Rammer <daniel@union.ai>
Co-authored-by: Honnix <honnix@users.noreply.github.com>
Co-authored-by: Kevin Su <pingsutw@gmail.com>
Co-authored-by: Kevin Su <pingsutw@apache.org>
Co-authored-by: Katrina Rogan <katroganGH@gmail.com>
Co-authored-by: Jeev B <jeevb@users.noreply.github.com>
Co-authored-by: Dan Rammer <daniel@union.ai>
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
3 participants