Skip to content
This repository has been archived by the owner on Oct 9, 2023. It is now read-only.

Carry over hash value for all literal types in remote caching #378

Merged
merged 1 commit into from
Jul 18, 2023
Merged

Carry over hash value for all literal types in remote caching #378

merged 1 commit into from
Jul 18, 2023

Conversation

nicholasjng
Copy link
Contributor

TL;DR

Previously, the hash on an input literal would be ignored if the literal was of either the Collection or Map type. This is fixed here by moving the hash-checking branch up in front of the special type logic, so that an empty literal with hash is constructed in either case if a hash is found.

Adds a regression test to verify the hashes are carried over into the representer literals for both map and collection types.

Type

  • Bug Fix
  • Feature
  • Plugin

Are all requirements met?

  • Code completed
  • Smoke tested
  • Unit tests added
  • Code documentation added
  • Any pending items have an associated Issue

Complete description

See the discussion in flyteorg/flytekit#1720 for the problem statement. This PR is a direct follow-up.

The remote cache fix was missing from that PR, and that is exactly what is added here. The code here is the same fix approach, just in Go this time.

Tracking Issue

N/A

@welcome
Copy link

welcome bot commented Jul 15, 2023

Thank you for opening this pull request! 🙌

These tips will help get your PR across the finish line:

  • Most of the repos have a PR template; if not, fill it out to the best of your knowledge.
  • Sign off your commits (Reference: DCO Guide).

Copy link
Contributor

@eapolinario eapolinario left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking good, just one small comment.


// Ensure that hash values on maps and collections are carried over
// into representing literals.
func TestHashCarryOnContainers(t *testing.T) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you move these tests to the table-tests in TestHashLiteralMap_LiteralsWithHashSet instead?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in f4d830e.

Previously, the hash set on an input literal would be ignored if
the literal was of either the `Collection` or `Map` type. This is
fixed here by moving the hash-checking branch up in front of the
special type logic, so that an empty literal with hash is constructed
in either case if a hash is found.

Adds a regression test to verify the hashes are carried over into the
representer literals for both map and collection types.

Signed-off-by: Nicholas Junge <nicholas.junge@web.de>
@codecov
Copy link

codecov bot commented Jul 18, 2023

Codecov Report

Merging #378 (cb2101d) into master (7ddd993) will increase coverage by 1.41%.
The diff coverage is 100.00%.

❗ Current head cb2101d differs from pull request most recent head f4d830e. Consider uploading reports for the commit f4d830e to get more accurate results

@@            Coverage Diff             @@
##           master     #378      +/-   ##
==========================================
+ Coverage   62.89%   64.30%   +1.41%     
==========================================
  Files         154      154              
  Lines       13029    10563    -2466     
==========================================
- Hits         8195     6793    -1402     
+ Misses       4221     3157    -1064     
  Partials      613      613              
Flag Coverage Δ
unittests ?

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
...s/pluginmachinery/tasklog/templatescheme_enumer.go 0.00% <ø> (ø)
go/tasks/pluginmachinery/catalog/hashing.go 94.73% <100.00%> (+0.39%) ⬆️
go/tasks/plugins/webapi/bigquery/config_flags.go 45.71% <100.00%> (+7.47%) ⬆️

... and 132 files with indirect coverage changes

@eapolinario eapolinario merged commit 13ef7d6 into flyteorg:master Jul 18, 2023
7 checks passed
eapolinario pushed a commit that referenced this pull request Sep 6, 2023
Previously, the hash set on an input literal would be ignored if
the literal was of either the `Collection` or `Map` type. This is
fixed here by moving the hash-checking branch up in front of the
special type logic, so that an empty literal with hash is constructed
in either case if a hash is found.

Adds a regression test to verify the hashes are carried over into the
representer literals for both map and collection types.

Signed-off-by: Nicholas Junge <nicholas.junge@web.de>
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
2 participants