Skip to content
This repository has been archived by the owner on Oct 9, 2023. It is now read-only.

Switch to official dask operator #366

Merged

Conversation

bstadlbauer
Copy link
Member

TL;DR

This replaces the go dask client from github.com/bstadlbauer/dask-k8s-operator-go-client with the official one from github.com/dask/dask-kubernetes/v2023.

The operator has been moved over from my repo to the official one in dask/dask-kubernetes#731, so the code is essentially the same, but I'm going ahead and will archive my version.

Type

  • Bug Fix
  • Feature
  • Plugin

Signed-off-by: Bernhard Stadlbauer <11799671+bstadlbauer@users.noreply.github.com>
Signed-off-by: Bernhard Stadlbauer <11799671+bstadlbauer@users.noreply.github.com>
@@ -53,7 +52,7 @@ func getDefaults(ctx context.Context, taskCtx pluginsCore.TaskExecutionContext,
return nil, errors.Errorf(errors.BadTaskSpecification, "task is missing a default image")
}

defaultEnvVars := []v1.EnvVar{}
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@hamersaw I've snuck in a few mini improvements that GoLand said I could do

@codecov
Copy link

codecov bot commented Jun 27, 2023

Codecov Report

Merging #366 (8cc6727) into master (dfdf6f9) will increase coverage by 0.01%.
The diff coverage is 66.66%.

❗ Current head 8cc6727 differs from pull request most recent head efcfcca. Consider uploading reports for the commit efcfcca to get more accurate results

@@            Coverage Diff             @@
##           master     #366      +/-   ##
==========================================
+ Coverage   64.05%   64.07%   +0.01%     
==========================================
  Files         152      152              
  Lines       10392    10392              
==========================================
+ Hits         6657     6659       +2     
+ Misses       3124     3122       -2     
  Partials      611      611              
Impacted Files Coverage Δ
go/tasks/plugins/webapi/bigquery/plugin.go 67.44% <50.00%> (ø)
go/tasks/plugins/k8s/dask/dask.go 85.95% <100.00%> (ø)

... and 1 file with indirect coverage changes

github.com/coocood/freecache v1.1.1
github.com/dask/dask-kubernetes/v2023 v2023.0.0-20230626103304-abd02cd17b26
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is due to dask-kubernetes not prefixing their tags with a v (i.e. 2023.6.0 vs. v2023.6.0)

@hamersaw hamersaw merged commit 2c393c0 into flyteorg:master Jun 27, 2023
6 of 7 checks passed
eapolinario pushed a commit that referenced this pull request Sep 6, 2023
* Switch to using official `dask` plugin

Signed-off-by: Bernhard Stadlbauer <11799671+bstadlbauer@users.noreply.github.com>

* Fix small code nits

Signed-off-by: Bernhard Stadlbauer <11799671+bstadlbauer@users.noreply.github.com>

---------

Signed-off-by: Bernhard Stadlbauer <11799671+bstadlbauer@users.noreply.github.com>
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
2 participants