feat: wip max in-flight pulls #81

vadasambar · 2023-12-11T05:31:33Z

Fixes #77

vadasambar · 2023-12-14T05:30:56Z

For now, pulling changes from #83 for testing this PR.

vadasambar · 2023-12-19T06:32:46Z

Ran tests:

$ cd cmd/plugin
$ go test -run TestNodePublishVolumeAsyncInFlightPulls

Test result

I1219 12:01:28.785605   28062 server.go:108] Listening for connections on address: &net.UnixAddr{Name:"//tmp/csi.sock", Net:"unix"}
I1219 12:01:30.789678   28062 node_server.go:62] mount request: volume_id:"docker.io/library/redis:latest" target_path:"test-path1" volume_capability:<access_mode:<mode:SINGLE_NODE_READER_ONLY > > volume_context:<key:"pullAlways" value:"true" > 
I1219 12:01:30.795363   28062 pullexecutor.go:123] pull image "docker.io/library/redis:latest" 
I1219 12:01:30.887674   28062 node_server.go:62] mount request: volume_id:"docker.io/library/ubuntu:latest" target_path:"test-path2" volume_capability:<access_mode:<mode:SINGLE_NODE_READER_ONLY > > volume_context:<key:"pullAlways" value:"true" > 
E1219 12:01:35.788257   28062 node_server_test.go:266] context deadline exceeded; retrying: rpc error: code = DeadlineExceeded desc = context deadline exceeded
I1219 12:01:35.795929   28062 pullexecutor.go:123] pull image "docker.io/library/ubuntu:latest" 
E1219 12:01:35.888196   28062 node_server_test.go:281] context deadline exceeded; retrying: rpc error: code = DeadlineExceeded desc = context deadline exceeded
I1219 12:01:35.888497   28062 node_server.go:62] mount request: volume_id:"docker.io/library/redis:latest" target_path:"test-path1" volume_capability:<access_mode:<mode:SINGLE_NODE_READER_ONLY > > volume_context:<key:"pullAlways" value:"true" > 
I1219 12:01:35.989549   28062 node_server.go:62] mount request: volume_id:"docker.io/library/ubuntu:latest" target_path:"test-path2" volume_capability:<access_mode:<mode:SINGLE_NODE_READER_ONLY > > volume_context:<key:"pullAlways" value:"true" > 
I1219 12:01:45.896044   28062 node_server_test.go:235] server was stopped
PASS
ok  	github.com/warm-metal/csi-driver-image/cmd/plugin	17.135s

- makes the tests easier to run - edit docs to run node server with containerd image refactor: remove commented out code chore: switch over from criapi `v1alpha2` -> `v1`

test: use mock puller and mounter in the node server tests - makes the tests easier to run - edit docs to run node server with containerd image feat: first implementation of the tokens logic test: add test for in-flight pulls refactor: use string instead of docker named ref - works for now but we might face problems in the future if the docker named ref starts using pointer fields chore: fix syntax error after rebase refactor: use named ref string instead of named ref struct directly - because named ref struct might not always satisfy equality (it does no because the struct is shallow) feat: expose `--max-in-flight-pulls` as a flag - so that users can set the value through flag or helm chart docs: add log message for when the image is downloaded - helps with testing max in-flight image pulls

- because of syntax errors

- makes it easier to test `--max-in-flight-pulls` - expose the flags via helm values - add integration test to test `--max-in-flight-pulls` flag - move metrics integration tests to generic integration tests

- use the correct namespace (`kube-system` instead of `default`)

mbtamuli · 2024-01-15T06:02:46Z

cmd/plugin/node_server.go

@@ -78,7 +78,7 @@ func (n NodeServer) NodePublishVolume(ctx context.Context, req *csi.NodePublishV
 		return
 	}

-	// to trigger CI
+	// to trigger CI again


@vadasambar You might consider using allow-empty. You'll not need to remove any unnecessary lines later.

git commit --allow-empty --message "🤖 Trigger CI run"

This is helpful. Thank you Mriyam.

- fixes issue where warm metal downloads images serially

github-actions · 2024-03-05T00:38:05Z

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed in 7 days if no further activity occurs.

github-actions · 2024-03-13T00:39:08Z

Closing this PR after a prolonged period of inactivity. Please create a new PR if the changes of the PR are still relevant.

vadasambar mentioned this pull request Dec 11, 2023

Support in-flight requests for warm metal driver #77

Closed

4 tasks

vadasambar changed the title ~~feat: max in-flight pulls~~ [DON'T REVIEW] feat: wip max in-flight pulls Dec 11, 2023

vadasambar mentioned this pull request Dec 14, 2023

test: use mock puller and mounter in the node server tests #83

Merged

vadasambar changed the title ~~[DON'T REVIEW] feat: wip max in-flight pulls~~ feat: wip max in-flight pulls Dec 19, 2023

vadasambar mentioned this pull request Jan 11, 2024

feat: make hostNetwork configurable #107

Merged

vadasambar added 13 commits January 15, 2024 10:42

test: use mock puller and mounter in the node server tests

0815c27

- makes the tests easier to run - edit docs to run node server with containerd image refactor: remove commented out code chore: switch over from criapi `v1alpha2` -> `v1`

chore: fix missed merge conflict

b909d2c

test: fix node server tests failing

1c7bb79

- because of syntax errors

feat: add flag for async pull timeout

0fcdc0b

- makes it easier to test `--max-in-flight-pulls` - expose the flags via helm values - add integration test to test `--max-in-flight-pulls` flag - move metrics integration tests to generic integration tests

fix: ./test-metrics.sh not found in CI

41bf90d

test: fix CI tests failing for max-in-flight pulls

2a93b0d

- use the correct namespace (`kube-system` instead of `default`)

test: add debug ssh step to debug failing CI

eaa4e3a

refactor: remove redundant code from test script

c04b199

test: trigger CI with debug ssh action

1bb30d0

test: uncomment integration tests

d402232

test: trigger CI

0e85cb6

test: trigger CI again

60dc8fb

mbtamuli reviewed Jan 15, 2024

View reviewed changes

vadasambar added 5 commits January 15, 2024 22:57

🤖 Trigger CI run

886ef30

test: add debug ssh step

b191740

🤖 Trigger CI run 2

372f29d

test: remove ssh step

f8f112e

refactor: use mutex per key instead of per goroutine

af17fe5

- fixes issue where warm metal downloads images serially

mugdha-adhav assigned vadasambar Feb 3, 2024

github-actions bot added the Stale label Mar 5, 2024

github-actions bot closed this Mar 13, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: wip max in-flight pulls #81

feat: wip max in-flight pulls #81

vadasambar commented Dec 11, 2023

vadasambar commented Dec 14, 2023

vadasambar commented Dec 19, 2023

mbtamuli Jan 15, 2024

vadasambar Jan 15, 2024

github-actions bot commented Mar 5, 2024

github-actions bot commented Mar 13, 2024

feat: wip max in-flight pulls #81

feat: wip max in-flight pulls #81

Conversation

vadasambar commented Dec 11, 2023

vadasambar commented Dec 14, 2023

vadasambar commented Dec 19, 2023

mbtamuli Jan 15, 2024

Choose a reason for hiding this comment

vadasambar Jan 15, 2024

Choose a reason for hiding this comment

github-actions bot commented Mar 5, 2024

github-actions bot commented Mar 13, 2024