Copy of 16M+ file to container causes invalid data frame error

[maorfr]

Hello,

We have been trying to implement a copy mechanism in to a container.
We are using this code:

stdin := bytes.NewReader(byteArray)
req := clientset.Core().RESTClient().Post().
	Resource("pods").
	Name("podName").
	Namespace("namespace").
	SubResource("exec")
scheme := runtime.NewScheme()
check(err)

parameterCodec := runtime.NewParameterCodec(scheme)
req.VersionedParams(&core_v1.PodExecOptions{
	Command:   strings.Fields("cp /dev/stdin /tmp/file"),
	Container: containerName,
	Stdin:     stdin != nil,
	Stdout:    true,
	Stderr:    true,
	TTY:       false,
}, parameterCodec)

exec, err := remotecommand.NewSPDYExecutor(config, "POST", req.URL())
check(err)

var stdout, stderr bytes.Buffer
err = exec.Stream(remotecommand.StreamOptions{
	Stdin:  stdin,
	Stdout: &stdout,
	Stderr: &stderr,
	Tty:    false,
})
check(err)

The complete code can be found here

We noticed that when trying to copy a file which is 16M and up, the action is not successful and we get an output of E0904 12:56:33.953212 15894 v2.go:105] invalid data frame.

Any file that is under 16M (16777215 and less) is being copied without any error.

Can anyone help on this?

Thanks in advance!

[deads2k]

@liggitt size seems suspicious compared to recent fixes.

[liggitt]

What version of client-go and apiserver is this seen with?

[maorfr]

@deads2k @liggitt thanks for the quick response!

Current versions (from glide.yaml):

- package: k8s.io/api
  version: kubernetes-1.11.2
  subpackages:
  - core/v1
- package: k8s.io/apimachinery
  version: kubernetes-1.11.2
  subpackages:
  - pkg/apis/meta/v1
- package: k8s.io/client-go
  version: kubernetes-1.11.2
  subpackages:
  - kubernetes
  - rest
  - tools/clientcmd

But we have also tried it with:

- package: k8s.io/api
  subpackages:
  - core/v1
- package: k8s.io/apimachinery
  subpackages:
  - pkg/apis/meta/v1
- package: k8s.io/client-go
  version: v8.0.0
  subpackages:
  - kubernetes
  - rest
  - tools/clientcmd

And got the same results.

API Server image is: k8s.gcr.io/kube-apiserver:v1.11.1

Thanks again :)

[maorfr]

Hey @deads2k, @liggitt,
Any news on this one?
Thanks!

[liggitt]

are you able to recreate on a version of the server with kubernetes/kubernetes#67902 fixed (v1.12.0+, v1.11.4+)?

[liggitt]

it looks like the bytes.Buffer you are providing as input implements a WriteTo function which io.Copy uses to attempt to write the entire buffer to the stream in one call, which results in a data frame that is too large.

We can fix this by wrapping the provided stdin stream in an io.Reader that only implements Read so that it always copies via a buffer.

In the meantime, you can do the same to work around the issue, instead of providing the bytes.Buffer directly as stdin

[liggitt]

fixed in kubernetes/kubernetes#70999

[maorfr]

Thanks @liggitt!

That solves the problem!

[maorfr]

Would you like me to close the issue or leave it open?

[liggitt]

Would you like me to close the issue or leave it open?

you can leave it, it'll get closed when the pull request merges