Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Only upload files if changed #338

Closed
wenerme opened this issue Feb 24, 2022 · 4 comments · Fixed by #352
Closed

Only upload files if changed #338

wenerme opened this issue Feb 24, 2022 · 4 comments · Fixed by #352
Labels
enhancement New feature or request

Comments

@wenerme
Copy link

wenerme commented Feb 24, 2022

k0s and airgap bundle is about 500mb, should upload only changed

  1. size+mtime to detect change
  2. use sum to detect change

How do you think ?

BTW, I can contribute to this.

@wenerme wenerme changed the title upload file is changed only upload file if changed Feb 24, 2022
@kke kke changed the title only upload file if changed Only upload files if changed Feb 25, 2022
@kke kke added the enhancement New feature or request label Feb 25, 2022
@kke
Copy link
Contributor

kke commented Feb 25, 2022

k0s itself will only be reuploaded if you give a local path to it:

spec:
  hosts:
    - role: controller
      k0sBinaryPath: ~/k0s

The apply will also be treated as an upgrade, this is to ease k0s development / debugging with locally compiled binaries. If you let k0sctl manage the local cache, it should only upload the k0s binary if the version differs:

spec:
  hosts:
    - role: controller
      uploadBinary: true

For other uploads, to have a meaningful mtime, it should be set after upload to match the local file. I don't think checksumming is necessary.

@kke
Copy link
Contributor

kke commented Feb 25, 2022

It would be nice if the remote stat would be implemented in https://github.com/k0sproject/rig which is used by k0sctl. It should probably return a fs.FileInfo derivative, something like:

type FileInfo fs.FileInfo
func (c Linux) Stat(h Host, path string) (FileInfo, error) {}
func (w Windows) Stat(h Host, path string) (FileInfo, error) {}

@wenerme
Copy link
Author

wenerme commented Mar 7, 2022

Thought about add this to rig, but maybe this is too much, I finished my work by using ansible. Running k0s on zfs, there are extra steps before k0sctl apply.

phase in k0sctl ≈ ansible tasks, k0sctl may become something like apenella/go-ansible, but the executor is shell over ssh instead of ansible. From Configuration Complexity Clock, k0sctl is about 2am.

@twz123
Copy link
Member

twz123 commented Mar 25, 2022

I still see file uploads of unchanged files in my test setup using 0.13.0-rc.1 ... 🤔

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants