Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Allow spec.hosts.files to have url sources, set owner, fix dir recursion #231

Merged
merged 3 commits into from Oct 28, 2021
Merged

Allow spec.hosts.files to have url sources, set owner, fix dir recursion #231

merged 3 commits into from Oct 28, 2021

Conversation

kke
Copy link
Contributor

@kke kke commented Sep 30, 2021 •
edited

Fixes #224
Fixes #244

File src can now be a URL:

spec:
  hosts:
    - ssh:
       address: 10.0.0.1
    files:
      - name: image-bundle
        src: https://10.0.0.1/image-bundle.tgz
        dstDir: /root/foo/

Also dst is added so it is possible to rename the files:

      - name: image-bundle
        src: https://10.0.0.1/image-bundle.tgz
        dst: /root/foo/bundle.tar.gz

You can also combine it with dstDir:

      - name: image-bundle
        src: https://10.0.0.1/image-bundle.tgz
        dstDir: /root/foo
        dst: bundle.tar.gz

The file/dir ownership can now be set:

      - name: image-bundle
        src: ./upload
        dstDir: /var/foo/images
        user: foo
        group: foo

The perm can now be a string or an octal (or an integer if you really want to):

      - name: image-bundle
        perm: 0700
      - name: image-bundle
        perm: "0700"

If you omit the permission, local permissions will be replicated. Local executables will be executable on remote hosts.

You can set the perm-mode for created directories:

      - name: image-bundle
        src: ./upload
        dstDir:/var/foo/images
        dirPerm: 0700

If the src is a directory, it can now recurse into it.

@bephinix
Copy link
Contributor

@kke Looks great, this should fix a lot of use cases. 💯

This was referenced Oct 12, 2021
Closed
Closed
@kke
Copy link
Contributor Author

kke commented Oct 12, 2021

Note to self: see it doesn't break the permissions that were fixed in #248

@kke
Copy link
Contributor Author

kke commented Oct 19, 2021

Needs smoke tests for peace of mind

@kke
Copy link
Contributor Author

kke commented Oct 19, 2021

Some unexpectedly complicated stuff going on here ..

@kke
Allow spec.host.files to have url sources
5949cf1 
Lint

Add smoke tests, this probably fails as the abs-paths need to be cut

Maybe fix globbing

The amount of little stuff for such a simple thing..

Remove debug output

Dont upload plain directories

Simplified, fixed

Wrong dir in test

Another wrong dir

Chmod test

Extra space

More debug output

Even more debug output

Go get deprecated

Wrong dir again

Configurable chown and dirchmod

Test owner and dirPerm

Update README.md

Chown fixes

Quotes quotes quotes

Rebase aftermath
@kke kke marked this pull request as ready for review October 26, 2021 12:11
@kke kke changed the title Allow spec.host.files to have url sources Allow spec.host.files to have url sources, improve owner/permission setting and dir recursion Oct 26, 2021
@kke
Copy link
Contributor Author

kke commented Oct 26, 2021

@bephinix It's finally done. It has integration tests so I'm fairly confident it works, but feel free to play around and confirm if you have the time :)

@kke kke requested a review from jnummelin October 26, 2021 12:18
@kke kke changed the title Allow spec.host.files to have url sources, improve owner/permission setting and dir recursion Allow spec.hosts.files to have url sources, improve owner/permission setting and dir recursion Oct 26, 2021
@kke kke changed the title Allow spec.hosts.files to have url sources, improve owner/permission setting and dir recursion Allow spec.hosts.files to have url sources, improve owner/permission setting, fix dir recursion Oct 26, 2021
@kke kke changed the title Allow spec.hosts.files to have url sources, improve owner/permission setting, fix dir recursion Allow spec.hosts.files to have url sources, configurable owner, fix dir recursion Oct 26, 2021
@kke kke changed the title Allow spec.hosts.files to have url sources, configurable owner, fix dir recursion Allow spec.hosts.files to have url sources, set owner, fix dir recursion Oct 26, 2021
@kke kke added bug Something isn't working enhancement New feature or request labels Oct 26, 2021
pdf
pdf reviewed Oct 28, 2021
View reviewed changes
README.md Outdated Show resolved Hide resolved
@kke
Copy link
Contributor Author

kke commented Oct 28, 2021

There's other stuff that warrant a beta, I'll merge this and we'll fix any issues that come up.

@kke kke merged commit a18f0e5 into main Oct 28, 2021
@kke kke deleted the url-downloads branch October 28, 2021 13:23
@bephinix
Copy link
Contributor

bephinix commented Nov 1, 2021

@kke Thanks! Seems to work but I will test it the next days in more complex deployments. :)

@danmx
Copy link

danmx commented Nov 8, 2021

Will this work when the file already exists?

@kke
Copy link
Contributor Author

kke commented Nov 9, 2021

Will this work when the file already exists?

I believe it will overwrite it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@pdf pdf pdf left review comments

@jnummelin jnummelin Awaiting requested review from jnummelin

Assignees
No one assigned
Labels
bug Something isn't working enhancement New feature or request
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Files upload fails when passed a directory src Option to download Files from Remote Locations
4 participants
@kke @bephinix @danmx @pdf