Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support GET-only HTTP servers for image sources #472

Merged
merged 3 commits into from
Nov 9, 2018
Merged

Conversation

moio
Copy link
Collaborator

@moio moio commented Nov 7, 2018

Some HTTP services, notably GitHub Releases, do not support HEAD requests to their endpoints.

terraform-provider-libvirt will thus fail on something along the lines of:

resource "libvirt_volume" "centos7_volume" {
  name = "centos7"
  source = "https://github.com/moio/sumaform-images/releases/download/4.0.0/centos7.qcow2"
}

With:

libvirt_volume.centos7_volume: Error accessing remote resource: https://github.com/moio/sumaform-images/releases/download/4.0.0/centos7.qcow2 - 403 Forbidden

I observed this phenomenon in sumaform when trying to use images built with Packer in CircleCI (sumaform-images project).

This PR removes the assumption a HEAD request will succeed. If it doesn't, it will defer getting the Content-Length we need for the libvirt volume definition at GET time.

Test code has been adapted too, namely, the internal Web Server also sets Content-Length now.

@moio moio requested a review from MalloZup November 7, 2018 14:32
Copy link
Collaborator

@MalloZup MalloZup left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thx @moio looks good i will have a detailed look once i have time

thank you much for this , hope to see you again in a PR 😎

@@ -143,12 +143,11 @@ func resourceLibvirtVolumeCreate(d *schema.ResourceData, meta interface{}) error

// update the image in the description, even if the file has not changed
size, err := img.Size()
if err != nil {
return err
if err == nil {
Copy link
Collaborator

@MalloZup MalloZup Nov 8, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if we do so, we are not checking and returning the error of the above function
or i am missing something here? 🤔 ( i think in case we need to handle a speicif error, we should do it instead of bypassing the whole errors)

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

At the moment there's not many other failure scenarios but I see the general point. If this is the only thing holding back merging just +1 this and I will fix it of course.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@moio thanks, yop otherwise looks clean to me 🎸

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like how you've pushed the size information down into the copier (most of a2e28a3). But I'd rather leave this particular hunk as it stands. To have httpImage.Size() support endpoints that 403 HEAD:

$ curl -sLI https://github.com/moio/sumaform-images/releases/download/4.0.0/centos7.qcow2 | grep 'HTTP\|Location'
HTTP/1.1 302 Found
Location: https://github-production-release-asset-2e65be.s3.amazonaws.com/...
HTTP/1.1 403 Forbidden

how about a "you can't make me read the body" GET ;) :

func (i *httpImage) Size() (uint64, error) {
  response, err := http.Head(i.url.String())
  if err != nil {
    return 0, err
  }
  if response.StatusCode == 403 {
    response, err = http.Get(i.url.String())
    if err != nil {
      return 0, err
    }
    response.Body.Close()
  }
  if response.StatusCode != 200 {
    ...
  }
  ...
}

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice thx @wking and @moio for this 🗾

Copy link
Collaborator

@MalloZup MalloZup left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎴

@moio
Copy link
Collaborator Author

moio commented Nov 9, 2018

Oooh how much more I like my patch now!

Thank you very much @wking!

Copy link
Collaborator

@MalloZup MalloZup left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Niiicee now

@MalloZup
Copy link
Collaborator

MalloZup commented Nov 9, 2018

For curiosity how @wking helped here? :) i am walking at moment i will merge all pr after including the wking one. Thx for prs

@moio moio merged commit 8597016 into dmacvicar:master Nov 9, 2018
@moio
Copy link
Collaborator Author

moio commented Nov 9, 2018

@MalloZup #472 (comment)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants