Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

OpenStack: support "file" scheme for custom os image urls #3273

Merged
merged 2 commits into from Mar 27, 2020

Conversation

Fedosin
Copy link
Contributor

@Fedosin Fedosin commented Mar 11, 2020

Now we support http(s) schemes only, but for disconnected installs it's very convenient to specify the local file path to the image file.

This commit adds "file" scheme support, so users can set the location as "file:///path/to/image".

@openshift-ci-robot openshift-ci-robot added the size/M Denotes a PR that changes 30-99 lines, ignoring generated files. label Mar 11, 2020
@Fedosin Fedosin changed the title OpenStack: support "file" scheme for custom image urls OpenStack: support "file" scheme for custom os image urls Mar 11, 2020
@rheinzma
Copy link

Can I still refrence the file with SHA Checksum so the installer checks the integrity of the file provided ? file:///path/to/file?sha256=xxxxx ?

@Fedosin
Copy link
Contributor Author

Fedosin commented Mar 11, 2020

@rheinzma unfortunately no, but I think checksum validation doesn't make sense here, since you normally trust files on your local file system. If necessary you can check it before the installation with sha256sum command.

@Fedosin
Copy link
Contributor Author

Fedosin commented Mar 11, 2020

/retest

@Fedosin
Copy link
Contributor Author

Fedosin commented Mar 11, 2020

/retest

@jstuever
Copy link
Contributor

/uncc @jstuever
/cc @iamemilio

@openshift-ci-robot openshift-ci-robot requested review from iamemilio and removed request for jstuever March 13, 2020 16:12
@Fedosin
Copy link
Contributor Author

Fedosin commented Mar 14, 2020

/retest

@iamemilio
Copy link

/lgtm

@openshift-ci-robot openshift-ci-robot added the lgtm Indicates that a PR is ready to be merged. label Mar 18, 2020
@iamemilio
Copy link

/lgtm

Comment on lines 52 to 62
_, err := url.ParseRequestURI(rhcosImage)
// We support only 'http(s)' and 'file' schemes, URLs with other schemes are considered as Glance image
// names.
url, err := url.Parse(rhcosImage)
if err != nil {
return rhcosImage, false
}

if url.Scheme != "http" && url.Scheme != "https" && url.Scheme != "file" {
return rhcosImage, false
}

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

technically one could provide a ftp URI and that would still work, but now providing something like that means it's going to try to assume that as glance image id.

I think there needs to be better validation here in terms of whats allowed and what's not.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Now the installation fails if the user provides an ftp or any other non-http(s) url, because we will try to download the file using http protocol. This is wrong behavior.
With this patch I limit the supported schemes to http(s) and file, as it is said in the documentation. URLs with other schemes will be considered as Glance image names.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

but this function will return result for ftp as if that's an glance image though still.. will is incorrect.

we should proabably have validation to restrict the scheme to these values or empty for glance image.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In accordance with the documentation and as originally planned:

  1. A valid http(s) URL translates to the address from where we're going to download the file.
  2. A valid file URL translates to the image file location on the local file system.
  3. All other strings are translated to the Glance image name, even if they are ftp or s3 URLs.

As an alternative solution, I can offer to limit the schemes and return an error if the scheme is not http or file. That is:

  1. A valid http(s) URL translates to the address from where we're going to download the file.
  2. A valid file URL translates to the image file location on the local file system.
  3. Any other URL-like string leads to the error: "Unsupported URL scheme".
  4. All other strings are translated to the Glance image name.

@openshift-ci-robot openshift-ci-robot removed the lgtm Indicates that a PR is ready to be merged. label Mar 24, 2020
@Fedosin Fedosin force-pushed the file_locations branch 2 times, most recently from 02a5be4 to a252113 Compare March 24, 2020 19:47
@Fedosin
Copy link
Contributor Author

Fedosin commented Mar 24, 2020

/test e2e-openstack

@Fedosin
Copy link
Contributor Author

Fedosin commented Mar 25, 2020

/retest

Now we support http(s) schemes only, but for disconnected installs
it's very convenient to specify the local file path to the image file.

This commit adds "file" scheme support, so users can set the location as
"file:///path/to/image".
}
case "file":
localFilePath = filepath.FromSlash(url.Path)
default:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Personally I would have liked it to be part of validation of the install-config.yaml

But this is fine too.

/approve

@openshift-ci-robot openshift-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Mar 26, 2020
@abhinavdahiya
Copy link
Contributor

/approve

@openshift-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: abhinavdahiya

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@Fedosin
Copy link
Contributor Author

Fedosin commented Mar 27, 2020

/test e2e-openstack

@iamemilio
Copy link

/lgtm

@openshift-ci-robot openshift-ci-robot added the lgtm Indicates that a PR is ready to be merged. label Mar 27, 2020
@openshift-bot
Copy link
Contributor

/retest

Please review the full test history for this PR and help us cut down flakes.

1 similar comment
@openshift-bot
Copy link
Contributor

/retest

Please review the full test history for this PR and help us cut down flakes.

@openshift-merge-robot openshift-merge-robot merged commit f8e133f into openshift:master Mar 27, 2020
@openshift-ci-robot
Copy link
Contributor

openshift-ci-robot commented Mar 27, 2020

@Fedosin: The following tests failed, say /retest to rerun all failed tests:

Test name Commit Details Rerun command
ci/prow/e2e-aws-steps a252113 link /test e2e-aws-steps
ci/prow/e2e-aws-scaleup-rhel7 26a522a link /test e2e-aws-scaleup-rhel7
ci/prow/e2e-libvirt 26a522a link /test e2e-libvirt

Full PR test history. Your PR dashboard. Please help us cut down on flakes by linking to an open issue when you hit one in your PR.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. lgtm Indicates that a PR is ready to be merged. size/M Denotes a PR that changes 30-99 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

8 participants