oss-perf: modify default protocol for ossfs URL #972
Conversation
k8s-ci-robot
added
cncf-cla: yes
k8s-ci-robot
added
size/L
AlbeeSo
force-pushed
the
feat/modify-url-protocol
branch
from
e789fc7
to
e091c0b
Compare
pkg/oss/utils.go
Outdated
| if strings.HasPrefix(URL, "http://") || strings.HasPrefix(URL, "https://") { | ||
| return | ||
| } | ||
| if strings.HasSuffix(URL, "-internal.aliyuncs.com") { |
There was a problem hiding this comment.
I think at least URL like oss-cn-beijing-internal.aliyuncs.com/ is vaild. Paths other than '/' should be considered too. scheme and hostname parts of URL are case insensitive, and should be considered.
I think we may parse the URL with net/url package.
There was a problem hiding this comment.
https://help.aliyun.com/zh/oss/user-guide/regions-and-endpoints?spm=a2c4g.11186623.0.0.1e2030f0DijQDq#section-oao-7ao-11f
Endpoint not matches "(http://,https://)oss-{{region}}-{{pub/internal/null}}-aliyuncs.com" is invalid to access OSS by ossfs. Use the raw URL or the modified one will both cause an error.
There was a problem hiding this comment.
I don't get it. The provided link shows endpoint, which is only the hostname part of URL. URL is a term defined by rfc3986. Case insensitivity of scheme should be supported:
Although schemes are case-
insensitive, the canonical form is lowercase and documents that
specify schemes must do so with lowercase letters. An implementation
should accept uppercase letters as equivalent to lowercase in scheme
names (e.g., allow "HTTP" as well as "http") for the sake of
robustness but should only produce lowercase scheme names for
consistency.
DNS name should also be case insensitive (see rfc4343).
As for the path component. I've verified that oss-cn-beijing-internal.aliyuncs.com/ this URL works currently. We may also want to support this too.
There was a problem hiding this comment.
In fact, value of -ourl= for s3fs/ossfs is little bit different from a real URL. It's for the Endpoint part of Host, which is equal to BucketName.Endpoint. So 'URL' here can only be endpoint or the endpoint with protocol. I fixed case like oss-cn-beijing-internal.aliyuncs.com/ .
https://help.aliyun.com/zh/oss/developer-reference/common-http-headers?spm=a2c4g.11186623.0.0.3b2b3864sI0iwV#section-eq1-b2y-wdb
|
/lgtm |
k8s-ci-robot
added
the
lgtm
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: AlbeeSo, mowangdk The full list of commands accepted by this bot can be found here. The pull request process is described here
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
k8s-ci-robot
added
the
approved
What type of PR is this?
/kind feature
What this PR does / why we need it:
In current version, CSI will not check protocol of ossfs URL. Then, ossfs will use default protocol (http) if it is not specified explicitly.
In this PR, protocol will be set by CSI. If instance needs to access OSS with public endpoint, use https as default protocol instead.
Which issue(s) this PR fixes:
Fixes #
Special notes for your reviewer:
Does this PR introduce a user-facing change?
Additional documentation e.g., KEPs (Kubernetes Enhancement Proposals), usage docs, etc.: