Add a way to specify you want only https urls from metalink #100

Closed
nirik opened this Issue Jun 22, 2015 · 5 comments

Projects

None yet

4 participants

@nirik
Member
nirik commented Jun 22, 2015

Some folks want all their traffic to use ssl, we should offer a option to the metalink url that makes it return just https using mirrors. Something like ?method=https or the like in the url.

Along with this we might consider mailing mirror admins and asking if they would update to https where they have https available.

Once enough mirrors offered https we could make it the default perhaps.

@ralphbean ralphbean added the medium label Jan 11, 2016
@adrianreber adrianreber added a commit to adrianreber/mirrormanager2 that referenced this issue Jun 5, 2016
@adrianreber adrianreber mirrorlist: metalink and mirrorlist should work with the same data
This is in preparation of #100 (Add a way to specify you want only https
urls from metalink). Metalink and mirrorlist creation are pretty
similar. Both get a list of host ids and URLs to fill generate the final
result: [(hostid, [url, url, url]], ...]. In the case of the mirrorlist this list
is trimmed to only return one URL per host: [(hostid, url], ...].

The mirrorlist_client.wsgi then expects the URL to be a string and not a
list. The metalink expects a list of URLs.

This changes the protocol trim function to still return a list of URLs,
even if only one URL is returned.

This in preparation to allow the user to specify a certain URL if
desired. The protocol trim function will then be called before creating
the metalink and thus the trim function needs to return a format which
the metalink creation understands (list of URLs). Also the mirrorlist
creation is adapted to expect a list of URLs instead of a string.

Signed-off-by: Adrian Reber <adrian@lisas.de>
ed7e459
@adrianreber adrianreber added a commit to adrianreber/mirrormanager2 that referenced this issue Jun 5, 2016
@adrianreber adrianreber mirrorlist: add protocol option to metalink and mirrorlist
Fixes: Add a way to specify you want only https
       urls from metalink (#100)

This adds the option protocol to the metalink and mirrorlist. With this
option it is possible to specify the protocol all returned URLs should
have.

$ curl -s \
"http://mirrors.stg.fedoraproject.org/mirrorlist?repo=rawhide&arch=x86_64&country=jp&protocol=ftp"
ftp://ftp.jaist.ac.jp/pub/Linux/Fedora/development/rawhide/Everything/x86_64/os/
ftp://ftp.riken.jp/Linux/fedora/development/rawhide/Everything/x86_64/os/

$ curl -s \
"http://mirrors.stg.fedoraproject.org/mirrorlist?repo=rawhide&arch=x86_64&country=jp&protocol=rsync"
rsync://ftp.jaist.ac.jp/pub/Linux/Fedora/development/rawhide/Everything/x86_64/os/
rsync://ftp.riken.jp/fedora/development/rawhide/Everything/x86_64/os/

The same works for the metalink. Instead of returning all protocols the
metalink will now contain only the URLs specified via protocol.

Signed-off-by: Adrian Reber <adrian@lisas.de>
236532a
@adrianreber
Contributor

This is now running on the Fedora production systems.

@mdomsch
Member
mdomsch commented Jun 24, 2016

Glad to see this, but the whole point of metalinks is to let the clients
decide what they want to use - give them all the possible URLs and then
they choose what they want. This properly belongs in a dnf configuration,
not MM2. As it stands, the user now has to change the dnf configuration
anyhow, to add the specifier on the metalink URL.

On Jun 22, 2015 1:33 PM, "Kevin Fenzi" notifications@github.com wrote:

Some folks want all their traffic to use ssl, we should offer a option to
the metalink url that makes it return just https using mirrors. Something
like ?method=https or the like in the url.

Along with this we might consider mailing mirror admins and asking if they
would update to https where they have https available.

Once enough mirrors offered https we could make it the default perhaps.


Reply to this email directly or view it on GitHub
#100.

@adrianreber
Contributor

Yes, I agree with @mdomsch. The official way to make metalink clients use HTTPS must be different. This needs some additional work on other parts (could be dnf). At least there is now a way for people who want to use it and we also know that it works and it is a starting point. It was also a good way to get more HTTPS URLs in the database. But, yes, now that more HTTPS URLs are in the database an option to make dnf prefer HTTPS URLs if available in the metalink sounds like a good idea.

@nirik
Member
nirik commented Jun 24, 2016

Sure, that makes some sense... I can ask dnf / librepo maintainers about options there.

@nirik
Member
nirik commented Jun 24, 2016

And there is already https://bugzilla.redhat.com/show_bug.cgi?id=1229050 (dnf should provide a protocol option) and https://bugzilla.redhat.com/show_bug.cgi?id=1273051 (librepo should provide a supported protocol list).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment