Fixes #24631 - httpboot module #605

lzap · 2018-08-16T09:29:00Z

New module for nicer UEFI HTTP BOOT implementation:

Please pay extra attention to security during review. I tested with paths like '/EFI/../../../etc/passwd' or similar. Directory listing is not allowed. Symlinks are followed by design and they are followed even outside of TFTP directory. Not sure if this is a problem, we can limit that tho if needed.

stbenjam

I haven't tried end-to-end yet, but the proxy module looks good itself. I couldn't get it to do any nefarious traversals using the common tricks, sinatra seems pretty smart.

stbenjam · 2018-08-16T17:10:44Z

modules/httpboot/httpboot_api.rb

+
+  get "/*" do
+    file = Pathname.new(params[:splat].first).cleanpath
+    real_file = Pathname.new(File.join(Proxy::TFTP::Plugin.settings.tftproot, file)).realpath


Should this be Proxy::Httpboot::Plugin.settings.root_dir instead?

Also, if you get /EFI or a non-existent file, we return 500. It'd be nice if we return the right http error codes, 404, 503, etc.

Good catch, yeah absolutely 404. Changing.

So I am implementing extra verbose checks just to be sure.

timogoebel

Thanks. Note, that tests are missing.

timogoebel · 2018-08-16T20:04:51Z

modules/httpboot/httpboot_api.rb

+  helpers ::Proxy::Helpers
+
+  get "/*" do
+    file = Pathname.new(params[:splat].first).cleanpath


Is this safe to prevent ../../../etc/shadow ?

Sinatra filters it out, among other tricky ways to do path traversal.

Yep I tested this, but I was wondering if we should allow following symlinks outside of TFTP root directory.

I can prevent from both possible security issues by comparing absolute file path with root directory - if it's not the same then we have an error. You know what, let me do that, just for case. I don't want my name on a CVE like this :-)

I am doing this, we can't be sure if e.g. older version of Sinatra does this.

lzap · 2018-08-17T07:23:11Z

I have amended the last commit, thank you for review.

stbenjam · 2018-08-21T21:08:44Z

Looks good, but a few notes. First, smart Proxy is returning a Server header with no value:

[vagrant@centos7-devel ~]$ curl -I http://192.168.73.1:7000/httpboot/grub2/grubx64.efi
HTTP/1.1 200 OK 
Content-Type: application/octet-stream
Server: 
Last-Modified: Tue, 21 Aug 2018 20:59:12 GMT
Content-Length: 1107968
X-Content-Type-Options: nosniff
Date: Tue, 21 Aug 2018 21:06:26 GMT
Connection: Keep-Alive

This is not valid although most things seem happy to accept it except the edk2 firmware which causes it to freeze. That took a better part of the afternoon to figure out why my vm was locking up.

Can you add that to the smart proxy somewhere?

stbenjam · 2018-08-21T21:11:22Z

The other thing is how grub2 doesn't handle relative paths (https://bugzilla.redhat.com/show_bug.cgi?id=1616395). I have a proposed fix to grub2, but even then relative paths (e.g. boot/centos7-vmlinuz or whatever) will be relative to the EFI image, e.g. the grub2 folder in our default set up. Users will need a symlink /var/lib/tftpboot/grub2/boot -> /var/lib/tftpboot/boot (which the installer can do unless you have another idea).

lzap · 2018-08-23T10:24:04Z

Thanks rebased, it was intentionally set to blank but I explained why we add this back: https://projects.theforeman.org/issues/14394

stbenjam

Tested, works great

timogoebel · 2018-08-24T09:11:50Z

config/settings.d/httpboot.yml.example

+---
+# Enable publishing of a given directory under /EFI and /httpboot paths.
+# Directory listing is not possible, symlinks are followed but not outside
+# of the rood directory specified in this file.


Nit: typoroot

timogoebel · 2018-08-24T09:12:41Z

lib/launcher.rb

@@ -50,7 +50,7 @@ def http_app(http_port, plugins = http_plugins)
        :DoNotListen => true,
        :Port => http_port, # only being used to correctly log http port being used
        :Logger => ::Proxy::LogBuffer::Decorator.instance,
-        :ServerSoftware => '',
+        :ServerSoftware => "foreman-proxy/#{Proxy::VERSION}",


It's common security practice to not publish the software versions. Do we need this?

It's a common practice in the enterprise, not in the open source world I'd say. But to the point - we do advertise the same via /features endpoint anyway. And we must not remove this - Foreman now checks foreman version and shows you a warning for compatibility.

If this is a major concern, please file a RM issue to encrypt the version or something like that. But I guess all we can do is obscurity not security in this case.

Ok, fair enough.

I would like to see it as a separate commit. It's not obvious that this is part of the httpboot and I wouldn't look at this commit in git log if I was looking for the change.

timogoebel · 2018-08-24T09:15:51Z

config/settings.d/httpboot.yml.example

+# Directory listing is not possible, symlinks are followed but not outside
+# of the rood directory specified in this file.
+
+# Enables the module, make sure to enable TFTP module as well to allow


You can use requires in the plugin spec to make sure the tftp module is loaded.

Oh cool this one was exactly what I was looking for. Thanks.

lzap · 2018-08-24T09:25:39Z

Thanks amended the two things.

timogoebel · 2018-08-24T09:35:33Z

Thanks, @lzap and @stbenjam.

ekohl · 2018-08-24T09:36:38Z

lib/launcher.rb

@@ -50,7 +50,7 @@ def http_app(http_port, plugins = http_plugins)
        :DoNotListen => true,
        :Port => http_port, # only being used to correctly log http port being used
        :Logger => ::Proxy::LogBuffer::Decorator.instance,
-        :ServerSoftware => '',
+        :ServerSoftware => "foreman-proxy/#{Proxy::VERSION}",


I would like to see it as a separate commit. It's not obvious that this is part of the httpboot and I wouldn't look at this commit in git log if I was looking for the change.

ekohl · 2018-08-24T09:40:07Z

test/httpboot/httpboot_api_test.rb

+require 'httpboot/httpboot_plugin'
+require 'httpboot/httpboot_api'
+
+ENV['RACK_ENV'] = 'test'


It feels weird to set an env var here. Is this needed?

Copy pasted from other file, thought it's required.

lzap · 2018-08-27T15:29:23Z

I did not want to do this in a separate commit, this might be backported (as it's new module and there cannot be regressions). The feature does not work end-to-end without this.

ekohl · 2018-08-27T16:03:59Z

How does the code not work without it? Does the client reject empty server strings?

stbenjam · 2018-08-27T17:08:00Z

How does the code not work without it? Does the client reject empty server strings?

It would work fine without it (or at least I would guess so), but Sinatra sends Server: as the header. This breaks the UEFI firmware that Qemu uses: https://bugzilla.tianocore.org/show_bug.cgi?id=1102.

I didn't look at the code, but I would guess the HTTP client does a strchr on space or something.

theforeman-bot added Needs testing Not yet reviewed labels Aug 16, 2018

stbenjam reviewed Aug 16, 2018

View reviewed changes

timogoebel reviewed Aug 16, 2018

View reviewed changes

lzap force-pushed the httpboot-24631 branch from 277367b to ecf2662 Compare August 17, 2018 07:23

lzap force-pushed the httpboot-24631 branch from ecf2662 to 7afcd9b Compare August 23, 2018 10:23

stbenjam approved these changes Aug 23, 2018

View reviewed changes

theforeman-bot removed the Not yet reviewed label Aug 23, 2018

timogoebel reviewed Aug 24, 2018

View reviewed changes

Fixes #24631 - httpboot module

8b84238

lzap force-pushed the httpboot-24631 branch from 7afcd9b to 8b84238 Compare August 24, 2018 09:25

timogoebel approved these changes Aug 24, 2018

View reviewed changes

timogoebel added Ready to merge and removed Needs testing labels Aug 24, 2018

timogoebel merged commit 8af0137 into theforeman:develop Aug 24, 2018

ekohl reviewed Aug 24, 2018

View reviewed changes

lzap deleted the httpboot-24631 branch August 27, 2018 15:29

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fixes #24631 - httpboot module #605

Fixes #24631 - httpboot module #605

lzap commented Aug 16, 2018

stbenjam left a comment

stbenjam Aug 16, 2018

stbenjam Aug 16, 2018

lzap Aug 17, 2018

lzap Aug 17, 2018

timogoebel left a comment

timogoebel Aug 16, 2018

stbenjam Aug 16, 2018

lzap Aug 17, 2018

lzap Aug 17, 2018

lzap commented Aug 17, 2018

stbenjam commented Aug 21, 2018 •

edited

Loading

stbenjam commented Aug 21, 2018 •

edited

Loading

lzap commented Aug 23, 2018

stbenjam left a comment

timogoebel Aug 24, 2018

timogoebel Aug 24, 2018

lzap Aug 24, 2018

timogoebel Aug 24, 2018

ekohl Aug 24, 2018

timogoebel Aug 24, 2018

lzap Aug 24, 2018

lzap commented Aug 24, 2018

timogoebel commented Aug 24, 2018

ekohl Aug 24, 2018

ekohl Aug 24, 2018

lzap Aug 27, 2018

lzap commented Aug 27, 2018

ekohl commented Aug 27, 2018

stbenjam commented Aug 27, 2018 •

edited

Loading

Fixes #24631 - httpboot module #605

Fixes #24631 - httpboot module #605

Conversation

lzap commented Aug 16, 2018

stbenjam left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

timogoebel left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

lzap commented Aug 17, 2018

stbenjam commented Aug 21, 2018 • edited Loading

stbenjam commented Aug 21, 2018 • edited Loading

lzap commented Aug 23, 2018

stbenjam left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

lzap commented Aug 24, 2018

timogoebel commented Aug 24, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

lzap commented Aug 27, 2018

ekohl commented Aug 27, 2018

stbenjam commented Aug 27, 2018 • edited Loading

stbenjam commented Aug 21, 2018 •

edited

Loading

stbenjam commented Aug 21, 2018 •

edited

Loading

stbenjam commented Aug 27, 2018 •

edited

Loading