Item14380: Item14544: Initial rewrite of Proxy detect

Basically functional, but needs more work: - mod_perl not covered. - Forwarded-For should be separated from other Forwarded-* header permissions - Needs checkers to suggest correct configuration Tested with Apache2.4 proxy server, using non-standard http port, and https, with and without inclusion of -Port and -Proto headers which are not enabled by default. Needs testing with nginx.
foswiki · Nov 29, 2017 · 3745b32 · 3745b32
1 parent e798eee
commit 3745b32
Show file tree

Hide file tree

Showing 6 changed files with 161 additions and 74 deletions.
diff --git a/core/data/System/InstallationGuide.txt b/core/data/System/InstallationGuide.txt
@@ -1,4 +1,4 @@
-%META:TOPICINFO{author="ProjectContributor" date="1502239930" format="1.1"  version="1"}%
+%META:TOPICINFO{author="ProjectContributor" date="1511982212" format="1.1"  version="1"}%
 %META:TOPICPARENT{name="AdminDocumentationCategory"}%
 <noautolink>
 ---+!! Installation Guide
@@ -60,7 +60,7 @@ At some point, you will want to re-visit [[#ConfigureFoswiki][Configuring Foswik
       * If you have shell access, you can download the distribution directly using using this command: <pre>wget  https://sourceforge.net/projects/foswiki/files/latest/download</pre>
       * If you do not have shell access to your web server host, see the section [[#UploadingFoswikiDistribution][Uploading the Foswiki distribution to your web server host]] below.
    3 *Unpack the distribution file:*
-      * go to your web directory (usually /var/www) or in any directory you what to install Foswiki (designed after by  /path/to/foswiki/)
+      * go to your web directory (usually /var/www) or in any directory you what to install Foswiki (designated after by =/path/to/foswiki/=)
       * Untar and gunzip the distribution using this command: (modify to match version number)<verbartim>
        tar -xzvf foswiki-VERSION.tgz
       </verbatim>A new subdirectory called Foswiki-VERSION will be created.
@@ -162,14 +162,17 @@ In details:
 
 <div class='foswikiHelp'>
    * %T% If the Foswiki installation can be accessed by more than one !protocol://domain, ensure the additional alternative URLs are set in ={PermittedRedirectHostUrls}=. *Example:* if ={DefaultUrlHost}= is set to =https://wiki.company.com=, an example ={PermittedRedirectHostUrls}= might contain: <verbatim class='html'>https://company.com, http://111.222.123.234</verbatim>
-   * If your server requires a Proxy in order to access external resources like your mail server, this is configured on the "Security and Authentication" tab, "Proxies" sub-tab. Complete that before proceeding with the E-mail configuration.
+   * If your Foswiki server is hidden behind a reverse proxy, you should check [[https://foswiki.org/Support/FoswikiBehindReverseProxy][Foswiki Behind Reverse Proxy]] for more configuration hints.
    * If you do not want to or are unable to configure from the web interface, there is an alternate command line configuration documented in %IF{"$skin = 'plain'" then="<nop>System.<nop>InstallationGuide#ConfiguringFoswikiManually in your installation" else="[[InstallationGuide#ConfiguringFoswikiManually]]"}%.
 </div>
 
 %IF{"$skin = 'plain'" then="<div class='foswikiHelp' style='border:2px orange solid' >$percntT$percntNow that your Foswiki installation is functional, you are encouraged to view this installation guide on your Foswiki site rather than the static =INSTALL.html= file included in the distribution. This will make it easier to implement the steps which follow because you will be able to use the embedded hyperlinks to jump directly to the referenced pages in your installation. To view this guide in your installation, follow the link provided in the bootstrap banner, or enter =System.InstallationGuide= in the jump-box.</div>"}%
 
 ---+++ Step 6: Configure email
 
+<div class='foswikiHelp'> %T% If your server requires a Proxy in order to access external resources like your mail server, this is configured on the "Security and Authentication" tab, "Proxies" sub-tab.
+Complete that before proceeding with the E-mail configuration. </div>
+
 Outgoing e-mail is required for Foswiki to send registration confirmations, notifications of changes, password reset requests, etc.
 
    1 Select the =Mail= tab in left bar of confgiure and fill out the following parameters:

diff --git a/core/lib/Foswiki.spec b/core/lib/Foswiki.spec
@@ -1075,16 +1075,24 @@ $Foswiki::cfg{AccessibleHeaders} = ['Accept-Language', 'User-Agent'];
 # http://username:password@proxy.your.company:8080.
 $Foswiki::cfg{PROXY}{HOST} = undef;
 
-# **BOOLEAN LABEL="Client IP" **
-# Foswiki normally uses the REMOTE_ADDRESS as the client IP.  If Foswiki is behind
-# a proxy, Foswiki will see the proxy's address as the Client IP.  Enable this switch if you
-# want foswiki to recover the real Client IP from the =X-Forwarded-For= header. 
-# *Caution:* This header is easily spoofed. Only enable this flag if you are certain that
-# you trust the Proxy server.
+# **BOOLEAN LABEL="Forwarded Headers" **
+# Use the =Forwarded-*= headers to determine the Client IP, URL Protocol, Hostname and Port.
+# Foswiki normally uses the local server information for identifying the connection information.
+# However when a proxy server, load balancer, SSL Accelerator or other intermediate
+# devices are present, this local information will most likely be incorrect.
+# Enable this setting to make use of the Proxy headers provided by the Client or intermediate devices:
+#    * =X-Forwarded-For= _Identifies the client IP, overrides REMOTE_ADDRESS variable._
+#    * =X-Forwarded-Host= _Captures the hostname used by the client in it's initial request._
+#    * =X-Forwarded-Proto= _Specifies if the client used an HTTP or HTTPS secure connection._
+#    * =X-Forwarded-Port= _Specifies the original port used by the client._
+#    * =Forwarded:= _New standards based header replaces the X-Forwarded* headers._
+# <p/>
+# *Caution:* These headers are easily spoofed. Only enable this flag if you are certain that
+# a proxy server exists and that you trust the Proxy server.
 # <p/>
 # Note that this setting also impacts CGI Session IP matching. Changing this setting
 # will break all active sessions behind the proxy and require re-authentication.
-$Foswiki::cfg{PROXY}{UseForwardedForHeader} = $FALSE;
+$Foswiki::cfg{PROXY}{UseForwardedHeaders} = $FALSE;
 
 #---++ Anti-spam
 # Foswiki incorporates some simple anti-spam measures to protect

diff --git a/core/lib/Foswiki/Configure/Bootstrap.pm b/core/lib/Foswiki/Configure/Bootstrap.pm
@@ -421,7 +421,8 @@ Called by bootstrapConfig.  This handles the web environment specific settings o
 =cut
 
 sub bootstrapWebSettings {
-    my $script = shift;
+    my $script   = shift;
+    my $proxyMsg = '';
 
     local $Data::Dumper::Sortkeys = 1;
     local $Data::Dumper::Varname  = 'ENV';
@@ -440,15 +441,31 @@ sub bootstrapWebSettings {
         return 'Phase 2 boostrap bypassed - n/a in CLI environment\n';
     }
 
-    my $protocol = $ENV{HTTPS} ? 'https' : 'http';
     $Foswiki::cfg{ForceDefaultUrlHost} = 0;
 
-# Figure out the DefaultUrlHost - First check if there is a proxy forwarding requests
-# SMELL: This fails to account for multiple proxies where the forwarded_host is a list of proxies.
-    if ( $ENV{HTTP_X_FORWARDED_HOST} ) {
+    my ( $client, $protocol, $host, $port, $proxy ) =
+      Foswiki::Engine::_getConnectionData(1);
 
-# Detect if HTTPS in use.  Browsers appear to set the UPGRADE flag, and the json requests
-# for configure all come in with a https referer.
+    print STDERR
+"AUTOCONFIG: Engine detected:  client ($client) proto ($protocol) host ($host) port ($port) proxy ($proxy) )\n"
+      if (TRAUTO);
+    $port = ( $port && $port != 80 && $port != 443 ) ? ":$port" : '';
+
+    if ($proxy) {
+        $Foswiki::cfg{ForceDefaultUrlHost} = 1;
+        $proxyMsg = <<PROXY;
+%BR% A Proxy server was detected. {ForceDefaultUrlHost} has been enabled. 
+
+If this page is rendered without any styles and you are using SSL (https), your proxy server may be misconfigured.
+It must generate the =X-Forwarded-Proto header=. Try adding ?SSL=1 to the Foswiki URL to bypass this issue.
+PROXY
+        print STDERR
+"AUTOCONFIG: Proxy configuration detected. {ForceDefaultUrlHost} has been enabled.\n"
+          if (TRAUTO);
+    }
+
+    # Double-check protocol in case headers are incomplete
+    if ( $protocol eq 'http' ) {
         if (
             ( $ENV{QUERY_STRING} && $ENV{QUERY_STRING} =~ m/\bSSL=1\b/i )
             || (   $ENV{HTTP_REFERER}
@@ -458,47 +475,12 @@ sub bootstrapWebSettings {
         {
         # Browser is asking for https or refered from https, so override protcol
             $protocol = 'https';
-            print STDERR "AUTOCONFIG: Detected HTTPS\n";
+            print STDERR
+"AUTOCONFIG: Overriding protocol to HTTPS from QUERY_STRING or REFERRER\n";
         }
-
-        $Foswiki::cfg{DefaultUrlHost} =
-          "$protocol://" . $ENV{HTTP_X_FORWARDED_HOST};
-        $Foswiki::cfg{ForceDefaultUrlHost} =
-          1;    # Force the URL host when behind a proxy
-
-        print STDERR "AUTOCONFIG: Forcing DefaultUrlHost "
-          . $Foswiki::cfg{DefaultUrlHost}
-          . " from Proxy HTTP_X_FORWARDED_HOST "
-          . $ENV{HTTP_X_FORWARDED_HOST} . " \n"
-          if (TRAUTO);
-    }
-    elsif ( $ENV{HTTP_HOST} ) {
-        $Foswiki::cfg{DefaultUrlHost} = "$protocol://$ENV{HTTP_HOST}";
-        print STDERR
-"AUTOCONFIG: Set DefaultUrlHost $Foswiki::cfg{DefaultUrlHost} from HTTP_HOST $ENV{HTTP_HOST} \n"
-          if (TRAUTO);
-    }
-    elsif ( $ENV{SERVER_NAME} ) {
-        $Foswiki::cfg{DefaultUrlHost} = "$protocol://$ENV{SERVER_NAME}";
-        print STDERR
-"AUTOCONFIG: Set DefaultUrlHost $Foswiki::cfg{DefaultUrlHost} from SERVER_NAME $ENV{SERVER_NAME} \n"
-          if (TRAUTO);
-    }
-    elsif ( $ENV{SCRIPT_URI} ) {
-        ( $Foswiki::cfg{DefaultUrlHost} ) =
-          $ENV{SCRIPT_URI} =~ m#^(https?://[^/]+)/#;
-        print STDERR
-"AUTOCONFIG: Set DefaultUrlHost $Foswiki::cfg{DefaultUrlHost} from SCRIPT_URI $ENV{SCRIPT_URI} \n"
-          if (TRAUTO);
     }
-    else {
 
-        # OK, so this is barfilicious. Think of something better.
-        $Foswiki::cfg{DefaultUrlHost} = "$protocol://localhost";
-        print STDERR
-"AUTOCONFIG: barfilicious: Set DefaultUrlHost $Foswiki::cfg{DefaultUrlHost} \n"
-          if (TRAUTO);
-    }
+    $Foswiki::cfg{DefaultUrlHost} = "$protocol://" . $host . $port;
 
 # Examine the CGI path.   The 'view' script it typically removed from the
 # URL when using "Short URLs.  If this BEGIN block is being run by
@@ -645,6 +627,7 @@ To complete the bootstrap process you should either:
 %BR% *You have been logged in as a temporary administrator.*
 Any requests made to this Foswiki will be treated as requests made by an administrator with full rights
 Your temporary administrator rights will "stick" until you've logged out from this session.
+$proxyMsg
 BOOTS
 
     #'

diff --git a/core/lib/Foswiki/Engine.pm b/core/lib/Foswiki/Engine.pm
@@ -502,6 +502,107 @@ sub write {
     ASSERT('Pure virtual method - should never be called');
 }
 
+=begin TML
+
+---++ _getConnectionData($detectProxy) = ( $client, $proto, $host, $port, $proxy )
+
+Utility routine obtains the connection information from the headers.
+
+If $Foswiki::cfg{PROXY}{UseForwardedHeaders} is true, or if the $detectProxy flag is set
+this routine will return the connection data from the X-Forwarded-* or Forwarded: headers.
+
+Headers: 
+   * HTTP_X_FORWARDED_HOST 
+   * HTTP_X_FORWARDED_PORT
+   * HTTP_X_FORWARDED_FOR
+   * HTTP_X_FORWARDED_PROTO
+
+=cut
+
+sub _getConnectionData {
+
+    my $detectProxy = shift;
+
+    my ( $client, $proto, $host, $port, $proxy );
+
+    # These are the defaults populated in a conventional server, no proxy
+    $client = $ENV{REMOTE_ADDR};
+    $proto  = ( $ENV{HTTPS} && uc( $ENV{HTTPS} ) eq 'ON' ) ? 'https' : 'http';
+    $host   = $ENV{HTTP_HOST} || $ENV{SERVER_NAME};
+    $port   = $ENV{SERVER_PORT};
+
+    if ( $detectProxy || $Foswiki::cfg{PROXY}{UseForwardedHeaders} ) {
+        my ( $fwdClient, $fwdProto, $fwdHost, $fwdPort, $hostport );
+
+        if ( my $hdr = $ENV{HTTP_X_FORWARDED_FOR} ) {
+            my $ip = ( split /,\s?/, $hdr )[0];
+            if ( defined $ip ) {
+                $fwdClient = $ip;
+                print STDERR
+"AUTOCONFIG: Client IP detected from Proxy header ($hdr): $ip\n"
+                  if ($detectProxy);
+                $proxy = 1;
+            }
+        }
+
+        if ( my $hdr = $ENV{HTTP_X_FORWARDED_HOST} ) {
+            my $first = ( split /,\s?/, $hdr )[0];
+            if ( defined $first ) {
+                $first =~ s/:(\d+)$//;
+                $hostport = $1 if ($1);
+                $fwdHost = $first;
+                print STDERR
+"AUTOCONFIG: Hostname detected from Proxy header ($hdr): $fwdHost\n"
+                  if ($detectProxy);
+                $proxy = 1;
+            }
+        }
+
+        if ( my $hdr = $ENV{HTTP_X_FORWARDED_PROTO} ) {
+            my $first = ( split /,\s?/, $hdr )[0];
+            if ( defined $first ) {
+                $fwdProto = $first;
+                print STDERR
+"AUTOCONFIG: Protocol detected from Proxy header ($hdr): $fwdProto\n"
+                  if ($detectProxy);
+                $proxy = 1;
+            }
+        }
+
+        if ( my $hdr = $ENV{HTTP_X_FORWARDED_PORT} ) {
+            my $first = ( split /,\s?/, $hdr )[0];
+            if ( defined $first ) {
+                $fwdPort = $first;
+                print STDERR
+"AUTOCONFIG: Port detected from Proxy header ($hdr): $fwdPort\n"
+                  if ($detectProxy);
+                $proxy = 1;
+            }
+        }
+        elsif ($hostport) {
+            $fwdPort = $hostport;
+            print STDERR
+"AUTOCONFIG: Port recovered from Proxy FOWARDED_HOST header: $fwdPort\n"
+              if ($detectProxy);
+        }
+
+        $client = $fwdClient if $fwdClient;
+
+        $host = $fwdHost if $fwdHost;
+        $port = $fwdPort if $fwdPort;
+
+        if ( !defined $fwdProto && defined $fwdPort && $fwdPort == 443 ) {
+            $fwdProto = 'https';
+            print STDERR
+              "AUTOCONFIG: proto overridden to https due to port 443 detected\n"
+              if ($detectProxy);
+        }
+        $proto = $fwdProto if $fwdProto;
+    }
+
+    return ( $client, $proto, $host, $port, $proxy );
+}
+
 1;
 __END__
 Foswiki - The Free and Open Source Wiki, http://foswiki.org/

diff --git a/core/lib/Foswiki/Engine/CGI.pm b/core/lib/Foswiki/Engine/CGI.pm
@@ -105,24 +105,15 @@ sub run {
 sub prepareConnection {
     my ( $this, $req ) = @_;
 
-    $req->remoteAddress( $ENV{REMOTE_ADDR} );
-    if ( $Foswiki::cfg{PROXY}{UseForwardedForHeader}
-        && defined $ENV{HTTP_X_FORWARDED_FOR} )
-    {
-        my @addrs = split /,\s?/, $ENV{HTTP_X_FORWARDED_FOR};
-        $req->remoteAddress( $addrs[0] );
-    }
+    my ( $client, $protocol, $host, $port ) =
+      Foswiki::Engine::_getConnectionData();
 
+    $req->remoteAddress($client);
     $req->method( $ENV{REQUEST_METHOD} );
-
-    if ( $ENV{HTTPS} && uc( $ENV{HTTPS} ) eq 'ON' ) {
-        $req->secure(1);
-    }
-
-    if ( $ENV{SERVER_PORT} && $ENV{SERVER_PORT} == 443 ) {
+    if ( $protocol eq 'https' ) {
         $req->secure(1);
     }
-    $req->serverPort( $ENV{SERVER_PORT} );
+    $req->serverPort($port);
 }
 
 sub prepareQueryParameters {

diff --git a/core/lib/Foswiki/Request.pm b/core/lib/Foswiki/Request.pm
@@ -294,15 +294,16 @@ sub url {
     }
     $name =~ s(//+)(/)g;
     if ($full) {
+
         if ( $Foswiki::cfg{ForceDefaultUrlHost} ) {
             $url = $Foswiki::cfg{DefaultUrlHost};
         }
         else {
-            my $vh = $this->header('X-Forwarded-Host') || $this->header('Host');
-            $url =
-                $vh
-              ? $this->protocol . '://' . $vh
-              : $Foswiki::cfg{DefaultUrlHost};
+            my ( $client, $protocol, $host, $port ) =
+              Foswiki::Engine::_getConnectionData();
+            $port = ( $port && $port != 80 && $port != 443 ) ? ":$port" : '';
+
+            $url = $protocol . '://' . $host . $port;
         }
         return $url if $base;
         $url .= $name;