From 59a89027fd1f2fe67ad7e431d63d9b36f854f007 Mon Sep 17 00:00:00 2001 From: Mateusz Uzdowski Date: Wed, 29 Jan 2020 14:14:19 +1300 Subject: [PATCH] Update external http request documentation (proxy information). --- .../external_http_requests_with_proxy.md | 84 +++++++------------ 1 file changed, 32 insertions(+), 52 deletions(-) diff --git a/docs/en/03_How_tos/external_http_requests_with_proxy.md b/docs/en/03_How_tos/external_http_requests_with_proxy.md index b41d13ea..34ef2d88 100644 --- a/docs/en/03_How_tos/external_http_requests_with_proxy.md +++ b/docs/en/03_How_tos/external_http_requests_with_proxy.md @@ -3,26 +3,39 @@ summary: How to access external resources via the CWP proxy. # External HTTP requests with proxy -All CWP environments are positioned behind a proxy. All requests out to external services must go through this proxy. +All requests to resources outside of your stack must use a proxy. CWP environments have no direct, unproxied HTTP connectivity out to the internet nor to other CWP stacks internally. -By default, we configure the PHP environment with `http_proxy` and `https_proxy` environment variables, as well as the `SS_OUTBOUND_PROXY` and `SS_OUTBOUND_PROXY_PORT` constants. These environment variables can be used within your code to send requests through the proxy. +## Explicit configuration -If you are seeing issues connecting to an external service, double check to make sure you are going through the proxy -(compare against the constants `SS_OUTBOUND_PROXY` and `SS_OUTBOUND_PROXY_PORT`). +To explicitly configure the proxy when making external HTTP requests, use the `SS_OUTBOUND_PROXY` and `SS_OUTBOUND_PROXY_PORT` PHP constants provided. Configuration of the outgoing proxy is dependent on the library you are using, so please consult relevant documentation. -Because of this proxy, outgoing requests have a different IP than configured for incoming requests to your CWP hostname. +For example curl can be configured by setting the `CURLOPT_PROXY` option via `curl_setopt` before making the actual request. + +If your library does not expose proxy configuration, you might be able to force it by configuring `http_proxy` and `https_proxy` environment variables through PHP's `putenv`. See [`InitialisationMiddleware`](https://github.com/silverstripe/cwp-core/blob/2/src/Control/InitialisationMiddleware.php) for a more specific code example. + +## Special case: curl from request handlers + +Plain `curl` requests will work out of the box, as long as you are making them from within request handlers (this might work for other libraries too). + +This is because the [`InitialisationMiddleware`](https://github.com/silverstripe/cwp-core/blob/2/src/Control/InitialisationMiddleware.php) sets the `http_proxy` and `https_proxy` environment variables automatically during the request execution. + +For requests made outside of request handling pipeline, you have to fall back to explicit library configuration. + +## Originating IP + +Outgoing requests originate at the proxy. This means they have an origin IP different from your stack's IP. + +If third party providers require whitelisting of IPs (for example on API requests performed through PHP on a CWP server), use CWP global proxy origin IPs: -If third party providers require whitelisting of IPs (for example on API requests performed through PHP on a CWP server), -please add the following IP addresses: * 202.55.102.136 * 202.55.96.178 +* 220.247.132.110 +* 103.253.48.27 +* 220.247.133.24 +* 103.253.48.56 -You can look up the implementation details in the `CwpInitialisationFilter` in the `cwp-core` module. - -## Known limitations +Proxy origin IPs are shared by all stacks on CWP, which may be important during your solution's security assessment. -Requests to internally resolved CWP hostnames will fail to connect when using the proxy. -Additionally requests over https (i.e. https://yourstack-uat.cwp.govt.nz) will fail to connect to the webserver via the CWP proxy and will fail without using the proxy too. We recommend adding a custom domain to the stack via the [CWP Service Desk](https://www.cwp.govt.nz/service-desk) and using that instead (i.e. yourstack-test.govt.nz). This will ensure that the request is resolved via external DNS. ## A note on `fopen()` and `file_get_contents()` @@ -32,53 +45,19 @@ Use of `fopen()` and `file_get_contents()` to retrieve remote content are discou Instead, CWP recommends using cURL commands, or ideally using a PHP library such as [Guzzle](https://github.com/guzzle/guzzle). -## Disabling egress proxy +## Disabling the proxy -In some situations you might want to disable the proxy for requests that remain within the CWP network. You can do so by customising cURL configuration per request: +For CWP customers, there is no reason to disable the outgoing CWP proxy. -```php -$ch = curl_init(); -curl_setopt($ch, CURLOPT_PROXY, false); -curl_setopt($ch, CURLOPT_PROXYPORT, false); -curl_setopt($ch, CURLOPT_URL, ''); -echo curl_exec($ch); -``` +Even if certain requests seem to work without the proxy, CWP is not able to guarantee continued operation of non-proxy connectivity. -You can also disable automatic proxy configuration globally by putting the following in one of your config files: - -```yaml ---- -Name: mysiteconfig -After: '#cwpcoreconfig' ---- -CWP\Core\Control\InitialisationMiddleware: - egress_proxy_default_enabled: false -``` - -## Excluding domains from egress proxy - -It's possible to exclude just some domains from being forced through the proxy if disabling it completely is not -desirable. By default this is used by the Solr and Docvert services internal to CWP. The configuration can be appended -to as follows: - -```yaml ---- -Name: mysiteconfig -After: '#cwpcoreconfig' ---- -CwpInitialisationFilter: - egress_proxy_exclude_domains: - - somewhere.cwp.govt.nz -``` - -
-Regardless of the excluded domains in your configuration, any external request not passed through the proxy will result in a 400-type error response code. -
+The only valid use case for proxy being disabled on stacks is for interfacing with platform internals. This is purposefully designed into the recipe components. You should never need to do it yourself. ## Troubleshooting external requests + If you're unable to make a connection to an external endpoint, verify that your request is being set with the `SS_OUTBOUND_PROXY` and `SS_OUTBOUND_PROXY_PORT` constants. You can use the [`curl_getinfo`](http://php.net/manual/en/function.curl-getinfo.php) function to see what configuration was used for the request. -Some third-party libraries like Guzzle may not automatically configure the proxy, meaning you'll need to ensure the environment variables are used when initialising the client: +Some third-party libraries like Guzzle may not automatically configure the proxy, meaning you'll need to ensure the appropriate constants are used when initialising the client: ```php use SilverStripe\Control\Director; @@ -95,6 +74,7 @@ $client = new GuzzleHttp\Client([ ``` If you're still not able to make external requests and have confirmed that your requests are using the CWP proxy settings, please raise a ticket via the [CWP Service Desk](https://www.cwp.govt.nz/service-desk) and include: + * the URL of the external service you're attempting to connect to * the relevant portion of code making the external request * steps to reproduce the issue, such as a URL or the name of a BuildTask