feature #30602 [BrowserKit] Add support for HttpClient (fabpot, THERA…

…GE Kévin)

This PR was merged into the 4.3-dev branch.

Discussion
----------

[BrowserKit] Add support for HttpClient

| Q             | A
| ------------- | ---
| Branch?       | master
| Bug fix?      | no
| New feature?  | yes
| BC breaks?    | no
| Deprecations? | no <!-- don't forget to update UPGRADE-*.md and src/**/CHANGELOG.md files -->
| Tests pass?   | yes    <!-- please add some, will be required by reviewers -->
| Fixed tickets | part of #30502
| License       | MIT
| Doc PR        | not yet

When combining the power of the new HttpClient component with the BrowserKit and Mime components, we can makes something really powerful... a full/better/awesome replacement for https://github.com/FriendsOfPHP/Goutte.

So, this PR is about integrating the HttpClient component with BrowserKit to give users a high-level interface to ease usages in the most common use cases.

Scraping websites can be done like this:

```php
use Symfony\Component\BrowserKit\HttpBrowser;
use Symfony\Component\HttpClient\HttpClient;

$client = HttpClient::create();
$browser = new HttpBrowser($client);

$browser->request('GET', 'https://example.com/');
$browser->clickLink('Log In');
$browser->submitForm('Sign In', ['username' => 'me', 'password' => 'pass']);
$browser->clickLink('Subscriptions')->filter('table tr:nth-child(2) td:nth-child(2)')->each(function ($node) {
    echo trim($node->text())."\n";
});
```

And voilà! Nice, isn't?

Want to add HTTP cache? Sure:

```php
use Symfony\Component\HttpKernel\HttpCache\Store;

$client = HttpClient::create();
$store = new Store(sys_get_temp_dir().'/http-cache-store');

$browser = new HttpBrowser($client, $store);

// ...
```

Want logging and debugging of HTTP Cache? Yep:

```php
use Psr\Log\AbstractLogger;

class EchoLogger extends AbstractLogger
{
    public function log($level, $message, array $context = [])
    {
        echo $message."\n";
    }
}

$browser = new HttpBrowser($client, $store, new EchoLogger());
```

The first time you run your code, you will get an output similar to:

```
Request: GET https://twig.symfony.com/
Response: 200 https://twig.symfony.com/
Cache: GET /: miss, store
Request: GET https://twig.symfony.com/doc/2.x/
Response: 200 https://twig.symfony.com/doc/2.x/
Cache: GET /doc/2.x/: miss, store
```

But then:

```
Cache: GET /: fresh
Cache: GET /doc/2.x/: fresh
```

Limit is the sky here as you get the full power of all the Symfony ecosystem.

Under the hood, these examples leverage HttpFoundation, HttpKernel (with HttpCache),
DomCrawler, BrowserKit, CssSelector, HttpClient, Mime, ...

Excited?

P.S. : Tests need to wait for the HttpClient Mock class to land into master.

Commits
-------

b5b2a25 Add tests for HttpBrowser
dd55845 [BrowserKit] added support for HttpClient

src/Symfony/Component/BrowserKit/CHANGELOG.md

@@ -4,6 +4,7 @@ CHANGELOG
 4.3.0
 -----
 
+ * Added `HttpBrowser`, an implementation of a browser with the HttpClient component
  * Renamed `Client` to `AbstractBrowser`
  * Marked `Response` final.
  * Deprecated `Response::buildHeader()`

src/Symfony/Component/BrowserKit/HttpBrowser.php

@@ -0,0 +1,109 @@
+<?php
+
+/*
+ * This file is part of the Symfony package.
+ *
+ * (c) Fabien Potencier <fabien@symfony.com>
+ *
+ * For the full copyright and license information, please view the LICENSE
+ * file that was distributed with this source code.
+ */
+
+namespace Symfony\Component\BrowserKit;
+
+use Symfony\Component\HttpClient\HttpClient;
+use Symfony\Component\Mime\Part\AbstractPart;
+use Symfony\Component\Mime\Part\Multipart\FormDataPart;
+use Symfony\Component\Mime\Part\TextPart;
+use Symfony\Contracts\HttpClient\HttpClientInterface;
+
+/**
+ * An implementation of a browser using the HttpClient component
+ * to make real HTTP requests.
+ *
+ * @author Fabien Potencier <fabien@symfony.com>
+ *
+ * @final
+ */
+class HttpBrowser extends AbstractBrowser
+{
+    private $client;
+
+    public function __construct(HttpClientInterface $client = null, History $history = null, CookieJar $cookieJar = null)
+    {
+        if (!class_exists(HttpClient::class)) {
+            throw new \LogicException(sprintf('You cannot use "%s" as the HttpClient component is not installed. Try running "composer require symfony/http-client".', __CLASS__));
+        }
+
+        $this->client = $client ?? HttpClient::create();
+
+        parent::__construct([], $history, $cookieJar);
+    }
+
+    protected function doRequest($request)
+    {
+        $headers = $this->getHeaders($request);
+        $body = '';
+        if (null !== $part = $this->getBody($request)) {
+            $headers = array_merge($headers, $part->getPreparedHeaders()->toArray());
+            $body = $part->bodyToIterable();
+        }
+        $response = $this->client->request($request->getMethod(), $request->getUri(), [
+            'headers' => $headers,
+            'body' => $body,
+            'max_redirects' => 0,
+        ]);
+
+        return new Response($response->getContent(false), $response->getStatusCode(), $response->getHeaders(false));
+    }
+
+    private function getBody(Request $request): ?AbstractPart
+    {
+        if (\in_array($request->getMethod(), ['GET', 'HEAD'])) {
+            return null;
+        }
+
+        if (!class_exists(AbstractPart::class)) {
+            throw new \LogicException('You cannot pass non-empty bodies as the Mime component is not installed. Try running "composer require symfony/mime".');
+        }
+
+        if (null !== $content = $request->getContent()) {
+            return new TextPart($content, 'utf-8', 'plain', '8bit');
+        }
+
+        $fields = $request->getParameters();
+        foreach ($request->getFiles() as $name => $file) {
+            if (!isset($file['tmp_name'])) {
+                continue;
+            }
+
+            $fields[$name] = DataPart::fromPath($file['tmp_name'], $file['name']);
+        }
+
+        return new FormDataPart($fields);
+    }
+
+    private function getHeaders(Request $request): array
+    {
+        $headers = [];
+        foreach ($request->getServer() as $key => $value) {
+            $key = strtolower(str_replace('_', '-', $key));
+            $contentHeaders = ['content-length' => true, 'content-md5' => true, 'content-type' => true];
+            if (0 === strpos($key, 'http-')) {
+                $headers[substr($key, 5)] = $value;
+            } elseif (isset($contentHeaders[$key])) {
+                // CONTENT_* are not prefixed with HTTP_
+                $headers[$key] = $value;
+            }
+        }
+        $cookies = [];
+        foreach ($this->getCookieJar()->allRawValues($request->getUri()) as $name => $value) {
+            $cookies[] = $name.'='.$value;
+        }
+        if ($cookies) {
+            $headers['cookie'] = implode('; ', $cookies);
+        }
+
+        return $headers;
+    }
+}

src/Symfony/Component/BrowserKit/README.md

@@ -4,6 +4,9 @@ BrowserKit Component
 The BrowserKit component simulates the behavior of a web browser, allowing you
 to make requests, click on links and submit forms programmatically.
 
+The component comes with a concrete implementation that uses the HttpClient
+component to make real HTTP requests.
+
 Resources
 ---------