[fetcher] Deprecation and removal of ROME Fetcher

[PatrickGotthard]

Hi @ALL,

we decided to mark all ROME Fetcher classes as deprecated in version 1.6 and to drop it completely in version 2.0. ROME Fetcher was useful some years ago when internet connections were slow and expensive, because it supports ETags, GZip compression, RFC3229 delta encoding and caching.

Nowadays there are libraries like the well known Apache HttpClient. It supports ETags and GZip compression out-of-the-box, delta encoding seems to be used very rarely and there are also much better caching solutions (like EhCache) compared to our simple file and in-memory based implementations.

In addition, ROME fetcher is not flexible enough for many use-cases because the user has no full control over the underlying HTTP connection and sometimes even new ROME functionalities cannot be used without updating ROME Fetcher too.

Last but not least, the most comprehensive ROME Fetcher implementation (HttpClientFeedFetcher) uses Apache HttpClient in version 3 that has some security vulnerabilities. We are not able to update it to version 4 without breaking backwards compatiblity and simply removing the implementation in version 1.6 was not an option for us.

We hope you support our decision. It will help us to become faster and to concentrate on the core features.

Please give us some feedback, especially when you don't support our decision!

Regards,
Patrick

[mishako]

Here is a migration example. This is how you used to retrieve a feed with rome-fetcher:

URL url = new URL("http://stackoverflow.com/feeds/tag?tagnames=rome");
FeedFetcher feedFetcher = new HttpClientFeedFetcher();
SyndFeed feed = feedFetcher.retrieveFeed(url);
System.out.println(feed.getTitle());

This is how you do the same thing with Apache HttpClient 4:

<dependency>
  <groupId>org.apache.httpcomponents</groupId>
  <artifactId>httpclient</artifactId>
  <version>4.5.2</version>
</dependency>

String url = "http://stackoverflow.com/feeds/tag?tagnames=rome";
try (CloseableHttpClient client = HttpClients.createMinimal()) {
  HttpUriRequest request = new HttpGet(url);
  try (CloseableHttpResponse response = client.execute(request);
       InputStream stream = response.getEntity().getContent()) {
    SyndFeedInput input = new SyndFeedInput();
    SyndFeed feed = input.build(new XmlReader(stream));
    System.out.println(feed.getTitle());
  }
}

It's a bit more code, but on the upside, you get to configure every detail of the request.

Edit 1: try-with-resources for http client
Edit 2: renamed method to request

[idalv]

Here is how you could pull a feed with Spring's RestTemple. Hope it helps someone:

        RestTemplate restTemplate = new RestTemplate();
        SyndFeed syndFeed = restTemplate.execute(feedUrl, HttpMethod.GET, null, response -> {
            SyndFeedInput input = new SyndFeedInput();
            try {
                return input.build(new XmlReader(response.getBody()));
            } catch (FeedException e) {
                throw new IOException("Could not parse response", e);
            }
        });

[dimi-nk]

In case this helps anyone, an example using Java's HttpClient:

private static final String FEED_URL = "https://stackoverflow.com/feeds/tag?tagnames=rome";

private static SyndFeed bodyToFeed(InputStream body) {
  try {
    return new SyndFeedInput().build(new XmlReader(body));
  } catch (FeedException | IOException e) {
    throw new CompletionException(e);
  }
}

private static List<String> getEntryTitles(SyndFeed feed) {
  return feed.getEntries().stream().map(SyndEntry::getTitle).collect(toList());
}

public CompletableFuture<List<String>> getTitles() {
  HttpClient client = HttpClient.newHttpClient();
  HttpRequest request = HttpRequest.newBuilder().uri(URI.create(FEED_URL)).build();
  return client
      .sendAsync(request, BodyHandlers.ofInputStream())
      .thenApply(HttpResponse::body)
      .thenApply(RssReader::bodyToFeed)
      .thenApply(RssReader::getEntryTitles)
      .orTimeout(2, SECONDS)
      .exceptionally(
          t -> {
            log.error(format("Could not fetch feed from '%s'", FEED_URL), t);
            return emptyList();
          });
}