Skip to content

Latest commit

 

History

History
26 lines (14 loc) · 687 Bytes

File metadata and controls

26 lines (14 loc) · 687 Bytes

streams-processor-urls

Purpose

Module makes http calls to links and preserves content and metadata for downstream processing.

Prerequisites

  • String value array in activity.links

Optionally, module can convert messages to ActivityStreams format

LinkResolverProcessor.class processes activities like this:

LinkResolverProcessor.png

LinkCrawlerProcessor.class processes activities like this:

LinkCrawlerProcessor.png