Skip to content

DenBeke/Aggregator

Repository files navigation

Aggregator

This script is a simple aggregator for fetching web content of different websites. Websites that need to be parsed, are added in the config/schema.json, with their corresponding schema.

The aggregator will automatically fetch all the pages of each feed, and save them in a JSON file stored in the output directory.

Configuration of the aggregator can be done in the config/config.json file.

Building

Install dependencies

$ go get github.com/PuerkitoBio/goquery
$ go get github.com/kennygrant/sanitize

Build & Run Aggregator

$ go build
$ ./Aggregator

Acknowledgements

Author

Mathias Beke

About

Simple html parser for aggregating web content

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages