Finds threads by looking for patterns in subjects in a mailman archive and makes a summary
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
tools
.gitignore
.travis.yml
LICENSE
README.md
crawler.go
mailman-summarizer-7cd37e4f2137.json.enc
makerss.go
oldreader_output.png

README.md

Build Status

mailman-summarizer

General Idea:

Usage:

installing go:

 export GOPATH=$HOME/go

dependencies:

 go get -u github.com/PuerkitoBio/goquery
 go get -u github.com/gocolly/colly/...
 go get -u github.com/gorilla/feeds/...

building:

 go build

running:

 ./mailman-summarizer -h

Because we use case sensitive operations when doing string comparisons or sorting lists you have to manually:

 ./mailman-summarizer -topic GW,S3,keystone,civet,gw,s3

Example Output if subscribing to the feed in https://theoldreader.com

Oldreader

Deployment

This repo uses travis and a cronjob to run the script and deploy the XML to google cloud object store

  • needs a google cloud project, a service account, a key in json, a key encrypted with travis
  • then needs the gsutil in the travis build env

The URL to the produced feed is: https://storage.googleapis.com/ceph-rgw-users/feed.xml

Sources

Started as a fork of colly-example.

https://github.com/PuerkitoBio/goquery https://github.com/gocolly/colly

TODOs

Scattered all over the source code.

Also: wouldn't it be fun with a favicon in the feed? =) =)

https://snook.ca/archives/rss/add_logo_to_feed