Lagotto 3.12.1
Lagotto 3.12.1 was released on January 4, 2015 with the following changes:
- log responses from external APIs to JSON file in logstash format (#214)
- better handling of ActiveJob errors (#215)
- run daily works import at the end of the day using a new
cron:nightly
rake task, and write to separatecron_import.log
(#216) - show all URLs associated with a work (and don't show the citeulike URL if there are no events) (#217)
Starting with this release, the raw responses from external sources are stored in a JSON file log/agent.log
in logstash format, using the source name and work pid as tags:
{
"message": "{\"total_rows\"=>328331, \"offset\"=>121903, \"rows\"=>[{\"id\"=>\"232944431433666561\", \"key\"=>\"10.1371/journal.pone.0042231\", \"value\"=>{\"_id\"=>\"232944431433666561\", \"_rev\"=>\"1-40047ae514c178f154fa7b0f877f146a\", \"text\"=>\"Role of the Irr Protein in the Regulation of Iron Metabolism in Rhodobacter sphaeroides http://t.co/JbOupbjK\", \"from_user_id\"=>38951828, \"from_user_name\"=>\"Test\", \"geo\"=>nil, \"profile_image_url_https\"=>\"https://si0.twimg.com/sticky/default_profile_images/default_profile_6_normal.png\", \"iso_language_code\"=>\"en\", \"to_user_name\"=>nil, \"entities\"=>{\"urls\"=>[{\"expanded_url\"=>\"http://bit.ly/O0DVOi\", \"indices\"=>[88, 108], \"display_url\"=>\"bit.ly/O0DVOi\", \"url\"=>\"http://t.co/JbOupbjK\"}, {\"expanded_url\"=>\"http://www.plosone.org/article/info:doi%2F10.1371%2Fjournal.pone.0042231\", \"display_url\"=>\"http://www.plosone.org/article/info:doi%2F10.1371%2Fjournal.pone.0042231\", \"url\"=>\"http://www.plosone.org/article/info:doi%2F10.1371%2Fjournal.pone.0042231\"}], \"hashtags\"=>[], \"user_mentions\"=>[]}, \"to_user_id\"=>0, \"id\"=>232944431433666560, \"to_user_id_str\"=>\"0\", \"source\"=>\"<a href="http://twitterfeed.com" rel="nofollow">twitterfeed</a>\", \"from_user_id_str\"=>\"38951828\", \"from_user\"=>\"TestCellBio\", \"created_at\"=>\"Tue, 07 Aug 2012 21:00:55 +0000\", \"to_user\"=>nil, \"id_str\"=>\"232944431433666561\", \"profile_image_url\"=>\"http://a0.twimg.com/sticky/default_profile_images/default_profile_6_normal.png\", \"metadata\"=>{\"result_type\"=>\"recent\"}}}]}",
"@timestamp": "2015-01-04T01:13:53.387-08:00",
"@version": "1",
"severity": "INFO",
"host": "rwc-prod-alm03",
"tags": [
"ActiveJob",
"SourceJob",
"2822afb7-1a37-4084-b3c2-b0b87f0370a0",
"twitter",
"10.1371/journal.pone.0042231"
]
}
This file can be further processed with logstash and made available for download.