647fb087ebed1be6115e6c8cc700af083f88606a John Leach john@johnleach.co.uk http://github.com/johnl/web-page-parser/commit/cabaf34d7a57958f292c64261edc1cebdae9d346 cabaf34d7a57958f292c64261edc1cebdae9d346 2009-06-20T14:41:36-07:00 2009-06-20T14:40:51-07:00 gem release 0.10 cf00434a80289d9dcc1167f7ece50e6b1f4c12cf John Leach john@johnleach.co.uk defbe5f7fd3359f747ec5d5f6d6a3ea3769cb674 John Leach john@johnleach.co.uk http://github.com/johnl/web-page-parser/commit/647fb087ebed1be6115e6c8cc700af083f88606a 647fb087ebed1be6115e6c8cc700af083f88606a 2009-06-20T14:40:11-07:00 2009-06-20T14:40:11-07:00 Add missing fixture 7745137.stm.html. Fix CONTENT Regexp and ICONV settinf for BbcNewsPageParserV1 366b353b309b0a9c12cd0ff4851f70aee81d829f John Leach john@johnleach.co.uk cf8d404e8d9fb3387dea8e9994658412cbe5e7fd John Leach john@johnleach.co.uk http://github.com/johnl/web-page-parser/commit/defbe5f7fd3359f747ec5d5f6d6a3ea3769cb674 defbe5f7fd3359f747ec5d5f6d6a3ea3769cb674 2009-05-25T05:36:43-07:00 2009-05-25T05:36:43-07:00 spec: BaseParser should not calculates a hash using the date b0d8b1a085ae1fb61f1161fdbbb5fae5bc84631d John Leach john@johnleach.co.uk bc55bf283b47747e4a331bbb67559f211a64cbe1 John Leach john@johnleach.co.uk http://github.com/johnl/web-page-parser/commit/cf8d404e8d9fb3387dea8e9994658412cbe5e7fd cf8d404e8d9fb3387dea8e9994658412cbe5e7fd 2009-05-25T04:58:53-07:00 2009-05-25T04:58:53-07:00 BbcNewsPageParser cannot parse popup pages 2f7006438ba1a3e43e9795ec2caf83f810c24431 John Leach john@johnleach.co.uk db316e6836c0b66fd2041abc47101c2578d1c8ff John Leach john@johnleach.co.uk http://github.com/johnl/web-page-parser/commit/bc55bf283b47747e4a331bbb67559f211a64cbe1 bc55bf283b47747e4a331bbb67559f211a64cbe1 2009-05-25T03:04:36-07:00 2009-05-25T03:04:36-07:00 spec: BbcNewsPageParserV2 should parse 'magazine' pages 7916ac6a2040065cf0772e832a9c39040a341483 John Leach john@johnleach.co.uk ac3d06e4017acd856af60b426644d36d0e56f722 John Leach john@johnleach.co.uk http://github.com/johnl/web-page-parser/commit/db316e6836c0b66fd2041abc47101c2578d1c8ff db316e6836c0b66fd2041abc47101c2578d1c8ff 2009-05-19T16:54:43-07:00 2009-05-19T16:54:43-07:00 BbcNewsPage: Add a couple more invalid url tests ff52356ed16d59b1206d2c70b568e9dd7e1cbf9c John Leach john@johnleach.co.uk 4a51900c43cb0bbf4b6c041a5b19be0e33d7a0fd John Leach john@johnleach.co.uk http://github.com/johnl/web-page-parser/commit/ac3d06e4017acd856af60b426644d36d0e56f722 ac3d06e4017acd856af60b426644d36d0e56f722 2009-05-09T17:28:32-07:00 2009-05-09T17:28:32-07:00 Big refactor of BaseParser to make sub-classing easier 9e71181eafd3c7b61ae130c0a6584187f0c957e1 John Leach john@johnleach.co.uk 62a3b578c4d54bc00ee5309d08ff8a739a0d8432 John Leach john@johnleach.co.uk http://github.com/johnl/web-page-parser/commit/4a51900c43cb0bbf4b6c041a5b19be0e33d7a0fd 4a51900c43cb0bbf4b6c041a5b19be0e33d7a0fd 2009-05-09T14:25:01-07:00 2009-05-09T14:25:01-07:00 Added LICENSE, README.rdoc and gemspec 2b74a0f3aa18b76977a428f047620e0796524318 John Leach john@johnleach.co.uk 9a0fc319d719e3ff9b2c0ae322061b84c42cfcef John Leach john@johnleach.co.uk http://github.com/johnl/web-page-parser/commit/62a3b578c4d54bc00ee5309d08ff8a739a0d8432 62a3b578c4d54bc00ee5309d08ff8a739a0d8432 2009-05-09T12:49:50-07:00 2009-05-09T12:47:18-07:00 spec: BbcNewsPageParserV2 should convert iso-8859-1 in the title to utf8 spec: BbcNewsPageParserV2 should convert iso-8859-1 in the content to utf8 7129836359628fede438b73f03dc4f612d60b2cf John Leach john@johnleach.co.uk 9e871e16c74a8b67b68b0d7bd110e9504c5204de John Leach john@johnleach.co.uk http://github.com/johnl/web-page-parser/commit/9a0fc319d719e3ff9b2c0ae322061b84c42cfcef 9a0fc319d719e3ff9b2c0ae322061b84c42cfcef 2009-05-09T06:07:09-07:00 2009-05-09T06:07:09-07:00 spec: AllPageParsers should return an empty array when there is no content available 3bc43d0515c0d3e1add4cda1e60707b1a5c33b3c John Leach john@johnleach.co.uk c2f003b07a8745eb45dd14bfd5b964ceec5d7c29 John Leach john@johnleach.co.uk http://github.com/johnl/web-page-parser/commit/9e871e16c74a8b67b68b0d7bd110e9504c5204de 9e871e16c74a8b67b68b0d7bd110e9504c5204de 2009-05-09T05:39:00-07:00 2009-05-09T05:39:00-07:00 bbc news parser should ignore newsforum pages a89a49de5e51774709045cc97e261d05c73fcb06 John Leach john@johnleach.co.uk 21ec04612aeffaf614a9323d633328936d430801 John Leach john@johnleach.co.uk http://github.com/johnl/web-page-parser/commit/c2f003b07a8745eb45dd14bfd5b964ceec5d7c29 c2f003b07a8745eb45dd14bfd5b964ceec5d7c29 2009-05-09T05:36:24-07:00 2009-05-09T05:24:44-07:00 bbc news: fix parsing of 'from our own correspondent' pages spec: BbcNewsPageParserFactory should ignore 'in pictures' articlesChange url regexps to Oniguruma. e83597736f8cac6809dfe0a6d8e460b8d950ae8e John Leach john@johnleach.co.uk 75580b1090b5a35ed66801cd0012efffa1d64074 John Leach john@johnleach.co.uk http://github.com/johnl/web-page-parser/commit/21ec04612aeffaf614a9323d633328936d430801 21ec04612aeffaf614a9323d633328936d430801 2009-04-26T11:13:33-07:00 2009-04-26T11:06:53-07:00 spec: BaseParser should decode basic html entities 65be8024da608c31905c42148813e6736667b420 John Leach john@johnleach.co.uk a9aac85cccaf86758c6c55e570f91d3358527f06 John Leach john@johnleach.co.uk http://github.com/johnl/web-page-parser/commit/75580b1090b5a35ed66801cd0012efffa1d64074 75580b1090b5a35ed66801cd0012efffa1d64074 2009-04-26T11:02:56-07:00 2009-04-26T11:02:56-07:00 spec: BbcNewsPageParserV2 should decode html entities use htmlentities library to decode html entities da9803a5b788030cf1dbeb163516e42ffdac001a John Leach john@johnleach.co.uk ecd1ab582004c497525fcb4e23d2e482dce1255d John Leach john@johnleach.co.uk http://github.com/johnl/web-page-parser/commit/a9aac85cccaf86758c6c55e570f91d3358527f06 a9aac85cccaf86758c6c55e570f91d3358527f06 2009-04-26T10:15:03-07:00 2009-04-26T10:15:03-07:00 spec: BbcNewsPageParserV2 should should parse the content spec: BbcNewsPageParserV2 should calculate a valid hash of the content 24f81268ff3cf8495a23a9366c006241096d1764 John Leach john@johnleach.co.uk 7e2657941e22d6ffd29faae17b40af74463670c3 John Leach john@johnleach.co.uk http://github.com/johnl/web-page-parser/commit/ecd1ab582004c497525fcb4e23d2e482dce1255d ecd1ab582004c497525fcb4e23d2e482dce1255d 2009-04-23T16:13:48-07:00 2009-04-23T16:13:48-07:00 spec: BbcNewsPageParserV2 should parse the date in UTC 079f92fc1f90ffc5c6066040ab86e41704f37463 John Leach john@johnleach.co.uk 5c358c826c8a704da6013c2b17c3ce02ed811f11 John Leach john@johnleach.co.uk http://github.com/johnl/web-page-parser/commit/7e2657941e22d6ffd29faae17b40af74463670c3 7e2657941e22d6ffd29faae17b40af74463670c3 2009-04-21T16:53:21-07:00 2009-04-21T16:53:21-07:00 Removed now unused parse! method spec: BbcNewsPageParserV1 should convert apostrophe and pound sign html entities in content spec: BbcNewsPageParserV1 should convert apostrophe and pound sign html entities in page titles d891a99430c3f2087533f82030dc934fd73fad0e John Leach john@johnleach.co.uk ef155a8f41d8ae378e410efc21371b8ea5fc12ed John Leach john@johnleach.co.uk http://github.com/johnl/web-page-parser/commit/5c358c826c8a704da6013c2b17c3ce02ed811f11 5c358c826c8a704da6013c2b17c3ce02ed811f11 2009-04-21T16:35:34-07:00 2009-04-21T16:35:34-07:00 Move parsing from parse! into attribute reader methods to allow lazy parsing, and easy inheritable parsing 992b05a6c7c697b4c5b4e49cbcb961917ee5573c John Leach john@johnleach.co.uk 19df56a4d814f8ecf1efc62727111a1879ed03ba John Leach john@johnleach.co.uk http://github.com/johnl/web-page-parser/commit/ef155a8f41d8ae378e410efc21371b8ea5fc12ed ef155a8f41d8ae378e410efc21371b8ea5fc12ed 2009-04-21T16:12:30-07:00 2009-04-21T16:12:22-07:00 spec: BbcNewsPageParserV2 should parse the title b09068f99eddf468480cfe3fb9c4f881eed343ff John Leach john@johnleach.co.uk d3f6164b046ae7be9b2b92bb2f00e601a106c918 John Leach john@johnleach.co.uk http://github.com/johnl/web-page-parser/commit/19df56a4d814f8ecf1efc62727111a1879ed03ba 19df56a4d814f8ecf1efc62727111a1879ed03ba 2009-04-21T16:01:23-07:00 2009-04-21T15:38:31-07:00 NewsPageParser: Make regexps constants 21e56e7464d3b316c42352550fbd40b6ce566984 John Leach john@johnleach.co.uk 09c353b48164f0223c73eaf0730d03b3a62d4d58 John Leach john@johnleach.co.uk http://github.com/johnl/web-page-parser/commit/d3f6164b046ae7be9b2b92bb2f00e601a106c918 d3f6164b046ae7be9b2b92bb2f00e601a106c918 2009-04-14T17:24:16-07:00 2009-04-14T17:24:16-07:00 Move BbcNewsPageParserV1 to Oniguruma regexp library ac3d7d057aed1e077a0462cac21d8201b8169d43 John Leach john@johnleach.co.uk a8f72fd4b7a3a22a0aa3508ffbf516749025916d John Leach john@johnleach.co.uk http://github.com/johnl/web-page-parser/commit/09c353b48164f0223c73eaf0730d03b3a62d4d58 09c353b48164f0223c73eaf0730d03b3a62d4d58 2009-04-14T16:45:21-07:00 2009-04-14T16:45:21-07:00 WebPageParser::BbcNewsPageParserV1 - should parse the title - should parse the date in UTC - should parse the content exactly like the old News Sniffer library 71ed061773b34d0a8b6aad62453b668dde76ade5 John Leach john@johnleach.co.uk 19a084b000dd25ea7afcafd11973b7952fd12937 John Leach john@johnleach.co.uk http://github.com/johnl/web-page-parser/commit/a8f72fd4b7a3a22a0aa3508ffbf516749025916d a8f72fd4b7a3a22a0aa3508ffbf516749025916d 2009-04-13T16:30:39-07:00 2009-04-13T16:30:39-07:00 Initial BbcNewsPageParser and factory spec: BbcNewsPageFactory should detect bbc news articles from the url spec: BbcNewsPageFactory should ignore pages with the wrong url format 512ec3407ac4dd38c1d51e5c87d4f26486df3782 John Leach john@johnleach.co.uk 6313b9c6afdce5b8bf563b04edcd4c851a1d88f3 John Leach john@johnleach.co.uk http://github.com/johnl/web-page-parser/commit/19a084b000dd25ea7afcafd11973b7952fd12937 19a084b000dd25ea7afcafd11973b7952fd12937 2009-04-13T15:59:43-07:00 2009-04-13T15:59:43-07:00 Change methods to use options hash Some documentation 8b122a4aa2ea4a8222efa402ecdd14a4026eb60f John Leach john@johnleach.co.uk bb643437f5cc227381435a4134b0d888cf80d7ec John Leach john@johnleach.co.uk http://github.com/johnl/web-page-parser/commit/6313b9c6afdce5b8bf563b04edcd4c851a1d88f3 6313b9c6afdce5b8bf563b04edcd4c851a1d88f3 2009-04-13T15:51:50-07:00 2009-04-13T15:51:50-07:00 spec: PageParserFactory should return nil if no PageParser can be found for the given url 6d6adca299abef98f4af948f3b2ace4702443cb8 John Leach john@johnleach.co.uk d9f7238dfa9c5141821f2ad4899ba697506c828e John Leach john@johnleach.co.uk http://github.com/johnl/web-page-parser/commit/bb643437f5cc227381435a4134b0d888cf80d7ec bb643437f5cc227381435a4134b0d888cf80d7ec 2009-04-13T15:19:19-07:00 2009-04-13T15:19:07-07:00 TestPageParser and TestPageParserFactory spec: ParserFactory should load parsers in the parsers directory spec: ParserFactory should provide the right PageParser for the given url ffac4cb81f7b6dc65386e511d7c61d7401daa7ac John Leach john@johnleach.co.uk 4c603aec6e50aec1d700de8c125ff10b99fc3d8b John Leach john@johnleach.co.uk http://github.com/johnl/web-page-parser/commit/d9f7238dfa9c5141821f2ad4899ba697506c828e d9f7238dfa9c5141821f2ad4899ba697506c828e 2009-04-13T07:24:36-07:00 2009-04-13T07:24:07-07:00 Renamed BaseParserFactory to ParserFactory spec: BaseParser should know about its subclasses e3091e5a99d40a0c31d065fb655c25332a5f4c5e John Leach john@johnleach.co.uk ce50fee85c0d5e7dc878fe3ce16b697ad29adbd2 John Leach john@johnleach.co.uk http://github.com/johnl/web-page-parser/commit/4c603aec6e50aec1d700de8c125ff10b99fc3d8b 4c603aec6e50aec1d700de8c125ff10b99fc3d8b 2009-04-13T07:12:21-07:00 2009-04-13T07:12:21-07:00 Renamed base.rb to base_parser.rb ea9e52517ed97cd1d041b08ff084451cc2a67123 John Leach john@johnleach.co.uk db16152a88541a1c5b2a4fbfcb2aa950be07e1ad John Leach john@johnleach.co.uk http://github.com/johnl/web-page-parser/commit/ce50fee85c0d5e7dc878fe3ce16b697ad29adbd2 ce50fee85c0d5e7dc878fe3ce16b697ad29adbd2 2009-04-13T07:11:26-07:00 2009-04-13T07:11:26-07:00 Move BaseParserFactory into base_parser_factory.rb ac076446c2c0f3974b998b471c6fae3b4df42ea7 John Leach john@johnleach.co.uk fb2d569af511006c320e4d514e75193b0dcf7033 John Leach john@johnleach.co.uk http://github.com/johnl/web-page-parser/commit/db16152a88541a1c5b2a4fbfcb2aa950be07e1ad db16152a88541a1c5b2a4fbfcb2aa950be07e1ad 2009-04-13T07:06:13-07:00 2009-04-13T07:03:21-07:00 Add spec Rake task 9afa588440c7552cac50bef3c0e6af6a4316d013 John Leach john@johnleach.co.uk