69598f64a5e560968d0629e767f0fe82a55ccbad Martin Matusiak numerodix@gmail.com http://github.com/numerodix/spiderfetch/commit/c79b4ea4ab4a5209b83fa5e1a72b888635b6e3fb c79b4ea4ab4a5209b83fa5e1a72b888635b6e3fb 2008-04-24T10:38:42-07:00 2008-04-24T10:38:42-07:00 add option to match on "src" attributes in tags 69d03678687756105c1483f55e012489e3d4d47d Martin Matusiak numerodix@gmail.com 8a57b78f5987166a6c35b3277bbf09d2b26d288a Martin Matusiak numerodix@gmail.com http://github.com/numerodix/spiderfetch/commit/69598f64a5e560968d0629e767f0fe82a55ccbad 69598f64a5e560968d0629e767f0fe82a55ccbad 2008-04-24T09:22:18-07:00 2008-04-24T09:22:18-07:00 add cert nocheck by default b3268fcb446475909f0e99f38b22a1c98748b27f Martin Matusiak numerodix@gmail.com 878aaf2aa23e98dfee61850337f1ca8b05f93102 Martin Matusiak numerodix@gmail.com http://github.com/numerodix/spiderfetch/commit/8a57b78f5987166a6c35b3277bbf09d2b26d288a 8a57b78f5987166a6c35b3277bbf09d2b26d288a 2008-04-24T09:05:25-07:00 2008-04-24T09:05:25-07:00 unset user agent string for wget 10cf4ec5e2bf8d944047d1d08b63fd85790de908 Martin Matusiak numerodix@gmail.com d4f7f88c26d10236f5ecf879f0b11fd54854befc Martin Matusiak numerodix@gmail.com http://github.com/numerodix/spiderfetch/commit/878aaf2aa23e98dfee61850337f1ca8b05f93102 878aaf2aa23e98dfee61850337f1ca8b05f93102 2008-04-24T07:30:01-07:00 2008-04-24T07:30:01-07:00 add some test cases f71922b5ed7074b83505e7d4ece3532b656c4c0f Martin Matusiak numerodix@gmail.com d41c4843d57263e94b9b351e9669da1e0e583b37 Martin Matusiak numerodix@gmail.com http://github.com/numerodix/spiderfetch/commit/d4f7f88c26d10236f5ecf879f0b11fd54854befc d4f7f88c26d10236f5ecf879f0b11fd54854befc 2008-04-24T07:24:25-07:00 2008-04-24T07:24:25-07:00 quote vars in bash script 9ea467dcfd7db72641c86e484e384c91672bcc40 Martin Matusiak numerodix@gmail.com 748fd984289f8900791051a0b32ad76f724e8e5b Martin Matusiak numerodix@gmail.com http://github.com/numerodix/spiderfetch/commit/d41c4843d57263e94b9b351e9669da1e0e583b37 d41c4843d57263e94b9b351e9669da1e0e583b37 2008-04-23T16:38:37-07:00 2008-04-23T16:38:37-07:00 ungeneralize exception catch 8ec334a6f99be6f2f68a03557560c3d4bee3b119 Martin Matusiak numerodix@gmail.com 46893f8f649af6d5f281aa1ab246ff04dce86947 Martin Matusiak numerodix@gmail.com http://github.com/numerodix/spiderfetch/commit/748fd984289f8900791051a0b32ad76f724e8e5b 748fd984289f8900791051a0b32ad76f724e8e5b 2008-04-23T15:03:08-07:00 2008-04-23T15:03:08-07:00 match backticks for quoting a60286b55eeb6d15c5fe67cab0cfffd82b19a61a Martin Matusiak numerodix@gmail.com 99d6eb3bf16c33142966ba54fa9733825a5e7903 Martin Matusiak numerodix@gmail.com http://github.com/numerodix/spiderfetch/commit/46893f8f649af6d5f281aa1ab246ff04dce86947 46893f8f649af6d5f281aa1ab246ff04dce86947 2008-04-23T10:54:10-07:00 2008-04-23T10:54:10-07:00 automate asx=>mms downloads 473ee9ec1b705291cc7ba6ec55f8ad5828d274d4 Martin Matusiak numerodix@gmail.com 46183e42429fe7793be90858c8bf5330f9a467d7 Martin Matusiak numerodix@gmail.com http://github.com/numerodix/spiderfetch/commit/99d6eb3bf16c33142966ba54fa9733825a5e7903 99d6eb3bf16c33142966ba54fa9733825a5e7903 2008-04-23T09:58:54-07:00 2008-04-23T09:58:54-07:00 regex: don't match on specific tag names catch parse error from URI::join 2d5da3f1e03aa9fb59c9b08aef5effc0bf196637 Martin Matusiak numerodix@gmail.com 863fca42ab8abc0804efd35ad7d304a832c882af Martin Matusiak numerodix@gmail.com http://github.com/numerodix/spiderfetch/commit/46183e42429fe7793be90858c8bf5330f9a467d7 46183e42429fe7793be90858c8bf5330f9a467d7 2008-04-23T09:42:10-07:00 2008-04-23T09:42:10-07:00 remove redundant import regex: match also in .asx files 76b7dd8c56da1b387ba8509537bd489b4d40d374 Martin Matusiak numerodix@gmail.com c6bbfdbfe7416461a0d21c69f2d8d27e8e3c1598 Martin Matusiak numerodix@gmail.com http://github.com/numerodix/spiderfetch/commit/863fca42ab8abc0804efd35ad7d304a832c882af 863fca42ab8abc0804efd35ad7d304a832c882af 2008-04-22T09:28:24-07:00 2008-04-22T09:28:24-07:00 remove naive optimization d5ebfb526a09ac65c3f56ca28db5a09d4eb4b471 Martin Matusiak numerodix@gmail.com 89c796738d19e9a289e1ba6c94e3df305c9111ec Martin Matusiak numerodix@gmail.com http://github.com/numerodix/spiderfetch/commit/c6bbfdbfe7416461a0d21c69f2d8d27e8e3c1598 c6bbfdbfe7416461a0d21c69f2d8d27e8e3c1598 2008-04-22T09:26:58-07:00 2008-04-22T09:26:58-07:00 generalize schema in protocol matching d4061fed88bb8f0fed572364a51a277850a635f3 Martin Matusiak numerodix@gmail.com 343da5e2ac795adfd270e84e711a068dc14953a0 Martin Matusiak numerodix@gmail.com http://github.com/numerodix/spiderfetch/commit/89c796738d19e9a289e1ba6c94e3df305c9111ec 89c796738d19e9a289e1ba6c94e3df305c9111ec 2008-04-22T09:22:19-07:00 2008-04-22T09:22:19-07:00 add copy banner 93045ad28407efa94904fa2befe8b59c50e79be7 Martin Matusiak numerodix@gmail.com 66dc920aaa06807a12c752d806bef6f5e8500337 Martin Matusiak numerodix@gmail.com http://github.com/numerodix/spiderfetch/commit/343da5e2ac795adfd270e84e711a068dc14953a0 343da5e2ac795adfd270e84e711a068dc14953a0 2008-04-22T09:14:52-07:00 2008-04-22T09:14:52-07:00 minor fixes 162930c01c7e41f6e0b744fe6d2360365841d6e9 Martin Matusiak numerodix@gmail.com 7eda335074d06e6b14a715489a552ee01a164ca4 Martin Matusiak numerodix@gmail.com http://github.com/numerodix/spiderfetch/commit/66dc920aaa06807a12c752d806bef6f5e8500337 66dc920aaa06807a12c752d806bef6f5e8500337 2008-04-22T09:08:28-07:00 2008-04-22T09:08:28-07:00 strip off filename from the end of the index url before concating it with item urls 5d9c5b4c44becfc6d6ea3c6c9e50d6fa806ff35c Martin Matusiak numerodix@gmail.com 7cd8b91652336ac14857c384402462b24dbd548a Martin Matusiak numerodix@gmail.com http://github.com/numerodix/spiderfetch/commit/7eda335074d06e6b14a715489a552ee01a164ca4 7eda335074d06e6b14a715489a552ee01a164ca4 2008-04-22T08:56:32-07:00 2008-04-22T08:56:32-07:00 make regex matching more robust (accept both single and double quotes around url) add option to use existing index file add option to dump index content to screen 4e634aca5b04b4a3d0cb29c3abeabefcb95a2421 Martin Matusiak numerodix@gmail.com 815663fcc14b0b0b9435a9d41b0465948950041e Martin Matusiak numerodix@gmail.com http://github.com/numerodix/spiderfetch/commit/7cd8b91652336ac14857c384402462b24dbd548a 7cd8b91652336ac14857c384402462b24dbd548a 2008-04-22T08:18:42-07:00 2008-04-22T08:18:42-07:00 fix ctrl+c behavior, now working reliably dbf6952b492639ef6d1c7d9d370deaa818035833 Martin Matusiak numerodix@gmail.com a7fc152addcdfbc8ccd80e22c60faa8b59f4bd5f Martin Matusiak numerodix@gmail.com http://github.com/numerodix/spiderfetch/commit/815663fcc14b0b0b9435a9d41b0465948950041e 815663fcc14b0b0b9435a9d41b0465948950041e 2008-04-22T06:34:53-07:00 2008-04-22T06:34:53-07:00 add trap, still broken 63c8e1ae9c0e3a2d33ca233a1d1ca134150126e2 Martin Matusiak numerodix@gmail.com 13d51774be93d2e62aedea6c533c30f2cdfcbe4b Martin Matusiak numerodix@gmail.com http://github.com/numerodix/spiderfetch/commit/a7fc152addcdfbc8ccd80e22c60faa8b59f4bd5f a7fc152addcdfbc8ccd80e22c60faa8b59f4bd5f 2008-04-21T21:54:51-07:00 2008-04-21T21:54:51-07:00 ctrl+c now kills wget (eventually) when held in, minor progress 3b9b55ccd99217d0370516f0eac689ceda81adca Martin Matusiak numerodix@gmail.com 3f9d64c2857ebbede3ee540a721e80777c5f6cb3 Martin Matusiak numerodix@gmail.com http://github.com/numerodix/spiderfetch/commit/13d51774be93d2e62aedea6c533c30f2cdfcbe4b 13d51774be93d2e62aedea6c533c30f2cdfcbe4b 2008-04-21T21:03:07-07:00 2008-04-21T21:03:07-07:00 kill accidental add ee1301e71a8411b363c0e1625b92901ebe5d325c Martin Matusiak numerodix@gmail.com Martin Matusiak numerodix@gmail.com http://github.com/numerodix/spiderfetch/commit/3f9d64c2857ebbede3ee540a721e80777c5f6cb3 3f9d64c2857ebbede3ee540a721e80777c5f6cb3 2008-04-21T21:02:38-07:00 2008-04-21T21:02:38-07:00 first working version nasty flaw: keyboardinterrupt does nothing to kill wget because it's being run in-thread b67311b3a738a7322785b6da91c6cd30c0368ba1 Martin Matusiak numerodix@gmail.com