54eaa9ffeed3b3bd2690acb69d1fbcaa92646f52 Ville Tuulos tuulos@dxfront.(none) http://github.com/tuulos/disco/commit/4ebdbf4ab738dfdeba8e0a1245aed709f0eec227 4ebdbf4ab738dfdeba8e0a1245aed709f0eec227 2009-10-04T22:10:52-07:00 2009-10-04T22:10:52-07:00 Fixed black/whitelisting broken by the new scheduler d1aaf28bd7550dea44f7b12360dec2893a292fb0 Ville Tuulos tuulos@dxfront.(none) 1013a7614a5c8e7c7b8fb2f1efe195b0b4b993a6 Ville Tuulos tuulos@dxfront.(none) http://github.com/tuulos/disco/commit/54eaa9ffeed3b3bd2690acb69d1fbcaa92646f52 54eaa9ffeed3b3bd2690acb69d1fbcaa92646f52 2009-10-04T21:26:43-07:00 2009-10-04T21:20:42-07:00 Don't print out "Worker killed" messages for every killed task which can easily hide the traceback from the first failing task 648a1234aa574615ce79becfaa6a9f1a75891ca6 Ville Tuulos tuulos@dxfront.(none) db76c80fc293edecc3102420abe51c6a3f7f291f Ville Tuulos tuulos@dxfront.(none) http://github.com/tuulos/disco/commit/1013a7614a5c8e7c7b8fb2f1efe195b0b4b993a6 1013a7614a5c8e7c7b8fb2f1efe195b0b4b993a6 2009-10-04T20:57:45-07:00 2009-10-04T20:57:45-07:00 Increased truncation limit to 10000 bytes from 1000 b9594ed5feb7abfeb31fcdb93dfc9195f5928391 Ville Tuulos tuulos@dxfront.(none) 7635c9a39d8394be817ae03a9d0400fca87de140 Ville Tuulos tuulos@dxfront.(none) http://github.com/tuulos/disco/commit/db76c80fc293edecc3102420abe51c6a3f7f291f db76c80fc293edecc3102420abe51c6a3f7f291f 2009-10-04T20:54:20-07:00 2009-10-04T20:54:20-07:00 Apply rate limit to all messages on stdout / stderr. Closes #21 138e84e24e912f081c4913e44f2c6730c7b51899 Ville Tuulos tuulos@dxfront.(none) 16e79d0501327d2c93853eebafc00e6f6e195d2b Ville Tuulos tuulos@dxfront.(none) http://github.com/tuulos/disco/commit/7635c9a39d8394be817ae03a9d0400fca87de140 7635c9a39d8394be817ae03a9d0400fca87de140 2009-10-04T19:38:04-07:00 2009-10-04T19:38:04-07:00 Set LC_ALL=C for disco-worker to make sure that the external sort uses the traditional sort order based on byte values. Closes #36 3c106b958c9e0ac76bdb8bf3e65a5742fddbd031 Ville Tuulos tuulos@dxfront.(none) c5723bf998803e38f07f847c08b511dee2e6298a Jarno Seppänen jseppanen@gmail.com http://github.com/tuulos/disco/commit/16e79d0501327d2c93853eebafc00e6f6e195d2b 16e79d0501327d2c93853eebafc00e6f6e195d2b 2009-10-04T19:20:29-07:00 2009-09-17T04:42:40-07:00 Fix ctrl-c handling, i.e., pass KeyboardInterrupt thru catch-all exception handlers a8b9f7d577526e4b73499da04b33995f05b70f2f Ville Tuulos tuulos@dxfront.(none) dc4e3071f0791810f6d11d666c3b2405422266ae Ville Tuulos tuulos@dxfront.(none) http://github.com/tuulos/disco/commit/c5723bf998803e38f07f847c08b511dee2e6298a c5723bf998803e38f07f847c08b511dee2e6298a 2009-10-04T18:48:45-07:00 2009-10-04T18:48:45-07:00 Added a task_failed tag to WARN messages befad975e2994b07a99f361523420be1be5616a4 Ville Tuulos tuulos@dxfront.(none) 50fd5f683eb359b6d40d1fcb738aba03dcd1ecd3 Ville Tuulos tuulos@dxfront.(none) http://github.com/tuulos/disco/commit/dc4e3071f0791810f6d11d666c3b2405422266ae dc4e3071f0791810f6d11d666c3b2405422266ae 2009-10-04T18:34:58-07:00 2009-10-04T18:34:58-07:00 Don't set DISCO_DEBUG by default 1f363d4cd468feb641c8a6350c1a5836082600a3 Ville Tuulos tuulos@dxfront.(none) 69e281d1b5cd4a0a238977da033005d913603a21 Ville Tuulos tuulos@dxfront.(none) http://github.com/tuulos/disco/commit/50fd5f683eb359b6d40d1fcb738aba03dcd1ecd3 50fd5f683eb359b6d40d1fcb738aba03dcd1ecd3 2009-10-04T18:28:46-07:00 2009-10-04T18:28:46-07:00 Kill all jobs when the scheduler dies 516258c99e808c160b09d8f5b2a719bfe8d51f3a Ville Tuulos tuulos@dxfront.(none) 5ce636e1027c8b7b089a35624e82d25ab34d2733 Ville Tuulos tuulos@dxfront.(none) http://github.com/tuulos/disco/commit/69e281d1b5cd4a0a238977da033005d913603a21 69e281d1b5cd4a0a238977da033005d913603a21 2009-10-04T18:09:51-07:00 2009-10-04T18:09:51-07:00 disco_server starts fair_scheduler, not supervisor ce812a56eb8a0569db88135a40a565391ed4abae Ville Tuulos tuulos@dxfront.(none) c001f1869607c924ffa66b6a29d94a609b954fc4 Ville Tuulos tuulos@dxfront.(none) http://github.com/tuulos/disco/commit/5ce636e1027c8b7b089a35624e82d25ab34d2733 5ce636e1027c8b7b089a35624e82d25ab34d2733 2009-10-04T18:09:00-07:00 2009-10-04T18:00:36-07:00 Don't print the "Spawn cmd" string to logs unless DISCO_DEBUG is specified 91b034914cee86960e0b89471b8a58db3e3625cb Ville Tuulos tuulos@dxfront.(none) 63aa62c1aef131c34fc3a8d8f5a9f129627628fb Ville Tuulos tuulos@dxfront.(none) http://github.com/tuulos/disco/commit/c001f1869607c924ffa66b6a29d94a609b954fc4 c001f1869607c924ffa66b6a29d94a609b954fc4 2009-10-04T17:59:39-07:00 2009-10-04T17:59:39-07:00 Link disco_server to scheduler, check that node exist in white/blacklisting 7674830074676ecd33e86c64cc6f0c7ffa69473e Ville Tuulos tuulos@dxfront.(none) 5ba19d09f09690d2a55bc6817070d52ac684612c Ville Tuulos tuulos@dxfront.(none) http://github.com/tuulos/disco/commit/63aa62c1aef131c34fc3a8d8f5a9f129627628fb 63aa62c1aef131c34fc3a8d8f5a9f129627628fb 2009-10-04T17:08:37-07:00 2009-10-04T17:08:37-07:00 Blacklisted nodes should not be included in AvailableNodes f851900c118d14825b1d2d7441292c7d45cbd1fb Ville Tuulos tuulos@dxfront.(none) 7828f33a8bac46384fcd81c33fe658186dafabd7 Ville Tuulos tuulos@dxfront.(none) http://github.com/tuulos/disco/commit/5ba19d09f09690d2a55bc6817070d52ac684612c 5ba19d09f09690d2a55bc6817070d52ac684612c 2009-10-04T16:43:41-07:00 2009-10-04T16:43:41-07:00 Fixed fair_scheduler:update_nodes fc2a619359ac2b2993c5e5322ba37ba266d64b88 Ville Tuulos tuulos@dxfront.(none) f0085934d0a7dd73a23bd2bffee61d599d30ce14 Ville Tuulos tuulos@dxfront.(none) http://github.com/tuulos/disco/commit/7828f33a8bac46384fcd81c33fe658186dafabd7 7828f33a8bac46384fcd81c33fe658186dafabd7 2009-10-04T16:35:08-07:00 2009-10-04T16:35:08-07:00 Fixed message formatting error in {'EXIT', worker_dies} 36bbb931d2767f4b6cf3b68ab9b8a51f78927937 Ville Tuulos tuulos@dxfront.(none) 0c5e2790790962d746ff9f57c700aa4ab67d53de Ville Tuulos tuulos@dxfront.(none) http://github.com/tuulos/disco/commit/f0085934d0a7dd73a23bd2bffee61d599d30ce14 f0085934d0a7dd73a23bd2bffee61d599d30ce14 2009-10-04T16:24:43-07:00 2009-10-04T16:24:43-07:00 fix blacklisting f7edac440fa1ca7c5f5718d6cd195600529751ba Ville Tuulos tuulos@dxfront.(none) 507c0df7d204475befc4c1de3c30428629db38ed Ville Tuulos tuulos@dxfront.(none) http://github.com/tuulos/disco/commit/0c5e2790790962d746ff9f57c700aa4ab67d53de 0c5e2790790962d746ff9f57c700aa4ab67d53de 2009-10-04T13:08:45-07:00 2009-10-04T13:08:45-07:00 Added a default value for fair_scheduler_alpha 9508e2e7d104dfd643cc47d7eb8034565d96c40c Ville Tuulos tuulos@dxfront.(none) 2354261e3d6d29efc16db618918eb9716ec2ec42 0d47a5c64e667100354c6df1cd8096938645e648 Ville Tuulos tuulos@dxfront.(none) http://github.com/tuulos/disco/commit/507c0df7d204475befc4c1de3c30428629db38ed 507c0df7d204475befc4c1de3c30428629db38ed 2009-10-04T12:47:20-07:00 2009-10-04T00:47:57-07:00 Merge branch 'scheduler' Conflicts: master/disco-master master/src/disco_server.erl master/src/disco_worker.erl master/src/handle_job.erl 21529b8dc0d1f5a8edb22b81d6696931a89be515 Ville Tuulos tuulos@dxfront.(none) bafa71996a190d24b8e33875747a5a64cb88da5c Ville Tuulos tuulos@dxfront.(none) http://github.com/tuulos/disco/commit/2354261e3d6d29efc16db618918eb9716ec2ec42 2354261e3d6d29efc16db618918eb9716ec2ec42 2009-09-09T19:03:31-07:00 2009-09-09T19:03:31-07:00 Added a note about the removal of the ancient disco module support in release notes e3b098b806ec6f9e0cf798d6ec1a37a158386933 Ville Tuulos tuulos@dxfront.(none) e6c9a811e040e3fd98a3f8716acfccfe4d378225 Ville Tuulos tuulos@dxfront.(none) http://github.com/tuulos/disco/commit/bafa71996a190d24b8e33875747a5a64cb88da5c bafa71996a190d24b8e33875747a5a64cb88da5c 2009-09-09T18:47:33-07:00 2009-09-09T18:47:33-07:00 Release notes for 0.2.3 6c1041b8796c16e5af124bc2d301fff01a42f3ae Ville Tuulos tuulos@dxfront.(none) e1e21375b493fb063da1accf894a8f8729f096f7 Ville Tuulos tuulos@dxfront.(none) http://github.com/tuulos/disco/commit/e6c9a811e040e3fd98a3f8716acfccfe4d378225 e6c9a811e040e3fd98a3f8716acfccfe4d378225 2009-09-09T18:04:42-07:00 2009-09-09T18:04:42-07:00 Updated version to 0.2.3 2ed827f458d6b5b3fe9e0c1761bad03bcd92bd65 Ville Tuulos tuulos@dxfront.(none) 4e1841aae02eb4ab142fe1a7d18890d1aac7aee2 Ville Tuulos tuulos@dxfront.(none) http://github.com/tuulos/disco/commit/e1e21375b493fb063da1accf894a8f8729f096f7 e1e21375b493fb063da1accf894a8f8729f096f7 2009-09-09T17:50:47-07:00 2009-09-09T17:50:47-07:00 Documentation updates for 0.2.3 e8187fc2fd81f8bb79445a466cf17a30eb43930a Ville Tuulos tuulos@dxfront.(none) f1d4a127c991eef574bac164f10fadf7b6981298 Ville Tuulos tuulos@dxfront.(none) http://github.com/tuulos/disco/commit/4e1841aae02eb4ab142fe1a7d18890d1aac7aee2 4e1841aae02eb4ab142fe1a7d18890d1aac7aee2 2009-09-09T16:58:37-07:00 2009-09-09T16:58:37-07:00 Failed URL is now included CommException cef88d68e3c9152155cf0f22fe7b04910d1a37e3 Ville Tuulos tuulos@dxfront.(none) 7885a37f0e80124ab737f6dd0e172df73a2f2d74 Ville Tuulos tuulos@dxfront.(none) http://github.com/tuulos/disco/commit/f1d4a127c991eef574bac164f10fadf7b6981298 f1d4a127c991eef574bac164f10fadf7b6981298 2009-09-09T16:46:00-07:00 2009-09-09T16:46:00-07:00 All IOErrors and CommExceptions in disco-worker now raise data_err(), so they shouldn't crash the job anymore a4c9de1e538d91bcbcda0038b37b890e0c69ec41 Ville Tuulos tuulos@dxfront.(none) dc3a5e36b66f1346a216e33c43b19f60c6b074e6 Ville Tuulos tuulos@dxfront.(none) http://github.com/tuulos/disco/commit/7885a37f0e80124ab737f6dd0e172df73a2f2d74 7885a37f0e80124ab737f6dd0e172df73a2f2d74 2009-09-09T16:44:27-07:00 2009-09-09T16:44:27-07:00 Fixed Makefile, "cp -n" doesn't work on Linux 0cf6af1bedde5100f41aa48a0fcf37029441d0cf Ville Tuulos tuulos@dxfront.(none) 2f79c69879b4b5f565eb8023d680f91d6b787641 Ville Tuulos tuulos@dxfront.(none) http://github.com/tuulos/disco/commit/dc3a5e36b66f1346a216e33c43b19f60c6b074e6 dc3a5e36b66f1346a216e33c43b19f60c6b074e6 2009-09-09T16:44:10-07:00 2009-09-09T16:44:10-07:00 Fixed a PYTHONPATH issue in disco.py a636cf6cbdadc3546aaaf716a14f0be09172278d Ville Tuulos tuulos@dxfront.(none) 3467a9e5f14138c63d252d7fd233caf78c64a782 Ville Tuulos tuulos@dxfront.(none) http://github.com/tuulos/disco/commit/2f79c69879b4b5f565eb8023d680f91d6b787641 2f79c69879b4b5f565eb8023d680f91d6b787641 2009-09-09T15:02:16-07:00 2009-09-09T15:02:16-07:00 bugfix: Added timeouts to comm_curl to fix occasional issues with the connection getting stuck d369cf1395e573ca68b23447afe78e6446b080a0 Ville Tuulos tuulos@dxfront.(none) 8035333ac4926c6f3ab7ec776ec7ea90149b1d92 Ville Tuulos tuulos@dxfront.(none) http://github.com/tuulos/disco/commit/3467a9e5f14138c63d252d7fd233caf78c64a782 3467a9e5f14138c63d252d7fd233caf78c64a782 2009-09-09T14:53:59-07:00 2009-09-09T14:53:59-07:00 Be extra paranoid if connection in comm_httplib fails: check that the connection is actually pooled before deleting it 8be6b322ff700a119f3a53d8b774a1aececc4622 Ville Tuulos tuulos@dxfront.(none) 5c9d7a88e9dec69317d59384f2bedcec92c30f98 Jared Flatow jflatow@gmail.com http://github.com/tuulos/disco/commit/8035333ac4926c6f3ab7ec776ec7ea90149b1d92 8035333ac4926c6f3ab7ec776ec7ea90149b1d92 2009-09-08T20:42:49-07:00 2009-09-08T20:42:49-07:00 updated documentation for new startup script 19dd23057589e7633ccea785112c770366f2a2f1 Jared Flatow jflatow@gmail.com 1cb4eb16336efcd0ac61d882c2ca2bcc8f4e4e9e Jared Flatow jflatow@gmail.com http://github.com/tuulos/disco/commit/5c9d7a88e9dec69317d59384f2bedcec92c30f98 5c9d7a88e9dec69317d59384f2bedcec92c30f98 2009-09-08T18:33:48-07:00 2009-09-08T18:32:45-07:00 bugfix: comm_httplib only caught BadStatusLine and did not limit number of retries (closes #30) 71abc2f3bf88fbfb3f410b2aa61bf5025949e8a2 Jared Flatow jflatow@gmail.com