Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

schedule调度失败问题 #29

Closed
garykiller opened this issue Jul 7, 2017 · 2 comments
Closed

schedule调度失败问题 #29

garykiller opened this issue Jul 7, 2017 · 2 comments

Comments

@garykiller
Copy link

schedule触发以后,任务无法正常执行,截取sk端与scrapyd之间的log如下,请问可能是什么原因?

SK 端log:

2017-07-06 19:00:00,037 - SpiderKeeper.app - ERROR - [run_spider_job] (sqlite3.IntegrityError) sk_job_execution.service_job_execution_id may not be NULL [SQL: u'INSERT INTO sk_job_execution (date_created, date_modified, project_id, service_job_execution_id, job_name, job_instance_id, create_time, start_time, end_time, running_status, running_on) VALUES (CURRENT_TIMESTAMP, CURRENT_TIMESTAMP, ?, ?, ?, ?, ?, ?, ?, ?, ?)'] [parameters: (1, None, u'\u78a7\u6842\u56ed\u8206\u60c5\u5173\u952e\u8bcd1', 3, '2017-07-06 19:00:00.033838', None, None, 0, 'http://127.0.0.1:6800')]

ERROR in common [/opt/SpiderKeeper/SpiderKeeper/app/schedulers/common.py:41]:
[run_spider_job] (sqlite3.IntegrityError) sk_job_execution.service_job_execution_id may not be NULL [SQL: u'INSERT INTO sk_job_execution (date_created, date_modified, project_id, service_job_execution_id, job_name, job_instance_id, create_time, start_time, end_time, running_status, running_on) VALUES (CURRENT_TIMESTAMP, CURRENT_TIMESTAMP, ?, ?, ?, ?, ?, ?, ?, ?, ?)'] [parameters: (1, None, u'\u78a7\u6842\u56ed\u8206\u60c5\u5173\u952e\u8bcd2', 2, '2017-07-06 19:00:00.035478', None, None, 0, 'http://127.0.0.1:6800')]

2017-07-06 19:00:00,037 - SpiderKeeper.app - ERROR - [run_spider_job] (sqlite3.IntegrityError) sk_job_execution.service_job_execution_id may not be NULL [SQL: u'INSERT INTO sk_job_execution (date_created, date_modified, project_id, service_job_execution_id, job_name, job_instance_id, create_time, start_time, end_time, running_status, running_on) VALUES (CURRENT_TIMESTAMP, CURRENT_TIMESTAMP, ?, ?, ?, ?, ?, ?, ?, ?, ?)'] [parameters: (1, None, u'\u78a7\u6842\u56ed\u8206\u60c5\u5173\u952e\u8bcd2', 2, '2017-07-06 19:00:00.035478', None, None, 0, 'http://127.0.0.1:6800')]

ERROR in common [/opt/SpiderKeeper/SpiderKeeper/app/schedulers/common.py:41]:
[run_spider_job] (sqlite3.IntegrityError) sk_job_execution.service_job_execution_id may not be NULL [SQL: u'INSERT INTO sk_job_execution (date_created, date_modified, project_id, service_job_execution_id, job_name, job_instance_id, create_time, start_time, end_time, running_status, running_on) VALUES (CURRENT_TIMESTAMP, CURRENT_TIMESTAMP, ?, ?, ?, ?, ?, ?, ?, ?, ?)'] [parameters: (1, None, u'\u78a7\u6842\u56ed\u653f\u7b56\u4eba\u7269\u5173\u952e\u8bcd', 4, '2017-07-06 19:00:00.034746', None, None, 0, 'http://127.0.0.1:6800')]

2017-07-06 19:00:00,039 - SpiderKeeper.app - ERROR - [run_spider_job] (sqlite3.IntegrityError) sk_job_execution.service_job_execution_id may not be NULL [SQL: u'INSERT INTO sk_job_execution (date_created, date_modified, project_id, service_job_execution_id, job_name, job_instance_id, create_time, start_time, end_time, running_status, running_on) VALUES (CURRENT_TIMESTAMP, CURRENT_TIMESTAMP, ?, ?, ?, ?, ?, ?, ?, ?, ?)'] [parameters: (1, None, u'\u78a7\u6842\u56ed\u653f\u7b56\u4eba\u7269\u5173\u952e\u8bcd', 4, '2017-07-06 19:00:00.034746', None, None, 0, 'http://127.0.0.1:6800')]

ERROR in common [/opt/SpiderKeeper/SpiderKeeper/app/schedulers/common.py:41]:
[run_spider_job] (sqlite3.IntegrityError) sk_job_execution.service_job_execution_id may not be NULL [SQL: u'INSERT INTO sk_job_execution (date_created, date_modified, project_id, service_job_execution_id, job_name, job_instance_id, create_time, start_time, end_time, running_status, running_on) VALUES (CURRENT_TIMESTAMP, CURRENT_TIMESTAMP, ?, ?, ?, ?, ?, ?, ?, ?, ?)'] [parameters: (1, None, u'\u78a7\u6842\u56ed\u7ade\u4e89\u5bf9\u624b\u5173\u952e\u8bcd', 5, '2017-07-06 19:00:00.039962', None, None, 0, 'http://127.0.0.1:6800')]

2017-07-06 19:00:00,040 - SpiderKeeper.app - ERROR - [run_spider_job] (sqlite3.IntegrityError) sk_job_execution.service_job_execution_id may not be NULL [SQL: u'INSERT INTO sk_job_execution (date_created, date_modified, project_id, service_job_execution_id, job_name, job_instance_id, create_time, start_time, end_time, running_status, running_on) VALUES (CURRENT_TIMESTAMP, CURRENT_TIMESTAMP, ?, ?, ?, ?, ?, ?, ?, ?, ?)'] [parameters: (1, None, u'\u78a7\u6842\u56ed\u7ade\u4e89\u5bf9\u624b\u5173\u952e\u8bcd', 5, '2017-07-06 19:00:00.039962', None, None, 0, 'http://127.0.0.1:6800')]

ERROR in common [/opt/SpiderKeeper/SpiderKeeper/app/schedulers/common.py:41]:
[run_spider_job] This Session's transaction has been rolled back due to a previous exception during flush. To begin a new transaction with this Session, first issue Session.rollback(). Original exception was: (sqlite3.IntegrityError) sk_job_execution.service_job_execution_id may not be NULL [SQL: u'INSERT INTO sk_job_execution (date_created, date_modified, project_id, service_job_execution_id, job_name, job_instance_id, create_time, start_time, end_time, running_status, running_on) VALUES (CURRENT_TIMESTAMP, CURRENT_TIMESTAMP, ?, ?, ?, ?, ?, ?, ?, ?, ?)'] [parameters: (1, None, u'\u78a7\u6842\u56ed\u8206\u60c5\u5173\u952e\u8bcd2', 2, '2017-07-06 18:30:00.026636', None, None, 0, 'http://127.0.0.1:6800')]

Scrapyd端log:
2017-07-06T19:00:00+0800 [twisted.python.log#info] "127.0.0.1" - - [06/Jul/2017:10:59:59 +0000] "POST /schedule.json HTTP/1.1" 200 67 "-" "python-requests/2.13.0"
2017-07-06T19:00:00+0800 [twisted.python.log#info] "127.0.0.1" - - [06/Jul/2017:10:59:59 +0000] "POST /schedule.json HTTP/1.1" 200 67 "-" "python-requests/2.13.0"
2017-07-06T19:00:00+0800 [twisted.python.log#info] "127.0.0.1" - - [06/Jul/2017:10:59:59 +0000] "POST /schedule.json HTTP/1.1" 200 67 "-" "python-requests/2.13.0"
2017-07-06T19:00:00+0800 [twisted.python.log#info] "127.0.0.1" - - [06/Jul/2017:10:59:59 +0000] "POST /schedule.json HTTP/1.1" 200 67 "-" "python-requests/2.13.0"
2017-07-06T19:00:00+0800 [twisted.python.log#info] "127.0.0.1" - - [06/Jul/2017:11:00:00 +0000] "GET /listjobs.json?project=secrawler HTTP/1.1" 200 351 "-" "python-requests/2.13.0"
2017-07-06T19:00:00+0800 [twisted.python.log#info] "127.0.0.1" - - [06/Jul/2017:11:00:00 +0000] "GET /listjobs.json?project=news4 HTTP/1.1" 200 95 "-" "python-requests/2.13.0"
2017-07-06T19:00:00+0800 [twisted.python.log#info] "127.0.0.1" - - [06/Jul/2017:11:00:00 +0000] "GET /listjobs.json?project=news_all HTTP/1.1" 200 331 "-" "python-requests/2.13.0"
2017-07-06T19:00:01+0800 [Launcher,12876/stdout] Unable to format event {'log_namespace': u'log_legacy', 'log_time': 1499338801.834994, 'log_system': 'Launcher,12876/stdout', 'log_level': <LogLevel=info>, 'system': 'Launcher,12876/stdout', 'time': 1499338801.834994, 'log_text': 'ng.my&pn=0&cl=2&ct=0&tn=news&rn=20&ie=utf-8&bt=0&et=0\ntotal page:1\nhttp://www.baidu.com/s?rtt=2&wd=\xe7\xa2\xa7\xe6\xa1\x82\xe5\x9b\xad \xe9\x80\x80\xe6\x88\xbf site:enanyang.my&pn=0&tn=baiduwb&ie=utf-8\ntotal page:1\nhttp://tieba.baidu.com/f/search/res?isnew=1&kw=&qw=\xe7\xa2\xa7\xe6\xa1\x82\xe5\x9b\xad \xe9\x80\x80\xe6\x88\xbf site:enanyang.my&rn=10&un=&only_thread=1&sm=1&sd=&ed=&pn=0&ie=utf-8\ntotal page:1\nhttps://www.sogou.com/web?query=\xe7\xa2\xa7\xe6\xa1\x82\xe5\x9b\xad \xe9\x80\x80\xe6\x88\xbf site:enanyang.my&page=0&ie=utf8\nExtracting date from http://www.so.com/link?m=ajUS%2F4k3o89yP3cZN7Qfdzcvo4ZyKqq9GhEdc2wQ05hqgs4VcvOzSjW80ezfabEVSBca%2BTx064aSyGK%2B9JGJ5CtU0d1gxe99TOXJDub1muxcf4vKj0mBdQgVVfJ2mdTFr%2B3bQ%2Bd9OItJ%2BR1bV%2F2NAdC4ldkXeOqbS%2FUoeX%2Fp%2BX1zW83JiV8VsqPADBA1EqkZP4OpxpMXNFDQMR4Rz0UNgkFEaOIiK0YgzEtk5u%2FkOncII%2Fr6p7s%2FuYEZvcHJwayp5IY9Kh8Qyoq95GWSp3nksMXuXNc41048as6xTFBsZb7B0qjkrJ%2Bj0CXIWlr7a5uTdq95vTMsGl%2Ft6bX0xd%2F3NSZcJisUpj3eDAH9KRw41MbA%3D\ntotal page:1\nhttp://weixin.sogou.com/weixin?query=\xe7\xa2\xa7\xe6\xa1\x82\xe5\x9b\xad \xe9\x80\x80\xe6\x88\xbf site:enanyang.my&type=2&page=0\ntotal page:1\nhttps://www.so.com/s?q=\xe7\xa2\xa7\xe6\xa1\x82\xe5\x9b\xad \xe9\x80\x80\xe6\x88\xbf site:enanyang.my&pn=0\ntotal page:1\nhttp://news.so.com/ns?q=\xe7\xa2\xa7\xe6\xa1\x82\xe5\x9b\xad \xe9\x80\x80\xe6\x88\xbf site:enanyang.my&pn=0&tn=news&rank=pdate&j=0&src=page\ntotal page:1\nhttp://www.bing.com/search?q=\xe7\xa2\xa7\xe6\xa1\x82\xe5\x9b\xad \xe9\x80\x80\xe6\x88\xbf site:enanyang.my&first=0\ntotal page:1\nhttp://www.baidu.com/s?wd=\xe7\xa2\xa7\xe6\xa1\x82\xe5\x9b\xad \xe9\x80\x80\xe6\x88\xbf site:www.qhcin.gov.cn&pn=0&ie=utf-8\ntotal page:1\nhttp://news.baidu.com/ns?word=\xe7\xa2\xa7\xe6\xa1\x82\xe5\x9b\xad \xe9\x80\x80\xe6\x88\xbf site:www.qhcin.gov.cn&pn=0&cl=2&ct=0&tn=news&rn=20&ie=utf-8&bt=0&et=0\ntotal page:1\nhttp://www.baidu.com/s?rtt=2&wd=\xe7\xa2\xa7\xe6\xa1\x82\xe5\x9b\xad \xe9\x80\x80\xe6\x88\xbf site:www.qhcin.gov.cn&pn=0&tn=baiduwb&ie=utf-8\nExtracting date from http://stock.hexun.com/2015-11-10/180470145.html\ntotal page:1\nhttp://tieba.baidu.com/f/search/res?isnew=1&kw=&qw=\xe7\xa2\xa7\xe6\xa1\x82\xe5\x9b\xad \xe9\x80\x80\xe6\x88\xbf site:www.qhcin.gov.cn&rn=10&un=&only_thread=1&sm=1&sd=&ed=&pn=0&ie=utf-8\ntotal page:1\nhttps://www.sogou.com/web?query=\xe7\xa2\xa7\xe6\xa1\x82\xe5\x9b\xad \xe9\x80\x80\xe6\x88\xbf site:www.qhcin.gov.cn&page=0&ie=utf8\ntotal page:1\nhttp://weixin.sogou.com/weixin?query=\xe7\xa2\xa7\xe6\xa1\x82\xe5\x9b\xad \xe9\x80\x80\xe6\x88\xbf site:www.qhcin.gov.cn&type=2&page=0\ntotal page:1\nhttps://www.so.com/s?q=\xe7\xa2\xa7\xe6\xa1\x82\xe5\x9b\xad \xe9\x80\x80\xe6\x88\xbf site:www.qhcin.gov.cn&pn=0\ntotal page:1\nhttp://news.so.com/ns?q=\xe7\xa2\xa7\xe6\xa1\x82\xe5\x9b\xad \xe9\x80\x80\xe6\x88\xbf site:www.qhcin.gov.cn&pn=0&tn=news&rank=pdate&j=0&src=page\ntotal page:1\nhttp://www.bing.com/search?q=\xe7\xa2\xa7\xe6\xa1\x82\xe5\x9b\xad \xe9\x80\x80\xe6\x88\xbf site:www.qhcin.gov.cn&first=0\nExtracting date from http://www.so.com/link?m=aC8wY9hcupuXdD2dJrJWQS5QUKxCBBWl3pYbjI9tr6QzLkj4P7Uua%2B4t8oZttb70g0H0vel5suJc1W8Q04Xz5JFYYpX3mKPJlWrc2TqR32wAbVbWP2oIq0v%2FkXhC7MHDe1R%2FFOPZh%2BexjAILT7T4ZjikI1PrydGyCKjSdO5i%2Fv4l0uHpi%2BoFuBprWjgE%3D\nExtracting date from http://www.so.com/link?m=avF3cmCdtrLPkJjd3BA0ujFh2G9A1wnp3wXfLEJjy%2FvKaGKU%2Bt1zM5K0FdL4d3eMyKQRa1ld4VcPX843Sc7jkuA4gD4pJou%2BnzobE9ACcw6c3Ct0g3%2B0exg8vl7t71GwX9e15cOMlNW5FgyfSoSGYJmIgNAD4SotTGNUIcT4MIn7uutq8w8q7uPhi%2FUWF4hK0XDNKx8SRxNE%3D\nExtracting date from http://www.so.com/link?m=aGwhsOV6kskLT3OHaAE7GUzMQqhkXNtXFTRKPCVwHt7qItcT%2FBFhxz29hDclC%2FbVzpjpLYHF6QdDdK8NPhyFFvGFSIZAqorvmPGdbLAw6gpfJ37wZosdzuwE0FpTdCehR%2BRoYvof7kD%2F%2Bgueitf%2F%2BFgO2a0ooMl%2F%2BxmMKVyYtEcgQdWeC347xErnRPF8N3xmEA4%2BpTPH0k9s%3D\nExtracting date from http://www.so.com/link?m=ap4wN3YXVA8glj2bLWcxyOHhGlLhpD7%2FR1vLPUx%2FfoeReZusXoR4JcOHmRWtEuf0nhIGEyFe14QvAgpKKqcfQYgkjNziN7BjW71kbKdn6tnjkY8QDgRK5M9IK%2BeGTOhiKObLWtlTMg7KtSRZ%2FrF0%2FOfj4jGU%2FcRFvTeClmVjXvyYE3DeeM0s7ej3oNrm%2BORTc59kC4tJ%2B86Q%3D\nExtracting date from http://www.so.com/link?m=a3ck%2B2lrBxRwHPbPJ6w1gQERV62ONUWAPF741rDWcz%2Fxo1CZNHtV%2BYrPkw3bGAHSx5PlKWVLrr9i69bQP1SVnWrzX52IjmJtoRPLrBGHCfJWew%2FEmVb12MZuCyjc9gboONohXEYP%2BSAKOC5SDg%2BOnJVNYz1drLDe%2BtIgrBS0Rea8J%2FCk%2FoJJK%2FvK5VGLos4sP7Ovk0448YzE%3D\nExtracting date from http://www.so.com/link?m=aGEEZtMmklGW%2F5fJM95Bl8VkIxQrCV2tr0tVwPXVYEv2SK7u%2BnyiGNIrtA%2FKjid9ULAU%2FgKrqyVy1Qtf4b8pJDMe4MrQnfQG2jjOdGSDhGLlUV%2FeQQHGEzUaTUQ%2FpvTGejb1ON0JdUpfinoIKNrcnV8VejMLUcS20p4tX2Gd3LC1vGL6q3HcQ1alW6QR1DAiTFFNWBvS0jj0%3D\nExtracting date from http://www.so.com/link?m=apC%2BdaVT2B2Hhi5j4AVL5CiIpr8WzPrdGVd3uCubJ0LdHKMrGwpCaie%2B4TFuY1gqePImOjRXUwbWQcBOt4puYIs1J5RnNLge0SxADFHTTODEKeclCxNyJHbWfEA89FHaI4WGW%2F3ksNWvzamEk5qmEEb2qzDzk0EJUUyGoDsALJhR%2FxP3LhwXSK44iUVzhrLjQ', 'log_format': u'{log_text}', 'message': ('ng.my&pn=0&cl=2&ct=0&tn=news&rn=20&ie=utf-8&bt=0&et=0\ntotal page:1\nhttp://www.baidu.com/s?rtt=2&wd=\xe7\xa2\xa7\xe6\xa1\x82\xe5\x9b\xad \xe9\x80\x80\xe6\x88\xbf site:enanyang.my&pn=0&tn=baiduwb&ie=utf-8\ntotal page:1\nhttp://tieba.baidu.com/f/search/res?isnew=1&kw=&qw=\xe7\xa2\xa7\xe6\xa1\x82\xe5\x9b\xad \xe9\x80\x80\xe6\x88\xbf site:enanyang.my&rn=10&un=&only_thread=1&sm=1&sd=&ed=&pn=0&ie=utf-8\ntotal page:1\nhttps://www.sogou.com/web?query=\xe7\xa2\xa7\xe6\xa1\x82\xe5\x9b\xad \xe9\x80\x80\xe6\x88\xbf site:enanyang.my&page=0&ie=utf8\nExtracting date from http://www.so.com/link?m=ajUS%2F4k3o89yP3cZN7Qfdzcvo4ZyKqq9GhEdc2wQ05hqgs4VcvOzSjW80ezfabEVSBca%2BTx064aSyGK%2B9JGJ5CtU0d1gxe99TOXJDub1muxcf4vKj0mBdQgVVfJ2mdTFr%2B3bQ%2Bd9OItJ%2BR1bV%2F2NAdC4ldkXeOqbS%2FUoeX%2Fp%2BX1zW83JiV8VsqPADBA1EqkZP4OpxpMXNFDQMR4Rz0UNgkFEaOIiK0YgzEtk5u%2FkOncII%2Fr6p7s%2FuYEZvcHJwayp5IY9Kh8Qyoq95GWSp3nksMXuXNc41048as6xTFBsZb7B0qjkrJ%2Bj0CXIWlr7a5uTdq95vTMsGl%2Ft6bX0xd%2F3NSZcJisUpj3eDAH9KRw41MbA%3D\ntotal page:1\nhttp://weixin.sogou.com/weixin?query=\xe7\xa2\xa7\xe6\xa1\x82\xe5\x9b\xad \xe9\x80\x80\xe6\x88\xbf site:enanyang.my&type=2&page=0\ntotal page:1\nhttps://www.so.com/s?q=\xe7\xa2\xa7\xe6\xa1\x82\xe5\x9b\xad \xe9\x80\x80\xe6\x88\xbf site:enanyang.my&pn=0\ntotal page:1\nhttp://news.so.com/ns?q=\xe7\xa2\xa7\xe6\xa1\x82\xe5\x9b\xad \xe9\x80\x80\xe6\x88\xbf site:enanyang.my&pn=0&tn=news&rank=pdate&j=0&src=page\ntotal page:1\nhttp://www.bing.com/search?q=\xe7\xa2\xa7\xe6\xa1\x82\xe5\x9b\xad \xe9\x80\x80\xe6\x88\xbf site:enanyang.my&first=0\ntotal page:1\nhttp://www.baidu.com/s?wd=\xe7\xa2\xa7\xe6\xa1\x82\xe5\x9b\xad \xe9\x80\x80\xe6\x88\xbf site:www.qhcin.gov.cn&pn=0&ie=utf-8\ntotal page:1\nhttp://news.baidu.com/ns?word=\xe7\xa2\xa7\xe6\xa1\x82\xe5\x9b\xad \xe9\x80\x80\xe6\x88\xbf site:www.qhcin.gov.cn&pn=0&cl=2&ct=0&tn=news&rn=20&ie=utf-8&bt=0&et=0\ntotal page:1\nhttp://www.baidu.com/s?rtt=2&wd=\xe7\xa2\xa7\xe6\xa1\x82\xe5\x9b\xad \xe9\x80\x80\xe6\x88\xbf site:www.qhcin.gov.cn&pn=0&tn=baiduwb&ie=utf-8\nExtracting date from http://stock.hexun.com/2015-11-10/180470145.html\ntotal page:1\nhttp://tieba.baidu.com/f/search/res?isnew=1&kw=&qw=\xe7\xa2\xa7\xe6\xa1\x82\xe5\x9b\xad \xe9\x80\x80\xe6\x88\xbf site:www.qhcin.gov.cn&rn=10&un=&only_thread=1&sm=1&sd=&ed=&pn=0&ie=utf-8\ntotal page:1\nhttps://www.sogou.com/web?query=\xe7\xa2\xa7\xe6\xa1\x82\xe5\x9b\xad \xe9\x80\x80\xe6\x88\xbf site:www.qhcin.gov.cn&page=0&ie=utf8\ntotal page:1\nhttp://weixin.sogou.com/weixin?query=\xe7\xa2\xa7\xe6\xa1\x82\xe5\x9b\xad \xe9\x80\x80\xe6\x88\xbf site:www.qhcin.gov.cn&type=2&page=0\ntotal page:1\nhttps://www.so.com/s?q=\xe7\xa2\xa7\xe6\xa1\x82\xe5\x9b\xad \xe9\x80\x80\xe6\x88\xbf site:www.qhcin.gov.cn&pn=0\ntotal page:1\nhttp://news.so.com/ns?q=\xe7\xa2\xa7\xe6\xa1\x82\xe5\x9b\xad \xe9\x80\x80\xe6\x88\xbf site:www.qhcin.gov.cn&pn=0&tn=news&rank=pdate&j=0&src=page\ntotal page:1\nhttp://www.bing.com/search?q=\xe7\xa2\xa7\xe6\xa1\x82\xe5\x9b\xad \xe9\x80\x80\xe6\x88\xbf site:www.qhcin.gov.cn&first=0\nExtracting date from http://www.so.com/link?m=aC8wY9hcupuXdD2dJrJWQS5QUKxCBBWl3pYbjI9tr6QzLkj4P7Uua%2B4t8oZttb70g0H0vel5suJc1W8Q04Xz5JFYYpX3mKPJlWrc2TqR32wAbVbWP2oIq0v%2FkXhC7MHDe1R%2FFOPZh%2BexjAILT7T4ZjikI1PrydGyCKjSdO5i%2Fv4l0uHpi%2BoFuBprWjgE%3D\nExtracting date from http://www.so.com/link?m=avF3cmCdtrLPkJjd3BA0ujFh2G9A1wnp3wXfLEJjy%2FvKaGKU%2Bt1zM5K0FdL4d3eMyKQRa1ld4VcPX843Sc7jkuA4gD4pJou%2BnzobE9ACcw6c3Ct0g3%2B0exg8vl7t71GwX9e15cOMlNW5FgyfSoSGYJmIgNAD4SotTGNUIcT4MIn7uutq8w8q7uPhi%2FUWF4hK0XDNKx8SRxNE%3D\nExtracting date from http://www.so.com/link?m=aGwhsOV6kskLT3OHaAE7GUzMQqhkXNtXFTRKPCVwHt7qItcT%2FBFhxz29hDclC%2FbVzpjpLYHF6QdDdK8NPhyFFvGFSIZAqorvmPGdbLAw6gpfJ37wZosdzuwE0FpTdCehR%2BRoYvof7kD%2F%2Bgueitf%2F%2BFgO2a0ooMl%2F%2BxmMKVyYtEcgQdWeC347xErnRPF8N3xmEA4%2BpTPH0k9s%3D\nExtracting date from http://www.so.com/link?m=ap4wN3YXVA8glj2bLWcxyOHhGlLhpD7%2FR1vLPUx%2FfoeReZusXoR4JcOHmRWtEuf0nhIGEyFe14QvAgpKKqcfQYgkjNziN7BjW71kbKdn6tnjkY8QDgRK5M9IK%2BeGTOhiKObLWtlTMg7KtSRZ%2FrF0%2FOfj4jGU%2FcRFvTeClmVjXvyYE3DeeM0s7ej3oNrm%2BORTc59kC4tJ%2B86Q%3D\nExtracting date from http://www.so.com/link?m=a3ck%2B2lrBxRwHPbPJ6w1gQERV62ONUWAPF741rDWcz%2Fxo1CZNHtV%2BYrPkw3bGAHSx5PlKWVLrr9i69bQP1SVnWrzX52IjmJtoRPLrBGHCfJWew%2FEmVb12MZuCyjc9gboONohXEYP%2BSAKOC5SDg%2BOnJVNYz1drLDe%2BtIgrBS0Rea8J%2FCk%2FoJJK%2FvK5VGLos4sP7Ovk0448YzE%3D\nExtracting date from http://www.so.com/link?m=aGEEZtMmklGW%2F5fJM95Bl8VkIxQrCV2tr0tVwPXVYEv2SK7u%2BnyiGNIrtA%2FKjid9ULAU%2FgKrqyVy1Qtf4b8pJDMe4MrQnfQG2jjOdGSDhGLlUV%2FeQQHGEzUaTUQ%2FpvTGejb1ON0JdUpfinoIKNrcnV8VejMLUcS20p4tX2Gd3LC1vGL6q3HcQ1alW6QR1DAiTFFNWBvS0jj0%3D\nExtracting date from http://www.so.com/link?m=apC%2BdaVT2B2Hhi5j4AVL5CiIpr8WzPrdGVd3uCubJ0LdHKMrGwpCaie%2B4TFuY1gqePImOjRXUwbWQcBOt4puYIs1J5RnNLge0SxADFHTTODEKeclCxNyJHbWfEA89FHaI4WGW%2F3ksNWvzamEk5qmEEb2qzDzk0EJUUyGoDsALJhR%2FxP3LhwXSK44iUVzhrLjQ',), 'isError': 0}: 'ascii' codec can't decode byte 0xe7 in position 99: ordinal not in range(128)
2017-07-06T19:00:05+0800 [twisted.python.log#info] "127.0.0.1" - - [06/Jul/2017:11:00:05 +0000] "GET /listjobs.json?project=secrawler HTTP/1.1" 200 351 "-" "python-requests/2.13.0"
2017-07-06T19:00:05+0800 [twisted.python.log#info] "127.0.0.1" - - [06/Jul/2017:11:00:05 +0000] "GET /listjobs.json?project=news4 HTTP/1.1" 200 95 "-" "python-requests/2.13.0"
2017-07-06T19:00:05+0800 [twisted.python.log#info] "127.0.0.1" - - [06/Jul/2017:11:00:05 +0000] "GET /listjobs.json?project=news_all HTTP/1.1" 200 331 "-" "python-requests/2.13.0"

@DormyMo
Copy link
Owner

DormyMo commented Jul 8, 2017

检查scrapyd服务是否正常

@garykiller
Copy link
Author

感谢答复,可能是scarpyd的问题,我重新启动了一下,运行正常了。但是encoding的问题仍然存在。

@DormyMo DormyMo closed this as completed Jul 25, 2017
Zephyrrus pushed a commit to Zephyrrus/UBGuardian that referenced this issue Oct 25, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants