New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

反馈区:欢迎提意见 #9

Closed
howie6879 opened this Issue Aug 25, 2017 · 59 comments

Comments

Projects
None yet
@howie6879
Copy link
Owner

howie6879 commented Aug 25, 2017

欢迎提建议,为了使项目更好,谢谢。
这里提供多种解决方案:https://github.com/owllook-for-you

@howie6879 howie6879 added the 反馈 label Aug 25, 2017

@zhang0peter

This comment has been minimized.

Copy link

zhang0peter commented Aug 31, 2017

没问题,感谢你的网站。

@yangfan920215

This comment has been minimized.

Copy link

yangfan920215 commented Sep 6, 2017

很不错的站点,对我学习python很有帮助,感谢

@howie6879 howie6879 changed the title 反馈区:欢迎提意见 反馈区:欢迎提意见 # https://github.com/owllook-for-you Sep 20, 2017

@howie6879 howie6879 changed the title 反馈区:欢迎提意见 # https://github.com/owllook-for-you 反馈区:欢迎提意见 Sep 20, 2017

@jiangxin123

This comment has been minimized.

Copy link

jiangxin123 commented Dec 5, 2017

安装包报错:uvloop does not support Windows at the moment

@howie6879

This comment has been minimized.

Copy link
Owner Author

howie6879 commented Dec 5, 2017

@jiangxin123

SANIC_NO_UVLOOP=true SANIC_NO_UJSON=true pip install sanic

https://github.com/channelcat/sanic

@cjwddtc

This comment has been minimized.

Copy link

cjwddtc commented Feb 11, 2018

网站部署到服务器上面之后和https://www.owllook.net/上面的不一样
举个例子搜索超级修真保镖搜到的源不一样而且会少很多如何解决

@howie6879

This comment has been minimized.

Copy link
Owner Author

howie6879 commented Feb 11, 2018

你好 @cjwddtc
你可以去掉缓存重新搜索一次 因为每次搜索结果会变 或者你也可以直接换搜索源 360的源不错

@cjwddtc

This comment has been minimized.

Copy link

cjwddtc commented Feb 11, 2018

如何清空缓存 @howie6879

@howie6879

This comment has been minimized.

Copy link
Owner Author

howie6879 commented Feb 11, 2018

@cjwddtc 去redis清除

@cjwddtc

This comment has been minimized.

Copy link

cjwddtc commented Feb 11, 2018

我阅读了一下源代码,为什么不把4个搜索引擎的结果合并到一起呢? @howie6879

@howie6879

This comment has been minimized.

Copy link
Owner Author

howie6879 commented Feb 11, 2018

@cjwddtc 没必要 一个就够了

@wxlost

This comment has been minimized.

Copy link

wxlost commented Feb 19, 2018

首页搜索小说正常.点排行榜没有数据.....请问在哪里看错误日志

@howie6879

This comment has been minimized.

Copy link
Owner Author

howie6879 commented Feb 20, 2018

@wxlost 需要运行owllook/spiders/spider_console.py,目前代码中是设定60分钟运行一次,运行的时候请酌情更改

@wxlost

This comment has been minimized.

Copy link

wxlost commented Feb 20, 2018

[root@xenserver cloudreve]# cd /home/owllook/owllook/spiders/
[root@xenserver spiders]# python3 spider_console.py
Traceback (most recent call last):
File "spider_console.py", line 10, in
from owllook.spiders import QidianRankingSpider, BdNovelSpider
File "../../owllook/spiders/init.py", line 1, in
from .qidian_ranking import QidianRankingSpider
File "../../owllook/spiders/qidian_ranking.py", line 9, in
from owllook.database.mongodb import MotorBaseOld
File "../../owllook/database/mongodb/init.py", line 1, in
from .motorbase import MotorBase, MotorBaseOld
File "../../owllook/database/mongodb/motorbase.py", line 8, in
@singleton
File "../../owllook/database/mongodb/motorbase.py", line 16, in MotorBase
MONGODB = CONFIG.MONGODB
AttributeError: type object 'Config' has no attribute 'MONGODB'

@wxlost

This comment has been minimized.

Copy link

wxlost commented Feb 20, 2018

@howie6879 直接启动报错.MONGODB要强制设置用户名和密码吗?

@BlueRagdoll

This comment has been minimized.

Copy link

BlueRagdoll commented Feb 25, 2018

@wxlost 我也是这个问题,请问你的解决了吗?

@howie6879

This comment has been minimized.

Copy link
Owner Author

howie6879 commented Feb 25, 2018

os.environ['MODE'] = 'PRO' 这段代码请先注释掉

@BlueRagdoll

This comment has been minimized.

Copy link

BlueRagdoll commented Feb 26, 2018

@howie6879 谢谢你了,刚看到你发的邮件,抱歉。

@317482454

This comment has been minimized.

Copy link

317482454 commented Jun 15, 2018

打扰了问个问题:
网站能正常运行,但是无法登陆,也不报错
我是否需要设置这里:
MONGODB = dict(
MONGO_HOST=os.getenv('MONGO_HOST', ""),
MONGO_PORT=os.getenv('MONGO_PORT', 27017),
MONGO_USERNAME=os.getenv('MONGO_USERNAME', ""),
MONGO_PASSWORD=os.getenv('MONGO_PASSWORD', ""),
DATABASE='owllook',
)

@howie6879

This comment has been minimized.

Copy link
Owner Author

howie6879 commented Jun 15, 2018

@317482454 注册成功了么?

@317482454

This comment has been minimized.

Copy link

317482454 commented Jun 15, 2018

@howie6879 注册直接跳转到主页,控制台输出:
2018-06-15 01:36:56,095 INFO root(55) | user ip is: None

@howie6879

This comment has been minimized.

Copy link
Owner Author

howie6879 commented Jun 15, 2018

@317482454 config文件 VAL_HOST = os.getenv('VAL_HOST', 'true') 改为VAL_HOST = 0 再试试

@317482454

This comment has been minimized.

Copy link

317482454 commented Jun 15, 2018

@howie6879 排行榜出现test,但是还无法登陆跟注册,会不会是因为没连接上mongo?

如果连接不上mongo会出现什么错误?

但是我去mongo里面看到了创建了owllook

@howie6879

This comment has been minimized.

Copy link
Owner Author

howie6879 commented Jun 15, 2018

@317482454
1.是否注册成功?注册成功mogo有user表
2.无法登陆报什么错?
无法描述可截图

@317482454

This comment has been minimized.

Copy link

317482454 commented Jun 15, 2018

1.注册成功了,user表有数据
image

2.没有报错,就是没反应

查看了login返回:status:0

image

@howie6879

This comment has been minimized.

Copy link
Owner Author

howie6879 commented Jun 15, 2018

image
这说明没获取到你的user以及pass参数 你看看你改动什么了吧,什么class id名字

@317482454

This comment has been minimized.

Copy link

317482454 commented Jun 15, 2018

return self.gen.throw(type, value, traceback)
File "/root/.local/share/virtualenvs/owllook-kjRL-ddR/lib/python3.6/site-packages/sanic/app.py", line 727, in _run_response_middleware
_response = await _response
File "/usr/local/lib/python3.6/asyncio/coroutines.py", line 129, in throw
return self.gen.throw(type, value, traceback)
File "server.py", line 75, in save_session
await app.session_interface.save(request, response)
File "/usr/local/lib/python3.6/asyncio/coroutines.py", line 129, in throw
return self.gen.throw(type, value, traceback)
File "/root/.local/share/virtualenvs/owllook-kjRL-ddR/lib/python3.6/site-packages/sanic_session/redis_session_interface.py", line 100, in save
await redis_connection.setex(key, self.expiry, val)
File "/root/.local/share/virtualenvs/owllook-kjRL-ddR/lib/python3.6/site-packages/asyncio_redis/protocol.py", line 676, in wrapper
result = yield from method(protocol_self, _NoTransaction, *a, **kw)
File "/root/.local/share/virtualenvs/owllook-kjRL-ddR/lib/python3.6/site-packages/asyncio_redis/protocol.py", line 1135, in _query
result = yield from self._get_answer(transaction, answer_f, _bypass=_bypass, call=call)
File "/root/.local/share/virtualenvs/owllook-kjRL-ddR/lib/python3.6/site-packages/asyncio_redis/protocol.py", line 1062, in _get_answer
result = yield from answer_f
asyncio_redis.exceptions.ErrorReply: MISCONF Redis is configured to save RDB snapshots, but is currently not able to persist on disk. Commands that may modify the data set are disabled. Please check Redis logs for details about the error.

强制关闭Redis快照导致不能持久化:
config set stop-writes-on-bgsave-error no

附上解决办法

@zhuhangyu

This comment has been minimized.

Copy link

zhuhangyu commented Jul 25, 2018

我用docker运行的,为啥访问8001端口,就直接被302跳转到你的官网了呢?

@zhuhangyu

This comment has been minimized.

Copy link

zhuhangyu commented Jul 25, 2018

2018-07-25 15:38:19,642 ERROR aiocache.log(126) | Unexpected error with RedisCache (172.17.0.1:6379)
Traceback (most recent call last):
File "/opt/owllook/fetcher/decorators.py", line 122, in wrapper
if await cache_instance.exists(cache_key):
File "/root/.local/share/virtualenvs/opt-zvmYt2-H/lib/python3.6/site-packages/aiocache/cache.py", line 41, in _enabled
return await fn(*args, **kwargs)
File "/root/.local/share/virtualenvs/opt-zvmYt2-H/lib/python3.6/site-packages/aiocache/cache.py", line 31, in _timeout
return await asyncio.wait_for(fn(self, *args, **kwargs), self.timeout)
File "/usr/local/lib/python3.6/asyncio/tasks.py", line 362, in wait_for
raise futures.TimeoutError()
concurrent.futures._base.TimeoutError

浏览器一直在转圈

@howie6879

This comment has been minimized.

Copy link
Owner Author

howie6879 commented Jul 25, 2018

@zhuhangyu mongodb 以及redis需要外网访问,你这个连接上了么?

@zhuhangyu

This comment has been minimized.

Copy link

zhuhangyu commented Jul 25, 2018

前面是防火墙问题,现在连上了,但是报权限错误
aioredis.errors.ReplyError: ERR invalid password
pymongo.errors.OperationFailure: Authentication failed.

可我用命令行登录redis和mongo,密码没错的

@howie6879

This comment has been minimized.

Copy link
Owner Author

howie6879 commented Jul 25, 2018

@zhuhangyu 应该是密码错了 你看看哪里输错没

@zhuhangyu

This comment has been minimized.

Copy link

zhuhangyu commented Jul 25, 2018

我把REDIS_PASSWORD='eNIuP'改为REDIS_PASSWORD=eNIuP
去掉单引号,redis就不报错

但是mongo去掉单引号,仍然报错

@zhuhangyu

This comment has been minimized.

Copy link

zhuhangyu commented Jul 25, 2018

mongo -u root -p --authenticationDatabase admin --host 172.17.0.1

MongoDB shell version: 3.2.20
Enter password:
connecting to: 172.17.0.1:27017/test

use admin
switched to db admin
show dbs
admin 0.000GB
local 0.000GB

mongo的用户名root,用命令行可以登录的

@howie6879

This comment has been minimized.

Copy link
Owner Author

howie6879 commented Jul 25, 2018

@zhuhangyu

MONGODB = dict(
        MONGO_HOST=os.getenv('MONGO_HOST', ""),
        MONGO_PORT=os.getenv('MONGO_PORT', 27017),
        MONGO_USERNAME=os.getenv('MONGO_USERNAME', ""),
        MONGO_PASSWORD=os.getenv('MONGO_PASSWORD', ""),
        DATABASE='owllook',
    )

你可以本地写个操作mongo的代码,用你的用户名密码连接操作集合owllook,连接代码可参考database/mongodb

@zhuhangyu

This comment has been minimized.

Copy link

zhuhangyu commented Jul 25, 2018

搞定了
1、dev_owllook.env里面的值不能用单引号括起来
2、之前用root账号,只能对admin数据库有权限,后来我另建了一个账号

@howie6879

This comment has been minimized.

Copy link
Owner Author

howie6879 commented Jul 25, 2018

@zhuhangyu 玩得愉快

@zhuhangyu

This comment has been minimized.

Copy link

zhuhangyu commented Jul 25, 2018

自己喜欢看书,没广告,舒服了。谢谢

@zhuhangyu

This comment has been minimized.

Copy link

zhuhangyu commented Jul 25, 2018

@howie6879 再请问,我点搜索排行,不管是本站的榜单,还是起点或纵横的榜单,都是空的,没有数据。我看你的官网是有数据的。

@howie6879

This comment has been minimized.

Copy link
Owner Author

howie6879 commented Jul 26, 2018

@zhuhangyu 看readme

@zhuhangyu

This comment has been minimized.

Copy link

zhuhangyu commented Jul 26, 2018

看了的
root@344e6905ac50:/opt/owllook/spiders# pipenv shell
Launching subshell in virtual environment…
root@344e6905ac50:/opt# . /root/.local/share/virtualenvs/opt-zvmYt2-H/bin/activate
(opt-zvmYt2-H) root@344e6905ac50:/opt# cd owllook/spiders/
(opt-zvmYt2-H) root@344e6905ac50:/opt/owllook/spiders# python spider_console.py
[2018-07-26 09:38:51,634] 218-INFO init::load_config():l27: owllook 启动模式:DEV

没报错,但还没数据

@zhang0peter

This comment has been minimized.

Copy link

zhang0peter commented Jul 27, 2018

大佬,你能不能用docker-compose,然后把mongdb和redis都放在docker中启动,这样我们就可以直接用了

@hsyysy

This comment has been minimized.

Copy link

hsyysy commented Sep 10, 2018

我有一个想法,仅供参考。
目前的分类机制是对于一个书名,有不同的提供网站。
但实际上每本书都有一个或几个官方发布网站,网站可以抓取到书的目录和每一章的前两段,那么
1.通过网友的共同劳动,像Wiki一样建立一个书名与官方发布页面地址的映射目录。
2.抓取官方发布的目录建立书籍的目录,然后通过每章前两段内容与其他网站内容的比对抽取完整章节。
而不用显示多个源。

可能存在以下问题
1.抓取官方目录和前两段可能涉嫌侵权。
2.文字对比可能正确率低。

@howie6879

This comment has been minimized.

Copy link
Owner Author

howie6879 commented Sep 10, 2018

@hsyysy 这样做我觉得还不如目前方便,而且目前还不用抓取章节内容,目前以搜索为驱动至少不会侵权

@DLDLDLDLDLDL

This comment has been minimized.

Copy link

DLDLDLDLDLDL commented Sep 22, 2018

你好,我是一名在校大学生,我想学python web 开发,请问有什么推荐的书吗

@howie6879

This comment has been minimized.

Copy link
Owner Author

howie6879 commented Sep 22, 2018

你好,我是一名在校大学生,我想学python web 开发,请问有什么推荐的书吗

@DLDLDLDLDLDL

可以阅读这个教程即可:
https://www.liaoxuefeng.com/wiki/0014316089557264a6b348958f449949df42a6d3a2e542c000

语法掌握好之后就可以玩flask 等进行web开发

@bosen365

This comment has been minimized.

Copy link

bosen365 commented Sep 28, 2018

@zhuhangyu 在.env 加一行 VAL_HOST=0

为何我添加了这一行,还是一样的302跳转呢,当前最新的代码编译运行的

@pppploi8

This comment has been minimized.

Copy link

pppploi8 commented Oct 3, 2018

bootcss的cdn地址已经更换了,作者不考虑更新下么,没部署直接用owllook页面样式都挂了好几天了

@howie6879

This comment has been minimized.

Copy link
Owner Author

howie6879 commented Oct 3, 2018

@jocking

This comment has been minimized.

Copy link

jocking commented Nov 12, 2018

@zhuhangyu 请问你的那个问题给解决了吗?

@jocking

This comment has been minimized.

Copy link

jocking commented Nov 12, 2018

@zhuhangyu
image 就是这个问题 排行榜没有数据

@zhuhangyu

This comment has been minimized.

Copy link

zhuhangyu commented Nov 22, 2018

@zhuhangyu
image 就是这个问题 排行榜没有数据

我没解决。不知道是不是查询的数据不够多。反正我就是自己看书使用的,无所谓了。

@zzslion

This comment has been minimized.

Copy link

zzslion commented Jan 17, 2019

@zhuhangyu
image 就是这个问题 排行榜没有数据

我没解决。不知道是不是查询的数据不够多。反正我就是自己看书使用的,无所谓了。

我也是这个问题同求怎么解决

@wooddance

This comment has been minimized.

Copy link

wooddance commented Jan 31, 2019

排行榜的问题似乎是因为程序不兼容最新版本的ruia。

image

使用 pipenv install 安装之后,这里ruia的版本号是0.4.5。然后手动运行python qidian_ranking.py时候,出现错误信息如下:
image

查看排行榜爬虫源码后发现是ruia的接口发生了变化,
image

如果更换ruia版本,如使用pipenv install ruia==0.1.0好像就没有问题了。

@howie6879 希望作者能看一下,谢谢。

@howie6879

This comment has been minimized.

Copy link
Owner Author

howie6879 commented Feb 1, 2019

@wooddance
你好 后面我会修复 如果你紧急需要的话可以尝试自己修复一下 很快的

@terrorist002

This comment has been minimized.

Copy link

terrorist002 commented Feb 6, 2019

请问ssl证书如何安装呢?

@terrorist002

This comment has been minimized.

Copy link

terrorist002 commented Feb 8, 2019

域名需要绑定吗?安装好了后,爬虫也能工作了,就是网站打不开。还是我没设置?

@howie6879 howie6879 closed this Feb 13, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment