Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[BUG] The content of [url] is Empty and content_id =acontentz. #45

Open
Kuan-Lun opened this issue Apr 23, 2024 · 7 comments
Open

[BUG] The content of [url] is Empty and content_id =acontentz. #45

Kuan-Lun opened this issue Apr 23, 2024 · 7 comments
Labels
bug Something isn't working

Comments

@Kuan-Lun
Copy link

Describe the bug（描述这个BUG）
A clear and concise description of what the bug is.

跳出的瀏覽器正常顯示，但錯誤訊息卻顯示為空。

To Reproduce（复现步骤）
复现的代码以及操作（例如分支选择、卷选择等等）

python 檔案

from linovelib2epub import Linovelib2Epub

if __name__ == "__main__":
    linovelib_epub = Linovelib2Epub(
        book_id=2264,
        select_volume_mode=True,
        log_level="DEBUG",
    )
    linovelib_epub.run()

執行

(.venv) PS D:\tmp\linovelib2epub> python .\main.py
2024-04-24,01:52:06 INFO     LinovelibMobileSpider _html_content_id=acontentz              linovelib_mobile_spider.py:33
                    INFO     LinovelibMobileSpider len(_mapping_dict)=104                  linovelib_mobile_spider.py:35
2024-04-24,01:52:07 INFO     LinovelibMobileSpider Succeed to get the novel of book_id:    linovelib_mobile_spider.py:88
                             2264
                    INFO     LinovelibMobileSpider book name:《月光下的异世界之旅》        linovelib_mobile_spider.py:98
                    INFO     LinovelibMobileSpider Succeed to get the catalog of book_id: linovelib_mobile_spider.py:148
                             2264
[?] Which volumes you want to download?(use SPACE to select one or multiple volumes):
 > [X] 第一卷
   [ ] 第二卷
   [ ] 第三卷
   [ ] 第四卷
   [ ] 第五卷
   [ ] 第六卷
   [ ] 第七卷
   [ ] 第八卷
   [ ] 第8.5卷
   [ ] 第九卷
   [ ] 第十卷
   [ ] 第十一卷
   [ ] 第十二卷

2024-04-24,01:52:10 INFO     LinovelibMobileSpider volume: 第一卷                         linovelib_mobile_spider.py:164
                    INFO     LinovelibMobileSpider chapter : 插图                         linovelib_mobile_spider.py:178

DevTools listening on ws://127.0.0.1:64783/devtools/browser/87ee5b70-09c3-4015-a5bf-bc617950e392
2024-04-24,01:52:12 INFO     LinovelibMobileSpider navigator.language.toLowerCase()=zh-tw linovelib_mobile_spider.py:368
2024-04-24,01:52:14 INFO     LinovelibMobileSpider  初始化 Driver 完毕...                 linovelib_mobile_spider.py:379
2024-04-24,01:52:25 WARNING  LinovelibMobileSpider                                        linovelib_mobile_spider.py:306
                             https://www.bilinovel.com/novel/2264/122121.html encountered
                             TimeoutException.
                    WARNING  LinovelibMobileSpider Retrying                               linovelib_mobile_spider.py:327
                             https://www.bilinovel.com/novel/2264/122121.html(1/10)...;
                             retry_interval: 1.76(s)
2024-04-24,01:52:28 DEBUG    LinovelibMobileSpider                                        linovelib_mobile_spider.py:202
                             page(https://www.bilinovel.com/novel/2264/122121.html)
                             size=17875
                    INFO     LinovelibMobileSpider chapter : [插图] New Title= [插圖]     linovelib_mobile_spider.py:210
                    CRITICAL LinovelibMobileSpider The content of                         linovelib_mobile_spider.py:224
                             https://www.bilinovel.com/novel/2264/122121.html is Empty
                             and content_id =acontentz.Please report this bug to [github
                             issue](https://github.com/lightnovel-center/linovelib2epub/i
                             ssues).

log 檔

2024-04-24,01:57:13 INFO     LinovelibMobileSpider  linovelib_mobile_spider.py:33    _html_content_id=acontentz
2024-04-24,01:57:13 INFO     LinovelibMobileSpider  linovelib_mobile_spider.py:35    len(_mapping_dict)=104
2024-04-24,01:57:14 INFO     LinovelibMobileSpider  linovelib_mobile_spider.py:88    Succeed to get the novel of book_id: 2264
2024-04-24,01:57:14 INFO     LinovelibMobileSpider  linovelib_mobile_spider.py:98    book name:《月光下的异世界之旅》
2024-04-24,01:57:14 INFO     LinovelibMobileSpider  linovelib_mobile_spider.py:148   Succeed to get the catalog of book_id: 2264
2024-04-24,01:57:17 INFO     LinovelibMobileSpider  linovelib_mobile_spider.py:164   volume: 第一卷
2024-04-24,01:57:17 INFO     LinovelibMobileSpider  linovelib_mobile_spider.py:178   chapter : 插图
2024-04-24,01:57:19 INFO     LinovelibMobileSpider  linovelib_mobile_spider.py:368   navigator.language.toLowerCase()=zh-tw
2024-04-24,01:57:23 INFO     LinovelibMobileSpider  linovelib_mobile_spider.py:379    初始化 Driver 完毕...
2024-04-24,01:57:25 DEBUG    LinovelibMobileSpider  linovelib_mobile_spider.py:202   page(https://www.bilinovel.com/novel/2264/122121.html) size=16861
2024-04-24,01:57:25 INFO     LinovelibMobileSpider  linovelib_mobile_spider.py:210   chapter : [插图] New Title= [插圖]
2024-04-24,01:57:25 CRITICAL LinovelibMobileSpider  linovelib_mobile_spider.py:224   The content of https://www.bilinovel.com/novel/2264/122121.html is Empty and content_id =acontentz.Please report this bug to [github issue](https://github.com/lightnovel-center/linovelib2epub/issues).

Expected behavior（期望的行为）
A clear and concise description of what you expected to happen.

可以正常抓取內容

Screenshots or Video（截图或者视频录制）
If applicable, add screenshots to help explain your problem.

錯誤前有正常顯示網頁

linovelib2epub-20240424-0216.mp4

Environment（软件环境）

  • 网络环境: 台灣
  • OS Verison: Windows 11 23H2 專業版
  • Python Version: 3.12
  • Pip Version: pip 24.0 from D:\tmp\linovelib2epub.venv\Lib\site-packages\pip (python 3.12)
  • Python packages:
Package            Version     Editable project location
------------------ ----------- -------------------------
aiofiles           23.2.1
aiohttp            3.9.5
aiosignal          1.3.1
ansicon            1.89.0
attrs              23.2.0
beautifulsoup4     4.12.3
blessed            1.20.0
Brotli             1.1.0
bs4                0.0.2
certifi            2024.2.2
cffi               1.16.0
charset-normalizer 3.3.2
click              8.1.7
colorama           0.4.6
cssselect          1.2.0
DataRecorder       3.4.14
demjson3           3.0.6
DownloadKit        2.0.0
DrissionPage       4.0.4.21
dynaconf           3.2.5
EbookLib           0.18
editor             1.6.6
esprima            4.0.1
et-xmlfile         1.1.0
fake-useragent     1.5.1
filelock           3.13.4
frozenlist         1.4.1
h11                0.14.0
idna               3.7
inquirer           3.2.4
jinxed             1.2.1
linovelib2epub     0.1.3       D:\tmp\linovelib2epub
lxml               5.2.1
markdown-it-py     3.0.0
mdurl              0.1.2
multidict          6.0.5
openpyxl           3.1.2
outcome            1.3.0.post0
pillow             10.3.0
pip                24.0
psutil             5.9.8
pycparser          2.22
Pygments           2.17.2
PySocks            1.7.1
readchar           4.0.6
requests           2.31.0
requests-file      2.0.0
rich               13.7.1
runs               1.2.2
selenium           4.19.0
setuptools         69.5.1
six                1.16.0
sniffio            1.3.1
sortedcontainers   2.4.0
soupsieve          2.5
tabulate           0.9.0
tldextract         5.1.2
trio               0.25.0
trio-websocket     0.11.1
typing_extensions  4.11.0
urllib3            2.2.1
uuid               1.30
wcwidth            0.2.13
websocket-client   1.7.0
wsproto            1.2.0
xmod               1.8.1
yarl               1.9.4
@Kuan-Lun Kuan-Lun added the bug Something isn't working label Apr 23, 2024
@wdpm
Copy link
Member

wdpm commented Apr 23, 2024 •
edited

你discord还在线吗？

@wdpm
Copy link
Member

wdpm commented Apr 23, 2024 •
edited

注意查看文档的更新，https://github.com/lightnovel-center/linovelib2epub?tab=readme-ov-file#linovelibmobile

现在繁体版必须显式指定 target 来进行请求。

from linovelib2epub import Linovelib2Epub, TargetSite

if __name__ == "__main__":
    linovelib_epub = Linovelib2Epub(
        book_id=2264,
        select_volume_mode=True,
        log_level="DEBUG",
        target_site=TargetSite.LINOVELIB_MOBILE_TRADITIONAL,
        # 是否挂代理取决于你的网络环境，我这边需要挂代理才能正常访问 bilinovel 的繁体版网站
        # 你那边应该是不需要的，可以注释掉下面这行
        # disable_proxy=False
    )
    linovelib_epub.run()

截图
image

@Kuan-Lun
Copy link
Author

我的錯，添加 target_site 可以正常運作。

要不要考慮把 target_site 改成一定要輸入的 input? 這也許能防呆。

@wdpm
Copy link
Member

wdpm commented Apr 25, 2024

防呆和方便不可兼得。

因为默认请求的目标是简体版网站，所以不需要指定。而需要繁体版的需要显式指定。
如果将这个参数改成必须显式指定，那么简体版的用户（例如我自己）在使用时就必须多打一个参数。

默认目标只有一个，而网站版本有两个，因此不可能同时满足两个版本的用户。

我不同意防呆设计，强制参数的话，使用文档中options那个表，如果全部都是防呆的强制参数，这个软件我自己都不会去用，太麻烦了。我之所以把使用的API设计的非常简单，就是因为我就想简单运行，按需调整。

这个设计也符合一般软件的设计原则，当默认设置不能满足你时，修改设置的主动权在于你。

@Kuan-Lun
Copy link
Author

或許可以考慮在錯誤訊息中提及 target_site 設定？因為他要求的是回報 issue，而不是更改 target_site。這樣的話也是一種防呆，只是在執行時才防。

 CRITICAL LinovelibMobileSpider The content of                         linovelib_mobile_spider.py:224
                             https://www.bilinovel.com/novel/2264/122121.html is Empty
                             and content_id =acontentz.Please report this bug to [github
                             issue](https://github.com/lightnovel-center/linovelib2epub/i
                             ssues).

@wdpm
Copy link
Member

wdpm commented Apr 25, 2024 •
edited

可以，后续我会改下这个情况出现时的文本描述。发生这个issue是存在多个可能的原因的：

  1. linovelib网站更新源码了，结构改变，这个提示是为了上报变更，后续我需要更新代码来适配。
  2. 用户可能处于非中国大陆地区的网络ip环境，但是想要请求繁体版网站，却没有指定target site。

这两种情况都会造成无法正常抓取正文内容。我会尽量在log中就近给出提示。因为最好的文档肯定也不是readme，而是近在咫尺的log提示。

感谢你的反馈和建议。

@wdpm
Copy link
Member

wdpm commented Apr 25, 2024

image
修改了提示。

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@wdpm @Kuan-Lun