[Bug] Error with retrieving novels from some sources

[k5924]

Source information

LightNovelPub

Steps to reproduce

Scenario 1: Refreshing library:

1. Pull to refresh library

Scenario 2: searching for novel in global search:

1. Search for novel_name in global search section

Expected behavior

On library refresh:

1. Toast message says updating
2. Notified that update is complete

On global search:

1. Novel is searched for in all sources
2. If novel exists display novel else show no results

Actual behavior

On library refresh:

1. Toast message says updating library
2. All toast messages from LightNovelPub says "novel_name: cannot convert null value to object" where novel_name is the actual name of the novel in my library

On global search:

1. Searches for novel
2. Displays "JSON Parse error: Unexpected token: <" in red text

LNReader version

1.1.7

Android version

Android 11

Other details

All other sources appear to be working except some sources like LNMTL displaying "Cannot convert null value to object" when searching for a novel in global search.

Acknowledgements

• I have searched the existing issues and this is a new ticket, NOT a duplicate or related to another open issue.
• I have written a short but informative title.
• I have updated the app to version 1.1.6.
• I have updated all installed extensions.
• If this is an issue with the app itself, I should be opening an issue in the app repository.
• I will fill out all of the requested information in this form.

[Kai-Nui]

T̶r̶a̶v̶i̶s̶T̶r̶a̶n̶s̶l̶a̶t̶i̶o̶n̶s̶ a̶l̶s̶o̶ p̶r̶o̶m̶p̶t̶s̶ "J̶S̶O̶N̶ P̶a̶r̶s̶e̶ e̶r̶r̶o̶r̶: U̶n̶e̶x̶p̶e̶c̶t̶e̶d̶ t̶o̶k̶e̶n̶: e̶" w̶h̶e̶n̶ a̶c̶c̶e̶s̶s̶i̶n̶g̶ a̶c̶t̶u̶a̶l̶ n̶o̶v̶e̶l̶ c̶h̶a̶p̶t̶e̶r̶s̶. U̶s̶i̶n̶g̶ w̶e̶b̶v̶i̶e̶w̶ f̶o̶r̶ t̶h̶e̶ r̶e̶a̶d̶e̶r̶ d̶o̶e̶s̶n̶'t̶ c̶h̶a̶n̶g̶e̶ a̶n̶y̶t̶h̶i̶n̶g̶. T̶h̶e̶ s̶e̶a̶r̶c̶h̶ w̶o̶r̶k̶s̶ f̶i̶n̶e̶ w̶i̶t̶h̶ t̶h̶i̶s̶ s̶o̶u̶r̶c̶e̶ t̶h̶o̶u̶g̶h̶. A̶n̶d̶ u̶s̶i̶n̶g̶ w̶e̶b̶v̶i̶e̶w̶ f̶r̶o̶m̶ n̶o̶v̶e̶l̶ p̶r̶o̶f̶i̶l̶e̶ s̶h̶o̶w̶s̶ t̶h̶a̶t̶ t̶h̶e̶r̶e̶ a̶r̶e̶ c̶h̶a̶p̶t̶e̶r̶s̶ a̶n̶d̶ d̶i̶s̶p̶l̶a̶y̶s̶ t̶h̶e̶m̶ p̶r̶o̶p̶e̶r̶l̶y̶.

Solved during v1.1.8 update

[k5924]

The webview behaviour is similar for LightNovelPub as looking at the webview, there are new chapters but I think the error where it cant convert a null value to object stops the new chapters from being loaded/added to the novel in the app

[Kai-Nui]

F̶o̶x̶a̶h̶o̶l̶i̶c̶ d̶o̶e̶s̶n̶'t̶ c̶o̶n̶n̶e̶c̶t̶ p̶r̶o̶p̶e̶r̶l̶y̶ w̶h̶e̶t̶h̶e̶r̶ i̶n̶ g̶l̶o̶b̶a̶l̶ s̶e̶a̶r̶c̶h̶ o̶r̶ b̶r̶o̶w̶s̶i̶n̶g̶. A̶l̶s̶o̶, l̶i̶n̶k̶s̶ i̶n̶ N̶o̶v̶e̶l̶U̶p̶d̶a̶t̶e̶s̶ t̶h̶a̶t̶ r̶e̶d̶i̶r̶e̶c̶t̶ t̶o̶ f̶o̶x̶a̶h̶o̶l̶i̶c̶ s̶h̶o̶w̶ C̶l̶o̶u̶d̶f̶l̶a̶r̶e̶ b̶r̶o̶w̶s̶e̶r̶ c̶h̶e̶c̶k̶ m̶e̶s̶s̶a̶g̶e̶ i̶n̶s̶t̶e̶a̶d̶ o̶f̶ a̶c̶t̶u̶a̶l̶ c̶h̶a̶p̶t̶e̶r̶ t̶e̶x̶t̶.

Fixed with v1.1.8 update. It's just that some foxaholic novel chapters have extra step of clicking link within post to access actual chapter text so it still doesn't work with the reader. Works with webview though.

[k5924]

I tried to webscrape the page in Python and it looks like cloudflare is displaying a captcha that needs to be solved before the actual page is displayed.

Edit: after looking at how to solve this, either a service like 2captcha to automatically solve the captcha would work but it would require people to pay for the service to solve the captcha on the page. The other way would be that if the page that is retrieved renders a captcha to solve, a webview would open prompting the user to solve it.