Fail to crawl zhihu.com

[pouloghost]

I've tried different mode, model. None can return a correct content. Either hit a safety captcha, or just return one sentence.

[Steven-Zhl]

I have also encountered this problem. Zhihu has adopted strict crawler/automated tool/headless browser detection technology to prevent content from being crawled. (These tools have their own characteristics and can be detected)

To circumvent this detection, you can learn about stealth and integrate it into puppeteer, although this may be somewhat unethical.

I have used playwright and playwright_stealth in Python to solve this problem, which shows that this method is feasible, but I am not familiar with puppeteer and cannot provide you with more detailed help.