[BUG] Whoogle returns a broken Page on search

[bitfl0wer]

Describe the bug
Recently, whoogle searches started returning more and more "broken" Pages on search. After waking up today, i found that my hosted whoogle-instance doesn't return ANY non-broken Pages anymore.

To Reproduce
I don't know if these steps create the problem specifically, but:

1. Set up whoogle on a VPS
2. Set whoogle as your default search engine
3. Search with whoogle every day
4. (Maybe) Encounter the problem

Deployment Method

• Heroku (one-click deploy)
• Docker
• run executable
• pip/pipx
• Other: [describe setup]

Version of Whoogle Search

• Latest build from [source] (i.e. GitHub, Docker Hub, pip, etc)
• Version [0.3.1]
• Not sure

Desktop (please complete the following information):

• OS: [macOS 11 on MacBook Pro 2021 (M1)]
• Browser [Firefox, Brave]
• Version [Firefox 87, Brave 1.22.70]

Additional context
This is what I had in the console of my VPS after waking up:
WARNING:waitress.queue:Task queue depth is 1 WARNING:waitress.queue:Task queue depth is 2 WARNING:waitress.queue:Task queue depth is 1 WARNING:waitress.queue:Task queue depth is 2 WARNING:waitress.queue:Task queue depth is 1 WARNING:waitress.queue:Task queue depth is 1 WARNING:waitress.queue:Task queue depth is 2 WARNING:waitress.queue:Task queue depth is 1 WARNING:waitress.queue:Task queue depth is 2 ERROR:app:Exception on / [GET] Traceback (most recent call last): File "/root/.local/pipx/venvs/whoogle-search/lib/python3.6/site-packages/flask/app.py", line 2446, in wsgi_app response = self.full_dispatch_request() File "/root/.local/pipx/venvs/whoogle-search/lib/python3.6/site-packages/flask/app.py", line 1951, in full_dispatch_request rv = self.handle_user_exception(e) File "/root/.local/pipx/venvs/whoogle-search/lib/python3.6/site-packages/flask/app.py", line 1820, in handle_user_exception reraise(exc_type, exc_value, tb) File "/root/.local/pipx/venvs/whoogle-search/lib/python3.6/site-packages/flask/_compat.py", line 39, in reraise raise value File "/root/.local/pipx/venvs/whoogle-search/lib/python3.6/site-packages/flask/app.py", line 1947, in full_dispatch_request rv = self.preprocess_request() File "/root/.local/pipx/venvs/whoogle-search/lib/python3.6/site-packages/flask/app.py", line 2241, in preprocess_request rv = func() File "/root/.local/pipx/venvs/whoogle-search/lib/python3.6/site-packages/app/routes.py", line 85, in before_request_func config=g.user_config) File "/root/.local/pipx/venvs/whoogle-search/lib/python3.6/site-packages/app/request.py", line 147, in __init__ self.mobile = 'Android' in normal_ua or 'iPhone' in normal_ua TypeError: argument of type 'NoneType' is not iterable WARNING:waitress.queue:Task queue depth is 1

Even after restarting, it still seems to return broken search results that look like this:

[trumad]

I believe you are suffering from #243

[benbusby]

Yes, this is a duplicate of that issue. There's a fix available now on develop if you want to run it manually, otherwise you can also use the beta or buildx-experimental Docker images. If you want to stick with pip, the fix will be available in the next update (hopefully this week or next).