[Bug] 1.8.1 Fails to Scan, URL malformed

[jshackles]

RomM version
1.8.1

Describe the bug
RomM is unable to scan. An error is thrown in the log.

To Reproduce
Steps to reproduce the behavior:

1. Install RomM
2. Push Scan
3. Receive error in log

Expected behavior
for the rom file or directory to scan normally and be added to the database

Additional context
I even tried a fresh database, and got the same error. It may be related to the rom file itself?

Log:

INFO: [RomM][2023-04-28 15:15:32] 🔎 Scanning
INFO: [RomM][2023-04-28 15:15:36] Platforms to be scanned: ps2
INFO: [RomM][2023-04-28 15:15:36] 🎮 ps2
INFO: [RomM][2023-04-28 15:15:37] Identified as PlayStation 2
INFO: [RomM][2023-04-28 15:15:38] Scanning Aqua Aqua.chd
INFO: - "GET /scan?platforms=[]&complete_rescan=false HTTP/1.0" 500 Internal Server Error
ERROR: Exception in ASGI application
Traceback (most recent call last):
File "/usr/local/lib/python3.10/dist-packages/uvicorn/protocols/http/h11_impl.py", line 407, in run_asgi
result = await app( # type: ignore[func-returns-value]
File "/usr/local/lib/python3.10/dist-packages/uvicorn/middleware/proxy_headers.py", line 78, in call
return await self.app(scope, receive, send)
File "/usr/local/lib/python3.10/dist-packages/fastapi/applications.py", line 271, in call
await super().call(scope, receive, send)
File "/usr/local/lib/python3.10/dist-packages/starlette/applications.py", line 118, in call
await self.middleware_stack(scope, receive, send)
File "/usr/local/lib/python3.10/dist-packages/starlette/middleware/errors.py", line 184, in call
raise exc
File "/usr/local/lib/python3.10/dist-packages/starlette/middleware/errors.py", line 162, in call
await self.app(scope, receive, _send)
File "/usr/local/lib/python3.10/dist-packages/starlette/middleware/cors.py", line 84, in call
await self.app(scope, receive, send)
File "/usr/local/lib/python3.10/dist-packages/starlette/middleware/exceptions.py", line 79, in call
raise exc
File "/usr/local/lib/python3.10/dist-packages/starlette/middleware/exceptions.py", line 68, in call
await self.app(scope, receive, sender)
File "/usr/local/lib/python3.10/dist-packages/fastapi/middleware/asyncexitstack.py", line 21, in call
raise e
File "/usr/local/lib/python3.10/dist-packages/fastapi/middleware/asyncexitstack.py", line 18, in call
await self.app(scope, receive, send)
File "/usr/local/lib/python3.10/dist-packages/starlette/routing.py", line 706, in call
await route.handle(scope, receive, send)
File "/usr/local/lib/python3.10/dist-packages/starlette/routing.py", line 276, in handle
await self.app(scope, receive, send)
File "/usr/local/lib/python3.10/dist-packages/starlette/routing.py", line 66, in app
response = await func(request)
File "/usr/local/lib/python3.10/dist-packages/fastapi/routing.py", line 237, in app
raw_response = await run_endpoint_function(
File "/usr/local/lib/python3.10/dist-packages/fastapi/routing.py", line 165, in run_endpoint_function
return await run_in_threadpool(dependant.call, **values)
File "/usr/local/lib/python3.10/dist-packages/starlette/concurrency.py", line 41, in run_in_threadpool
return await anyio.to_thread.run_sync(func, *args)
File "/usr/local/lib/python3.10/dist-packages/anyio/to_thread.py", line 31, in run_sync
return await get_asynclib().run_sync_in_worker_thread(
File "/usr/local/lib/python3.10/dist-packages/anyio/_backends/_asyncio.py", line 937, in run_sync_in_worker_thread
return await future
File "/usr/local/lib/python3.10/dist-packages/anyio/_backends/_asyncio.py", line 867, in run
result = context.run(func, *args)
File "/back/./endpoints/scan.py", line 50, in scan
scanned_rom: Rom = fastapi.scan_rom(scanned_platform, rom)
File "/back/./utils/fastapi.py", line 39, in scan_rom
rom_attrs.update(fs.get_screenshots(platform.slug, rom_attrs['file_name'], rom_attrs['url_screenshots']))
File "/back/./utils/fs.py", line 100, in get_screenshots
_store_screenshot(p_slug, file_name, url, idx)
File "/back/./utils/fs.py", line 79, in _store_screenshot
res = requests.get(url, stream=True)
File "/usr/local/lib/python3.10/dist-packages/requests/api.py", line 73, in get
return request("get", url, params=params, **kwargs)
File "/usr/local/lib/python3.10/dist-packages/requests/api.py", line 59, in request
return session.request(method=method, url=url, **kwargs)
File "/usr/local/lib/python3.10/dist-packages/requests/sessions.py", line 573, in request
prep = self.prepare_request(req)
File "/usr/local/lib/python3.10/dist-packages/requests/sessions.py", line 484, in prepare_request
p.prepare(
File "/usr/local/lib/python3.10/dist-packages/requests/models.py", line 368, in prepare
self.prepare_url(url, params)
File "/usr/local/lib/python3.10/dist-packages/requests/models.py", line 445, in prepare_url
raise InvalidURL(f"Invalid URL {url!r}: No host supplied")
requests.exceptions.InvalidURL: Invalid URL 'https:https://images.igdb.com/igdb/image/upload/t_original/scknvu.jpg': No host supplied

It looks like a bug in pulling the screenshot for the game. The URL looks malformed with "https:" at the beginning twice. Attempting to re-scan results in the same error in the logs.

[zurdi15]

Hello,

It seems that the IGDB API is returning a bad screenshot url property for that specific game. Can you try to exclude it with the config and try to rescan?

[jshackles]

That indeed resolved the issue. I guess since the data returned by the API may be malformed, you may want to have it run through some validation so that it can fail silently without crashing the scan process.

Cheers!

[zurdi15]

Yeah totally needed. I didn't thought about this scenario. I will remain this open until I make that verification.

Thank you for the feedback :)

[zurdi15]

Fixed in v1.8.2

Now every url from covers and screenshots are normalized