[fedora 36] error downloading http://mirror.in2p3.fr/pub/fedora/linux/releases/36/Everything/x86_64/os/Packages/d/device-mapper-libs-1.02.175-7.fc36.x86_64

[4383]

Hey there,

I initially faced issues with osbuild-composer where all my build failed without infos in logs (osbuild/osbuild-composer#2224).

By digging in journalctl I can see that each time I ran build I faced the following error:

juin 07 10:57:28 fedora python3[114996]: detected unhandled Python exception in '/usr/bin/osbuild'
juin 07 10:57:28 fedora osbuild-worker[114996]: Traceback (most recent call last):
juin 07 10:57:28 fedora osbuild-worker[114996]:   File "/usr/bin/osbuild", line 33, in <module>
juin 07 10:57:28 fedora osbuild-worker[114996]:     sys.exit(load_entry_point('osbuild==83', 'console_scripts', 'osbuild')())
juin 07 10:57:28 fedora osbuild-worker[114996]:   File "/usr/lib/python3.10/site-packages/osbuild/main_cli.py", line 163, in osbuild_cli
juin 07 10:57:28 fedora osbuild-worker[114996]:     manifest.download(object_store, monitor, args.libdir)
juin 07 10:57:28 fedora osbuild-worker[114996]:   File "/usr/lib/python3.10/site-packages/osbuild/pipeline.py", line 409, in download
juin 07 10:57:28 fedora osbuild-worker[114996]:     source.download(mgr, store, libdir)
juin 07 10:57:28 fedora osbuild-worker[114996]:   File "/usr/lib/python3.10/site-packages/osbuild/sources.py", line 41, in download
juin 07 10:57:28 fedora osbuild-worker[114996]:     reply = client.call_with_fds("download", args, fds)
juin 07 10:57:28 fedora osbuild-worker[114996]:   File "/usr/lib/python3.10/site-packages/osbuild/host.py", line 384, in call_with_fds
juin 07 10:57:28 fedora osbuild-worker[114996]:     raise error
juin 07 10:57:28 fedora osbuild-worker[114996]: osbuild.host.RemoteError: RuntimeError: curl: error downloading http://mirror.in2p3.fr/pub/fedora/linux/releases/36/Everything/x86_64/os/Packages/d/device-mapper-libs-1.02.175-7.fc36.x86_64.>
juin 07 10:57:28 fedora osbuild-worker[114996]:    File "/usr/lib/python3.10/site-packages/osbuild/host.py", line 268, in serve
juin 07 10:57:28 fedora osbuild-worker[114996]:     reply, reply_fds = self._handle_message(msg, fds)
juin 07 10:57:28 fedora osbuild-worker[114996]:   File "/usr/lib/python3.10/site-packages/osbuild/host.py", line 301, in _handle_message
juin 07 10:57:28 fedora osbuild-worker[114996]:     ret, fds = self.dispatch(name, args, fds)
juin 07 10:57:28 fedora osbuild-worker[114996]:   File "/usr/lib/python3.10/site-packages/osbuild/sources.py", line 106, in dispatch
juin 07 10:57:28 fedora osbuild-worker[114996]:     self.download(SourceService.load_items(fds))
juin 07 10:57:28 fedora osbuild-worker[114996]:   File "/usr/lib/python3.10/site-packages/osbuild/sources.py", line 82, in download
juin 07 10:57:28 fedora osbuild-worker[114996]:     for _ in executor.map(self.fetch_one, *zip(*transformed)):
juin 07 10:57:28 fedora osbuild-worker[114996]:   File "/usr/lib64/python3.10/concurrent/futures/_base.py", line 621, in result_iterator
juin 07 10:57:28 fedora osbuild-worker[114996]:     yield _result_or_cancel(fs.pop())
juin 07 10:57:28 fedora osbuild-worker[114996]:   File "/usr/lib64/python3.10/concurrent/futures/_base.py", line 319, in _result_or_cancel
juin 07 10:57:28 fedora osbuild-worker[114996]:     return fut.result(timeout)
juin 07 10:57:28 fedora osbuild-worker[114996]:   File "/usr/lib64/python3.10/concurrent/futures/_base.py", line 451, in result
juin 07 10:57:28 fedora osbuild-worker[114996]:     return self.__get_result()
juin 07 10:57:28 fedora osbuild-worker[114996]:   File "/usr/lib64/python3.10/concurrent/futures/_base.py", line 403, in __get_result
juin 07 10:57:28 fedora osbuild-worker[114996]:     raise self._exception
juin 07 10:57:28 fedora osbuild-worker[114996]:   File "/usr/lib64/python3.10/concurrent/futures/thread.py", line 58, in run
juin 07 10:57:28 fedora osbuild-worker[114996]:     result = self.fn(*self.args, **self.kwargs)
juin 07 10:57:28 fedora osbuild-worker[114996]:   File "/usr/lib/osbuild/sources/org.osbuild.curl", line 158, in fetch_one
juin 07 10:57:28 fedora osbuild-worker[114996]:     raise RuntimeError(f"curl: error downloading {url}: error code {return_code}")
juin 07 10:57:28 fedora osbuild-worker[26636]: time="2023-06-07T10:57:28+02:00" level=error msg="osbuild job failed: osbuild build failed" jobId=572c090f-1b37-455c-a6c0-6a27b813e6d0
juin 07 10:57:28 fedora osbuild-worker[26636]: time="2023-06-07T10:57:28+02:00" level=warning msg="Job '572c090f-1b37-455c-a6c0-6a27b813e6d0' (osbuild) failed: error decoding osbuild output: unexpected end of JSON input\nthe raw output:\n"

My builds failed because osbuild/host.py tried to pull from a mirror that doesn't exist: http://mirror.in2p3.fr/pub/fedora/linux/releases/36/Everything/x86_64/os/Packages/d/device-mapper-libs-1.02.175-7.fc36.x86_64

Indeed I'm using fedora 36 and this mirror seems to be a bit outdated and the last fedora version available is fedora 34 (http://mirror.in2p3.fr/linux.html).

However, I didn't find trace of how osbuilder determine this url.
Do you know how to fix that and do you know where/how is determined this url?

Thanks in advance

[thozza]

Closing on favor of osbuild/osbuild-composer#2224 since there is nothing that osbuild can do if the URL in manifest is not reachable.