Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Failed to load signature: Need more input #6032

Open
shenki opened this issue Jul 27, 2023 · 72 comments
Open

Failed to load signature: Need more input #6032

shenki opened this issue Jul 27, 2023 · 72 comments
Labels

Comments

@shenki
Copy link

shenki commented Jul 27, 2023

Describe the bug
fwupdmgr refresh downloads, but then reports Failed to load signature: Need more input.

Steps to Reproduce
fwupdmgr refresh

fwupd version information

compile   org.freedesktop.fwupd         1.9.3
compile   com.hughsie.libxmlb           0.3.10
compile   com.hughsie.libjcat           0.1.9
runtime   org.freedesktop.fwupd-efi     1.4
compile   org.freedesktop.gusb          0.4.5
runtime   com.dell.libsmbios            2.4
runtime   org.freedesktop.gusb          0.4.5
runtime   org.kernel                    6.3.0-1-amd64
runtime   org.freedesktop.fwupd         1.9.3

Debian testing, installed from apt. Fresh install, first boot. Had to disable secure boot to install.

**fwupd device information**

Please provide the output of the fwupd devices recognized in your system.

LENOVO 21CCS29201
│
├─Unknown Device:
│     Device ID:          6b5a06645e27c4e94b6e67d5185c46d11ec9ad67
│     GUID:               c96751f4-16e8-59ea-975d-69d3dac88d7d ← GPIO\ID_INTC1055:00
│   
├─12th Gen Intel Core™ i7-1270P:
│ │   Device ID:          4bde70ba4e39b28f9eab1628f9dd6e6244c03027
│ │   Current version:    0x0000042a
│ │   Vendor:             Intel
│ │   GUIDs:              b9a2dd81-159e-5537-a7db-e7101d164d3f ← cpu
│ │                       30249f37-d140-5d3e-9319-186b1bd5cac3 ← CPUID\PRO_0&FAM_06
│ │                       ab855c04-4ff6-54af-8a8a-d8193daa0cd8 ← CPUID\PRO_0&FAM_06&MOD_9A
│ │                       3ebbde86-d03e-549a-a8fd-02ebf9aa537a ← CPUID\PRO_0&FAM_06&MOD_9A&STP_3
│ │   Device Flags:       • Internal device
│ │ 
│ ├─USB4 host controller:
│ │     Device ID:        40a03cdbee3f958a36cd1f3b6fd9469b52aee4b8
│ │     Summary:          Unmatched performance for high-speed I/O
│ │     Vendor:           INTEL (THUNDERBOLT:0x8087)
│ │     GUIDs:            5041b15d-6e1c-5681-86b3-bb1d56993357 ← THUNDERBOLT\VEN_8087&DEV_463E
│ │                       e72e778e-94f7-5ed2-b560-1c1262ee217c ← TBT-fixed
│ │     Device Flags:     • Internal device
│ │                       • System requires external power source
│ │                       • Signed Payload
│ │   
│ └─USB4 host controller:
│       Device ID:        94aa454d747cacb16437c732155d773e40539e13
│       Summary:          Unmatched performance for high-speed I/O
│       Vendor:           INTEL (THUNDERBOLT:0x8087)
│       GUIDs:            5041b15d-6e1c-5681-86b3-bb1d56993357 ← THUNDERBOLT\VEN_8087&DEV_463E
│                         e72e778e-94f7-5ed2-b560-1c1262ee217c ← TBT-fixed
│       Device Flags:     • Internal device
│                         • System requires external power source
│                         • Signed Payload
│     
├─Alder Lake-P Integrated Graphics Controller:
│     Device ID:          5792b48846ce271fab11c4a545f7a3df0d36e00a
│     Current version:    0c
│     Vendor:             Intel Corporation (PCI:0x8086)
│     GUIDs:              eaad9970-8e4d-56da-88ab-41a8c1e2811f ← PCI\VEN_8086&DEV_46A6
│                         15bf9dad-22cf-57a8-9ca1-eb3b08e0070e ← PCI\VEN_8086&DEV_46A6&SUBSYS_17AA22E7
│     Device Flags:       • Internal device
│                         • Cryptographic hash verification is available
│   
├─MZVL2512HCJQ-00BL7:
│     Device ID:          04e17fcf7d3de91da49a163ffe4907855c3648be
│     Summary:            NVM Express solid state drive
│     Current version:    CL2QGXA7
│     Vendor:             Samsung (NVME:0x144D)
│     Serial Number:      S64KNF0T707922
│     GUIDs:              bec63ed7-a95f-54fe-b8cc-8e9fee64ba5a ← NVME\VEN_144D&DEV_A80A
│                         310f81b5-6fce-501e-acfb-487d10501e78 ← NVME\VEN_144D&DEV_A80A&SUBSYS_144DA801
│                         af35834a-86e9-5d6e-af3b-78ce4a42cf4a ← SAMSUNG MZVL2512HCJQ-00BL7
│     Device Flags:       • Internal device
│                         • Updatable
│                         • System requires external power source
│                         • Needs a reboot after installation
│                         • Device is usable for the duration of the update
│                         • Signed Payload
│   
├─System Firmware:
│ │   Device ID:          5bfc5b91c18ef8b751d3d052af77b69ebf738038
│ │   Summary:            UEFI ESRT device
│ │   Current version:    65562
│ │   Vendor:             LENOVO (DMI:LENOVO)
│ │   Update State:       Success
│ │   GUIDs:              34d84f45-4685-4019-b7e3-dba67b96ef7d
│ │                       230c8b18-8d9b-53ec-838b-6cfc0383493a ← main-system-firmware
│ │   Device Flags:       • Internal device
│ │                       • Updatable
│ │                       • System requires external power source
│ │                       • Needs a reboot after installation
│ │                       • Cryptographic hash verification is available
│ │                       • Device is usable for the duration of the update
│ │ 
│ ├─AMT [unprovisioned]:
│ │     Device ID:        8d5470e73fd9a31eaa460b2b6aea95483fe3f14c
│ │     Summary:          Hardware and firmware technology for remote out-of-band management
│ │     Current version:  16.0.15.1778
│ │     Bootloader Version:16.0.15.1778
│ │     Vendor:           Intel Corporation (MEI:0x8086)
│ │     GUIDs:            12f80028-b4b7-4b2d-aca8-46e0ff65814c
│ │                       fccad2fe-62ae-5879-b7a9-4ead7bce50f4 ← MEI\VEN_8086&DEV_51E0
│ │                       eae67b2f-2bc2-5c4f-8b82-b1b30ad69fff ← MEI\VEN_8086&DEV_51E0&SUBSYS_17AA22E7
│ │     Device Flags:     • Internal device
│ │   
│ ├─BootGuard Configuration:
│ │     Device ID:        b0d4430dfa6bde9f0c22680df36dbc8c15c80753
│ │     Current version:  01
│ │     Vendor:           Intel Corporation (MEI:0x8086)
│ │     GUIDs:            dd17041c-09ea-4b17-a271-5b989867ec65
│ │                       fccad2fe-62ae-5879-b7a9-4ead7bce50f4 ← MEI\VEN_8086&DEV_51E0
│ │                       eae67b2f-2bc2-5c4f-8b82-b1b30ad69fff ← MEI\VEN_8086&DEV_51E0&SUBSYS_17AA22E7
│ │     Device Flags:     • Internal device
│ │   
│ ├─UEFI Platform Key:
│ │     Device ID:        6924110cde4fa051bfdc600a60620dc7aa9d3c6a
│ │     Summary:          Lenovo Ltd. PK CA 2012
│ │     Vendor:           Lenovo Ltd.
│ │     GUID:             71599d14-9b31-5270-b3bd-74c494585820 ← UEFI\CRT_9AEF2123F4DE7C19AFABD909BB2C8CAC4411E07E
│ │   
│ └─UEFI dbx:
│       Device ID:        362301da643102b9f38477387e2193e57abaa590
│       Summary:          UEFI revocation database
│       Current version:  267
│       Minimum Version:  267
│       Vendor:           UEFI:Linux Foundation
│       Install Duration: 1 second
│       GUIDs:            14503b3d-73ce-5d06-8137-77c68972a341 ← UEFI\CRT_A9087D1044AD18F7A94916D284CBC01827CF23CD8F60B79072C9CAA1FEF4D649
│                         5971a208-da00-5fce-b5f5-1234342f9cf7 ← UEFI\CRT_A9087D1044AD18F7A94916D284CBC01827CF23CD8F60B79072C9CAA1FEF4D649&ARCH_X64
│                         c6682ade-b5ec-57c4-b687-676351208742 ← UEFI\CRT_A1117F516A32CEFCBA3F2D1ACE10A87972FD6BBE8FE0D0B996E09E65D802A503
│                         f8ba2887-9411-5c36-9cee-88995bb39731 ← UEFI\CRT_A1117F516A32CEFCBA3F2D1ACE10A87972FD6BBE8FE0D0B996E09E65D802A503&ARCH_X64
│       Device Flags:     • Internal device
│                         • Updatable
│                         • Needs a reboot after installation
│                         • Device is usable for the duration of the update
│                         • Only version upgrades are allowed
│                         • Signed Payload
│     
├─TPM:
│     Device ID:          c6a80ac3a22083423992a3cb15018989f37834d6
│     Current version:    1.512.0.0
│     Vendor:             ST Microelectronics (TPM:STM)
│     GUIDs:              ff71992e-52f7-5eea-94ef-883e56e034c6 ← system-tpm
│                         84df3581-f896-54d2-bd1a-372602f04c32 ← TPM\VEN_STM&DEV_0001
│                         bfaed10a-bbc1-525b-a329-35da2f63e918 ← TPM\VEN_STM&MOD_
│                         70b7b833-7e1a-550a-a291-b94a12d0f319 ← TPM\VEN_STM&DEV_0001&VER_2.0
│                         06f005e9-cb62-5d1a-82d9-13c534c53c48 ← TPM\VEN_STM&MOD_&VER_2.0
│     Device Flags:       • Internal device
│                         • System requires external power source
│                         • Needs a reboot after installation
│                         • Device can recover flash failures
│                         • Full disk encryption secrets may be invalidated when updating
│                         • Signed Payload
│   
├─UEFI Device Firmware:
│     Device ID:          a45df35ac0e948ee180fe216a5f703f32dda163f
│     Summary:            UEFI ESRT device
│     Current version:    70151
│     Vendor:             DMI:LENOVO
│     Update State:       Success
│     GUID:               4e88068b-41b2-4e05-893c-db0b43f7d348
│     Device Flags:       • Internal device
│                         • Updatable
│                         • System requires external power source
│                         • Needs a reboot after installation
│                         • Device is usable for the duration of the update
│   
├─UEFI Device Firmware:
│     Device ID:          349bb341230b1a86e5effe7dfe4337e1590227bd
│     Summary:            UEFI ESRT device
│     Current version:    1
│     Minimum Version:    1
│     Vendor:             DMI:LENOVO
│     Update State:       Success
│     GUID:               69585d92-b50a-4ad7-b265-2eb1ae066574
│     Device Flags:       • Internal device
│                         • Updatable
│                         • System requires external power source
│                         • Needs a reboot after installation
│                         • Device is usable for the duration of the update
│   
├─UEFI Device Firmware:
│     Device ID:          632acf4927c0b5fb53519d6beed3b60adb73f1d5
│     Summary:            UEFI ESRT device
│     Current version:    984818
│     Minimum Version:    1
│     Vendor:             DMI:LENOVO
│     Update State:       Success
│     GUID:               d208615d-3a6f-4aa4-b51f-87c5a025b30b
│     Device Flags:       • Internal device
│                         • Updatable
│                         • System requires external power source
│                         • Needs a reboot after installation
│                         • Device is usable for the duration of the update
│   
├─UEFI Device Firmware:
│     Device ID:          97b6fe9b220c7b9e1a3a1d9f404c00d4fe77ae7e
│     Summary:            UEFI ESRT device
│     Current version:    65549
│     Minimum Version:    65549
│     Vendor:             DMI:LENOVO
│     Update State:       Success
│     GUID:               ec01fae4-c67a-42b4-bada-a7c1b9900897
│     Device Flags:       • Internal device
│                         • Updatable
│                         • System requires external power source
│                         • Needs a reboot after installation
│                         • Device is usable for the duration of the update
│   
├─UEFI Device Firmware:
│     Device ID:          ae1abd099407b1d95698d69b7273f7fed5c6f35c
│     Summary:            UEFI ESRT device
│     Current version:    16908298
│     Minimum Version:    1
│     Vendor:             DMI:LENOVO
│     Update State:       Success
│     GUID:               7b355cab-d481-4df3-b78e-c7b8e187b5d7
│     Device Flags:       • Internal device
│                         • Updatable
│                         • System requires external power source
│                         • Needs a reboot after installation
│                         • Device is usable for the duration of the update
│   
├─UEFI Device Firmware:
│     Device ID:          4b78f537b7d3e281a2ecbc83048b8856fb9eb98e
│     Summary:            UEFI ESRT device
│     Current version:    1410863573
│     Minimum Version:    1
│     Vendor:             DMI:LENOVO
│     Update State:       Success
│     GUID:               04cb082c-77e9-4fd8-8832-94e0bdd2dfce
│     Device Flags:       • Internal device
│                         • Updatable
│                         • System requires external power source
│                         • Needs a reboot after installation
│                         • Device is usable for the duration of the update
│   
├─UEFI Device Firmware:
│     Device ID:          2656b5b7e7c4f91fef1537f93095449cda5fb264
│     Summary:            UEFI ESRT device
│     Current version:    1
│     Minimum Version:    1
│     Vendor:             DMI:LENOVO
│     Update State:       Success
│     GUID:               7e5534a3-2069-414b-90b6-3e365d2ccd09
│     Device Flags:       • Internal device
│                         • Updatable
│                         • System requires external power source
│                         • Needs a reboot after installation
│                         • Device is usable for the duration of the update
│   
├─UEFI Device Firmware:
│     Device ID:          de4826630a1fc5be5355c9b6b2c4052051f350c6
│     Summary:            UEFI ESRT device
│     Current version:    66048
│     Minimum Version:    66048
│     Vendor:             DMI:LENOVO
│     Update State:       Success
│     GUID:               dcde6a43-bdd2-4da4-ace7-d005c112bd13
│     Device Flags:       • Internal device
│                         • Updatable
│                         • System requires external power source
│                         • Needs a reboot after installation
│                         • Device is usable for the duration of the update
│   
├─UEFI Device Firmware:
│     Device ID:          2292ae5236790b47884e37cf162dcf23bfcd1c60
│     Summary:            UEFI ESRT device
│     Current version:    984818
│     Vendor:             DMI:LENOVO
│     Update State:       Success
│     GUID:               4e78ce68-5389-4a95-bf10-e3568c30caf8
│     Device Flags:       • Internal device
│                         • Updatable
│                         • System requires external power source
│                         • Needs a reboot after installation
│                         • Device is usable for the duration of the update
│   
├─UEFI Device Firmware:
│     Device ID:          f95c9218acd12697af946874bfe4239587209232
│     Summary:            UEFI ESRT device
│     Current version:    1
│     Vendor:             DMI:LENOVO
│     Update State:       Success
│     GUID:               76ca0ad8-4a14-4389-b7e5-fd88791762ad
│     Device Flags:       • Internal device
│                         • Updatable
│                         • System requires external power source
│                         • Needs a reboot after installation
│                         • Device is usable for the duration of the update
│   
├─UEFI Device Firmware:
│     Device ID:          d96de5c124b60ed6241ebcb6bb2c839cb5580786
│     Summary:            UEFI ESRT device
│     Current version:    1
│     Vendor:             DMI:LENOVO
│     Update State:       Success
│     GUID:               626d93db-2c42-48c3-915a-71f968a81b04
│     Device Flags:       • Internal device
│                         • Updatable
│                         • System requires external power source
│                         • Needs a reboot after installation
│                         • Device is usable for the duration of the update
│   
├─UEFI Device Firmware:
│     Device ID:          f37fb01122dd62c773f4e84ec89737e059712d59
│     Summary:            UEFI ESRT device
│     Current version:    1
│     Vendor:             DMI:LENOVO
│     Update State:       Success
│     GUID:               86a885ee-d71e-2ed6-0fc1-9d6ccc9677eb
│     Device Flags:       • Internal device
│                         • Updatable
│                         • System requires external power source
│                         • Needs a reboot after installation
│                         • Device is usable for the duration of the update
│   
├─UEFI Device Firmware:
│     Device ID:          36efb79c255f402f619fa9eb53cd659db51f2a04
│     Summary:            UEFI ESRT device
│     Current version:    12713984
│     Minimum Version:    57374
│     Vendor:             DMI:LENOVO
│     Update State:       Success
│     GUID:               09f77c9f-1c5d-4616-bafb-bbb19f557480
│     Device Flags:       • Internal device
│                         • Updatable
│                         • System requires external power source
│                         • Needs a reboot after installation
│                         • Device is usable for the duration of the update
│   
├─UEFI Device Firmware:
│     Device ID:          11a7fbdd09ce583e58b899660fe10f147fd155a2
│     Summary:            UEFI ESRT device
│     Current version:    532
│     Vendor:             DMI:LENOVO
│     Update State:       Success
│     GUID:               88523ddf-3aac-41f6-b15f-dc7bea434b39
│     Device Flags:       • Internal device
│                         • Updatable
│                         • System requires external power source
│                         • Needs a reboot after installation
│                         • Device is usable for the duration of the update
│   
└─UEFI Device Firmware:
      Device ID:          a083ebc5138e5e071ef7270cc9a8280722cc7adf
      Summary:            UEFI ESRT device
      Current version:    3549562
      Vendor:             DMI:LENOVO
      Update State:       Success
      GUID:               8bb5952d-450d-440b-8e40-026edcb5e638
      Device Flags:       • Internal device
                          • Updatable
                          • System requires external power source
                          • Needs a reboot after installation
                          • Device is usable for the duration of the update
@shenki shenki added the bug label Jul 27, 2023
@shenki
Copy link
Author

shenki commented Jul 27, 2023

I was working out why I had to disable secure boot, and it turns out Lenovo disables the Microsoft 3rd Party UEFI CA by default. Enabling that allowed the refresh operation to complete.

If that was really the cause, perhaps this issue could be about modifying the error message to indicate the signature failed due to UEFI CA issues?

@hughsie
Copy link
Member

hughsie commented Jul 27, 2023

I don't think disabling secure boot should affect anything like this -- can you try turning off secure boot again and doing fwupdmgr refresh --force again? It might have just been a transient network issue.

@cwrau
Copy link

cwrau commented Aug 2, 2023

I have the same problem, my secure boot is disabled and neither sudo fwupdmgr refresh nor sudo fwupdmgr refresh --force are working and both are failing with the following error;

Updating lvfs
Downloading…             [  -                                    ]
Failed to load signature: Need more input

@jknaus
Copy link

jknaus commented Aug 2, 2023

Same here (Lenovo TP490s, Fedora38)

sudo fwupdmgr refresh --force
Updating lvfs
Downloading…             [        |                              ]
Failed to load signature: Need more input

@ousatov-ua
Copy link

ousatov-ua commented Aug 2, 2023

Same here, Fedora 38. Appeared today.
Secure boot is disabled.

fwupdmgr refresh --force
Updating lvfs
Downloading… [ \ ]
Failed to load signature: Need more input

Tried to reinstall - did not help.

@ousatov-ua
Copy link

ousatov-ua commented Aug 2, 2023

Basically, this happens on enabling remote too:

fwupdmgr enable-remote lvfs
...

Agree and enable the remote? [Y|n]: Y
Authenticating… [ \ ]
Do you want to refresh this remote now? (Requires internet connection) [Y|n]: y
Downloading… [ - ]
Failed to load signature: Need more input

@artemkovalyov
Copy link

I've the same issue on Arch

@GodAndEmperor
Copy link

Lenovo P50, Fedora 38:
same issue as described by others above. After enabling CMS, the Download progress indicator looks a bit different, but issue remains:

fwupdmgr refresh --force
Updating lvfs
Downloading… [******************* ]
Failed to load signature: Need more input

@hughsie
Copy link
Member

hughsie commented Aug 2, 2023

This error is when we try to parse the signature in libjcat from the thing the CDN delivered. I'm on holiday on a little island with not much internet, so this will need someone else to debug this. I just refreshed just fine on my f38 machine I updated a few days ago, do it's not everyone affected. Did any recent update start causing this perhaps? Is it a transient issue?

@hughsie
Copy link
Member

hughsie commented Aug 2, 2023

For instance, the -vv output is going to be super useful, as is downgrading fwupd, libjcat, glib2, libcurl etc

@ousatov-ua
Copy link

Hi @hughsie !

fwupdmgr refresh --force -vv
(fwupdmgr:15194): GLib-DEBUG: 18:44:15.298: setenv()/putenv() are not thread-safe and should not be used after threads are created
(pkttyagent:15196): GLib-GIO-DEBUG: 18:44:15.300: Using cross-namespace EXTERNAL authentication (this will deadlock if server is GDBus < 2.73.3)
(fwupdmgr:15194): GLib-GIO-DEBUG: 18:44:15.305: _g_io_module_get_default: Found default implementation dconf (DConfSettingsBackend) for ‘gsettings-backend’
(fwupdmgr:15194): dconf-DEBUG: 18:44:15.305: watch_fast: "/system/proxy/" (establishing: 0, active: 0)
(fwupdmgr:15194): dconf-DEBUG: 18:44:15.305: watch_fast: "/system/proxy/http/" (establishing: 0, active: 0)
(fwupdmgr:15194): dconf-DEBUG: 18:44:15.305: watch_fast: "/system/proxy/https/" (establishing: 0, active: 0)
(fwupdmgr:15194): dconf-DEBUG: 18:44:15.305: watch_fast: "/system/proxy/ftp/" (establishing: 0, active: 0)
(fwupdmgr:15194): dconf-DEBUG: 18:44:15.305: watch_fast: "/system/proxy/socks/" (establishing: 0, active: 0)
(fwupdmgr:15194): dconf-DEBUG: 18:44:15.305: unwatch_fast: "/system/proxy/" (active: 0, establishing: 1)
(fwupdmgr:15194): dconf-DEBUG: 18:44:15.305: unwatch_fast: "/system/proxy/http/" (active: 0, establishing: 1)
(fwupdmgr:15194): dconf-DEBUG: 18:44:15.305: unwatch_fast: "/system/proxy/https/" (active: 0, establishing: 1)
(fwupdmgr:15194): dconf-DEBUG: 18:44:15.305: unwatch_fast: "/system/proxy/ftp/" (active: 0, establishing: 1)
(fwupdmgr:15194): dconf-DEBUG: 18:44:15.305: unwatch_fast: "/system/proxy/socks/" (active: 0, establishing: 1)
(fwupdmgr:15194): GLib-GIO-DEBUG: 18:44:15.305: Using cross-namespace EXTERNAL authentication (this will deadlock if server is GDBus < 2.73.3)
(fwupdmgr:15194): GLib-GIO-DEBUG: 18:44:15.306: Using cross-namespace EXTERNAL authentication (this will deadlock if server is GDBus < 2.73.3)
(fwupdmgr:15194): GLib-GIO-DEBUG: 18:44:15.306: _g_io_module_get_default: Found default implementation libproxy (GLibproxyResolver) for ‘gio-proxy-resolver’
(fwupdmgr:15194): dconf-DEBUG: 18:44:15.307: watch_established: "/system/proxy/" (establishing: 0)
(fwupdmgr:15194): dconf-DEBUG: 18:44:15.307: watch_established: "/system/proxy/http/" (establishing: 0)
(fwupdmgr:15194): dconf-DEBUG: 18:44:15.307: watch_established: "/system/proxy/https/" (establishing: 0)
(fwupdmgr:15194): dconf-DEBUG: 18:44:15.307: watch_established: "/system/proxy/ftp/" (establishing: 0)
(fwupdmgr:15194): dconf-DEBUG: 18:44:15.307: watch_established: "/system/proxy/socks/" (establishing: 0)
(fwupdmgr:15194): Fwupd-DEBUG: 18:44:15.307: Emitting ::status-changed() [idle]
Updating lvfs
Fwupd-INFO: 18:44:15.310: downloading https://cdn.fwupd.org/downloads/firmware.xml.xz.jcat
(fwupdmgr:15194): Fwupd-DEBUG: 18:44:15.310: Emitting ::status-changed() [downloading]
Downloading… [ \ ]Fwupd-INFO: 18:44:15.512: download progress: 100%
(fwupdmgr:15194): Fwupd-DEBUG: 18:44:15.512: Emitting ::status-changed() [idle]
Fwupd-INFO: 18:44:15.512: status-code was 200

Failed to load signature: Need more input

@hughsie
Copy link
Member

hughsie commented Aug 2, 2023

If you remove cdn. from the lvfs.conf does it work? If you get the .jcat file what does it look like?

@hughsie
Copy link
Member

hughsie commented Aug 2, 2023

And does the jcat cli load the file?

@ousatov-ua
Copy link

@hughsie I removed cdn. from config and it worked!

@ousatov-ua
Copy link

ousatov-ua commented Aug 2, 2023

@hughsie Cannot attach that .jcat - github does not support it
archive.tar.gz

It is in archive

@hughsie
Copy link
Member

hughsie commented Aug 2, 2023

Can you compare what the CDN sent and what the main server sent please?

@ousatov-ua
Copy link

ousatov-ua commented Aug 2, 2023

@hughsie
It is strange, but their content is identical -after extraction
Maybe some invisible symbols...

@ousatov-ua
Copy link

ousatov-ua commented Aug 2, 2023

Maybe the problem with archive itself

@ousatov-ua
Copy link

ousatov-ua commented Aug 2, 2023

These are them after extraction, formatted for readability
Archive.zip

@ousatov-ua
Copy link

@hughsie Seems like the issue is resolved. I can run
fwupdmgr refresh --force

for cdn

@GodAndEmperor
Copy link

I also don't get an error anymore:
fwupdmgr refresh --force
Updating lvfs
Downloading… [ - ]
Successfully downloaded new metadata: 2 local devices supported

@cwrau
Copy link

cwrau commented Aug 2, 2023

I also don't get the error anymore when executing it by hand, but the systemd service still fails, but without any error message, just the following;

2023-08-02T21:44:16.879379+0200 steve systemd[1]: Starting Refresh fwupd metadata and update motd...
2023-08-02T21:44:16.902677+0200 steve systemd[1]: fwupd-refresh.service: Main process exited, code=exited, status=1/FAILURE
2023-08-02T21:44:16.902802+0200 steve systemd[1]: fwupd-refresh.service: Failed with result 'exit-code'.
2023-08-02T21:44:16.916016+0200 steve systemd[1]: Failed to start Refresh fwupd metadata and update motd.
``'

@hughsie
Copy link
Member

hughsie commented Aug 3, 2023

So maybe the CDN went crazy? For the systemd service, can you add -v and see any more debug lines?

@cwrau
Copy link

cwrau commented Aug 3, 2023

So maybe the CDN went crazy? For the systemd service, can you add -v and see any more debug lines?

Sure;

2023-08-03T13:28:13.325515+0200 steve systemd[1]: Starting Refresh fwupd metadata and update motd...
2023-08-03T13:28:13.341499+0200 steve fwupdmgr[2999241]: (/usr/bin/fwupdmgr:2999241): GLib-DEBUG: 13:28:13.341: setenv()/putenv() are not thread-safe and should not be used after threads are created
2023-08-03T13:28:13.343537+0200 steve fwupdmgr[2999241]: (/usr/bin/fwupdmgr:2999241): GLib-GIO-DEBUG: 13:28:13.343: _g_io_module_get_default: Found default implementation dconf (DConfSettingsBackend) for ‘gsettings-backend’
2023-08-03T13:28:13.343567+0200 steve fwupdmgr[2999241]: (/usr/bin/fwupdmgr:2999241): dconf-DEBUG: 13:28:13.343: watch_fast: "/system/proxy/" (establishing: 0, active: 0)
2023-08-03T13:28:13.343711+0200 steve fwupdmgr[2999241]: (/usr/bin/fwupdmgr:2999241): dconf-DEBUG: 13:28:13.343: watch_fast: "/system/proxy/http/" (establishing: 0, active: 0)
2023-08-03T13:28:13.343756+0200 steve fwupdmgr[2999241]: (/usr/bin/fwupdmgr:2999241): dconf-DEBUG: 13:28:13.343: watch_fast: "/system/proxy/https/" (establishing: 0, active: 0)
2023-08-03T13:28:13.343786+0200 steve fwupdmgr[2999241]: (/usr/bin/fwupdmgr:2999241): dconf-DEBUG: 13:28:13.343: watch_fast: "/system/proxy/ftp/" (establishing: 0, active: 0)
2023-08-03T13:28:13.343808+0200 steve fwupdmgr[2999241]: (/usr/bin/fwupdmgr:2999241): dconf-DEBUG: 13:28:13.343: watch_fast: "/system/proxy/socks/" (establishing: 0, active: 0)
2023-08-03T13:28:13.343840+0200 steve fwupdmgr[2999241]: (/usr/bin/fwupdmgr:2999241): dconf-DEBUG: 13:28:13.343: unwatch_fast: "/system/proxy/" (active: 0, establishing: 1)
2023-08-03T13:28:13.343840+0200 steve fwupdmgr[2999241]: (/usr/bin/fwupdmgr:2999241): dconf-DEBUG: 13:28:13.343: unwatch_fast: "/system/proxy/http/" (active: 0, establishing: 1)
2023-08-03T13:28:13.343921+0200 steve fwupdmgr[2999241]: (/usr/bin/fwupdmgr:2999241): dconf-DEBUG: 13:28:13.343: unwatch_fast: "/system/proxy/https/" (active: 0, establishing: 1)
2023-08-03T13:28:13.343921+0200 steve fwupdmgr[2999241]: (/usr/bin/fwupdmgr:2999241): dconf-DEBUG: 13:28:13.343: unwatch_fast: "/system/proxy/ftp/" (active: 0, establishing: 1)
2023-08-03T13:28:13.343921+0200 steve fwupdmgr[2999241]: (/usr/bin/fwupdmgr:2999241): dconf-DEBUG: 13:28:13.343: unwatch_fast: "/system/proxy/socks/" (active: 0, establishing: 1)
2023-08-03T13:28:13.344117+0200 steve fwupdmgr[2999241]: (/usr/bin/fwupdmgr:2999241): dconf-DEBUG: 13:28:13.344: watch_established: "/system/proxy/" (establishing: 0)
2023-08-03T13:28:13.344152+0200 steve fwupdmgr[2999241]: (/usr/bin/fwupdmgr:2999241): dconf-DEBUG: 13:28:13.344: watch_established: "/system/proxy/http/" (establishing: 0)
2023-08-03T13:28:13.344192+0200 steve fwupdmgr[2999241]: (/usr/bin/fwupdmgr:2999241): dconf-DEBUG: 13:28:13.344: watch_established: "/system/proxy/https/" (establishing: 0)
2023-08-03T13:28:13.344223+0200 steve fwupdmgr[2999241]: (/usr/bin/fwupdmgr:2999241): dconf-DEBUG: 13:28:13.344: watch_established: "/system/proxy/ftp/" (establishing: 0)
2023-08-03T13:28:13.344257+0200 steve fwupdmgr[2999241]: (/usr/bin/fwupdmgr:2999241): dconf-DEBUG: 13:28:13.344: watch_established: "/system/proxy/socks/" (establishing: 0)
2023-08-03T13:28:13.344993+0200 steve fwupdmgr[2999241]: (/usr/bin/fwupdmgr:2999241): GLib-GIO-DEBUG: 13:28:13.344: _g_io_module_get_default: Found default implementation local (GLocalVfs) for ‘gio-vfs’
2023-08-03T13:28:13.345027+0200 steve fwupdmgr[2999241]: (/usr/bin/fwupdmgr:2999241): pxbackend-DEBUG: 13:28:13.345: px_config_sysconfig_set_config_file: Could not read file /etc/sysconfig/proxy
2023-08-03T13:28:13.345027+0200 steve fwupdmgr[2999241]: (/usr/bin/fwupdmgr:2999241): pxbackend-DEBUG: 13:28:13.345: Active config plugins:
2023-08-03T13:28:13.345027+0200 steve fwupdmgr[2999241]: (/usr/bin/fwupdmgr:2999241): pxbackend-DEBUG: 13:28:13.345:  - config-env
2023-08-03T13:28:13.345027+0200 steve fwupdmgr[2999241]: (/usr/bin/fwupdmgr:2999241): pxbackend-DEBUG: 13:28:13.345:  - config-kde
2023-08-03T13:28:13.345027+0200 steve fwupdmgr[2999241]: (/usr/bin/fwupdmgr:2999241): pxbackend-DEBUG: 13:28:13.345:  - config-gnome
2023-08-03T13:28:13.345091+0200 steve fwupdmgr[2999241]: (/usr/bin/fwupdmgr:2999241): pxbackend-DEBUG: 13:28:13.345:  - config-sysconfig
2023-08-03T13:28:13.346094+0200 steve fwupdmgr[2999241]: (/usr/bin/fwupdmgr:2999241): GLib-GIO-DEBUG: 13:28:13.346: Failed to initialize portal (GNetworkMonitorPortal) for gio-network-monitor: Not using portals
2023-08-03T13:28:13.346448+0200 steve fwupdmgr[2999241]: (/usr/bin/fwupdmgr:2999241): GLib-GIO-DEBUG: 13:28:13.346: Using cross-namespace EXTERNAL authentication (this will deadlock if server is GDBus < 2.73.3)
2023-08-03T13:28:13.346818+0200 steve fwupdmgr[2999241]: (/usr/bin/fwupdmgr:2999241): GLib-GIO-DEBUG: 13:28:13.346: Failed to initialize networkmanager (GNetworkMonitorNM) for gio-network-monitor: The connection is closed
2023-08-03T13:28:13.346870+0200 steve fwupdmgr[2999241]: (/usr/bin/fwupdmgr:2999241): GLib-GIO-DEBUG: 13:28:13.346: _g_io_module_get_default: Found default implementation netlink (GNetworkMonitorNetlink) for ‘gio-network-monitor’
2023-08-03T13:28:13.346915+0200 steve fwupdmgr[2999241]: (/usr/bin/fwupdmgr:2999241): pxbackend-DEBUG: 13:28:13.346: px_manager_on_network_changed: Network connection changed, clearing pac data
2023-08-03T13:28:13.346915+0200 steve fwupdmgr[2999241]: (/usr/bin/fwupdmgr:2999241): pxbackend-DEBUG: 13:28:13.346: px_manager_constructed: Up and running
2023-08-03T13:28:13.346915+0200 steve fwupdmgr[2999241]: (/usr/bin/fwupdmgr:2999241): GLib-GIO-DEBUG: 13:28:13.346: _g_io_module_get_default: Found default implementation libproxy (GLibproxyResolver) for ‘gio-proxy-resolver’
2023-08-03T13:28:13.348511+0200 steve systemd[1]: fwupd-refresh.service: Main process exited, code=exited, status=1/FAILURE

@operutka
Copy link

operutka commented Aug 8, 2023

Same problem here. Downloading firmware.xml.xz.jcat from the main server instead of CDN also works for me.

I compared firmware.xml.xz.jcat from the main server with the one I get from CDN. Both of them are gzipped JSONs, so I really don't get the .xml.xz part of the extension but who cares... :) After decompressing the one from the main server, I get the following JSON:

{"JcatVersionMajor": 0, "JcatVersionMinor": 1, "Items": [{"Id": "firmware-06672-stable.xml.xz", "AliasIds": ["firmware.xml.xz"], "Blobs": [{"Kind": 4, "Flags": 1, "Timestamp": 1691483156, "Data": "eacfbe188f1e337c983ed9c262e9b15a24182aac"}, {"Kind": 1, "Flags": 1, "Timestamp": 1691483156, "Data": "29ec1995117637f7c336064170dc911c22312b83554b13e1c694de0581320e14"}, {"Kind": 2, "Flags": 1, "Timestamp": 1691483162, "Data": "-----BEGIN PGP SIGNATURE-----\n\niQEzBAABCAAdFiEEP8a4BEEO0IQNjy+XSKbYDkU4usIFAmTR/BoACgkQSKbYDkU4\nusIqFgf/cm4MpOvBq3nOHQIfjJ8/SEQzf1onvVWAD231m4oVE3xgWMBWzc4uQRvt\nCJajR1j6OoHPezb3aZiXOUeTAv7adQZO7Hg4OK6kGIZl1hgL2kYnwbDldccVwUPo\n1I0pq3naoGWxgN+Z0i5IkdC2ul4KR9fnar/vzzh4vRimV9WAJJtM9vZdUM0/+miT\nVubLNbEWrIIJcCyIbdVpMWkyGkSRllxjmPGZpwd/73vn6AKI+oumlXsOQzA1Uf8W\nJCKeOCjC7/B53bX8dNCf9ThgWa3wfEVvVJSCxkq5gkhSPTP85QUA09xPcOZXao2A\n9nA7gf8jTFNEqhTde+VLrQq463TkcQ==\n=BnoP\n-----END PGP SIGNATURE-----\n"}, {"Kind": 3, "Flags": 1, "Timestamp": 1691483162, "Data": "-----BEGIN PKCS7-----\nMIIGUgYJKoZIhvcNAQcCoIIGQzCCBj8CAQExDTALBglghkgBZQMEAgEwCwYJKoZI\nhvcNAQcBoIIEOjCCBDYwggKeoAMCAQICDFprhisibP88kP07YjANBgkqhkiG9w0B\nAQsFADA6MRAwDgYDVQQDEwdMVkZTIENBMSYwJAYDVQQKEx1MaW51eCBWZW5kb3Ig\nRmlybXdhcmUgUHJvamVjdDAeFw0xODAxMTYwMDAwMDBaFw0yODAxMTYwMDAwMDBa\nMBkxFzAVBgNVBAMTDlJpY2hhcmQgSHVnaGVzMIIBIjANBgkqhkiG9w0BAQEFAAOC\nAQ8AMIIBCgKCAQEArR5nseG3+Zs+o41P9LTspiVSGVcp6ifNHhbKKxBvZZXsy0gX\n+P/VRlsHLiKQrFulQ8GbPODytFv0o+y/0MkiJxv/fY3yEZ2bwNpsSeXFQSGHI6yz\njaVNeNCu8lnnDtD7kiC8UNUNHcnA4+2h/Yv4k+KPqYF+Qb6nWAEIID1ObMnjeJUb\ndbPPvy12aasV3gZcZ+goYNFc0ua6OU/CNEuzAVVCTAJ/EpCdGpll1+6BGU5ImIUG\nTlMTWq2xmCfCPugakHrmWA66yHWwE6LmC/U7qQDWFemsSNnmzyBB9HPkqsW1DjHr\n+ZmNUPj3+q2UGnNwP/Cne462XbsZB569w7pnzQIDAQABo4HcMIHZMAwGA1UdEwEB\n/wQCMAAwNwYDVR0RBDAwLoYXaHR0cHM6Ly9md3VwZC5vcmcvbHZmcy+BE3JpY2hh\ncmRAaHVnaHNpZS5jb20wEwYDVR0lBAwwCgYIKwYBBQUHAwMwDwYDVR0PAQH/BAUD\nAweAADAdBgNVHQ4EFgQUMcrnDWk3DM1GJCqJK7EbAycgB+UwHwYDVR0jBBgwFoAU\nsY3q5COnfgmOte4x4GrdnjQ3ZawwKgYDVR0fBCMwITAfoB2gG4YZaHR0cDovL3d3\ndy5md3VwZC5vcmcvcGtpLzANBgkqhkiG9w0BAQsFAAOCAYEAjoXBOtVb1qPLuE7I\nShdSkk66JMNmzZnODbun3BaViUF3PPhuRiJ2y7Bu7loCnxHwzKpq5Hn4Untg1WhQ\nSsPEv6H2oc2E4HU1Gds2HEE1UL0VzulPEwaOFX0OZ37LmJ2VFvfgwfNmjAWZ+ngN\nsjuff3PWeHNsOwbS3To0CBlHqz9LlcA7Gxpxbz4DbUEvgTTKZFbasgUS6/QmbpPy\n880ThgD93yCg6Q002AooPMw8zW6gGUP7/2D4aKBjgM/IJDrJEicTdKDFtUacsN+d\nCfIEcn6F1rzPgwo0B/yLYVMpbL8WzxSS7cX5dBlkreH6Nf7g+A2NTryU/6FxoFxo\nnWJmLssNQCgx42Ywc8CgJRt1yWXxoZXOPap8zPvFAAmrfUlqfZlr2Wzcf2nxpSZJ\noQRzGDXDRiYNt28xjX408cJ+Wy+0oWwgnXCkfzMDnJQB8MX6+5Ah4v4mS/XjmOIi\nQ+hKCT99Yf0Eq3TWzFhyYducqLU904FbrrUSoi07DfxXa/8RMYIB3jCCAdoCAQEw\nSjA6MRAwDgYDVQQDEwdMVkZTIENBMSYwJAYDVQQKEx1MaW51eCBWZW5kb3IgRmly\nbXdhcmUgUHJvamVjdAIMWmuGKyJs/zyQ/TtiMAsGCWCGSAFlAwQCAaBpMBgGCSqG\nSIb3DQEJAzELBgkqhkiG9w0BBwEwHAYJKoZIhvcNAQkFMQ8XDTIzMDgwODA4MjYw\nMlowLwYJKoZIhvcNAQkEMSIEICnsGZURdjf3wzYGQXDckRwiMSuDVUsT4caU3gWB\nMg4UMA0GCSqGSIb3DQEBAQUABIIBABng4wodCLYNGreeKti2m4cG2JahoNDDrrv2\nLRMFPpkHvOmlPdO2GW00+xsD+EYbv1Xu3X6OD/AOpQwkupugA4kaLh4bE3iSvR3R\ncLi6bpMcA3icQpQXcfNtgQMUMkbzx+EkWmKGqRKlVSOn/rGd/031lSriev0P7DUQ\n4UEwoXai8iC/KDQsd2i4/GEBeUVtOzba1SH2q5uyBduSE0e5N6IxNISL7EywAb0K\n2UCS6E92xd0WyDQgV9QMzxfaqcqYKdnWtHJUgpHsbX1jXzmpmxER8WZrnHWmsyn2\nLU6BBtTOEzbP4gqUkCxqXh5Z0mfWyEDKYGdzMNi0fk25dFvvwVk=\n-----END PKCS7-----\n"}]}]}

However, decompressing the one I get from CDN fails:

# gzip -d firmware.xml.xz.jcat.gz 

gzip: firmware.xml.xz.jcat.gz: unexpected end of file

I also compared the gzipped binaries right after the download (i.e. without decompressing them) and it appears that the CDN truncates the original file. The file downloaded from CDN is four bytes shorter and it is a prefix of the original file from the main server (i.e. if I remove the last four bytes from the file downloaded from the main server, it matches exactly to the file downloaded from CDN).

@hughsie
Copy link
Member

hughsie commented Aug 8, 2023

What's the file size in bytes of the truncated version out of interest?

@vajdao
Copy link

vajdao commented Aug 8, 2023

I was facing this issue on Debian 12.1 Bookworm aswell, when I opened up "Discover" software centre. An error message popped up: "Failed to load signature: need more input".

Now, what I tried based on the previous comments are and every single attemts are failure:

Updating my BIOS/UEFI to the latest version: negative
Enabling secure boot: negative
Disabling secure boot: negative
sudo fwupdmgr refresh and sudo fwupdmgr refresh --force: negative & negative
Adding "invalid;bios" to the /etc/fwupd/daemon.conf's 'DisabledPlugins' line then doing a restart: negative

My UEFI menu doesn't have anything such as enable/disable capsule UEFI update or such, BUT all I know that my machine was perfectly working as intended, everything was normal, until my distro received a big update, and after that, I'm receiving these error messages are bombarding me, and it seems everyone else. I'm on Debian stable for a reason, for not to receive such bugs, but it seems, not even that is a warranty...

I can't seem to find any fix on this, and based on the comments, for now we are groping in the darkness.

@Lavode
Copy link

Lavode commented Aug 8, 2023

What's the file size in bytes of the truncated version out of interest?

› du -b *.jcat
2223	cdn.xml.xz.jcat
2227	main.xml.xz.jcat

I.e. they differ by four bytes. Specifically the last four bytes are missing on the CDN:

› diff <(xxd cdn.xml.xz.jcat) <(xxd main.xml.xz.jcat)
139c139,140
< 000008a0: d20f 2f33 5fff fedb fefd 07cd ccb9 da    ../3_..........
---
> 000008a0: d20f 2f33 5fff fedb fefd 07cd ccb9 dac4  ../3_...........
> 000008b0: 0c00 00                                  ...

Funnily enough the four bytes missing are exactly the four bytes which, in the gzip file format as part of its 8B trailer, indicate the original length of the uncompressed text (0x00 00 0c c4 = 3268B). Meaning a more forgiving implementation of gzip which ignores that part of the trailer will happily decompress it correctly.

@PocketSam
Copy link

$ diff firmware.xml.xz.jcat cdn.firmware.xml.xz.jcat
Binary files firmware.xml.xz.jcat and cdn.firmware.xml.xz.jcat differ

-rw-r--r--. 1 sam sam 2223 авг 8 06:25 cdn.firmware.xml.xz.jcat
-rw-r--r--. 1 sam sam 2227 авг 8 10:26 firmware.xml.xz.jcat

@hughsie
Copy link
Member

hughsie commented Sep 7, 2023

@cwrau that's a different issue, no? what does "fwupdmgr refresh --force" say for you on the interactive CLI?

@cwrau
Copy link

cwrau commented Sep 7, 2023

@cwrau that's a different issue, no? what does "fwupdmgr refresh --force" say for you on the interactive CLI?

Ah, ok, so I should open a new issue?

Running interactively works, as it has before, just the systemd refresh service fails

@hughsie
Copy link
Member

hughsie commented Sep 7, 2023

Yup, new issue please.

@Pentasis
Copy link

Pentasis commented Sep 15, 2023

I'm new at linux so forgive me if I sound like a noob. I'm on Kubuntu 23.04 for a few days now. Today (after updating OpenSSH via Discovery) I am getting this error whenever I open Discovery.

This is the output using -vv

(fwupdmgr:4100): GLib-DEBUG: 11:33:53.489: setenv()/putenv() are not thread-safe and should not be used after threads are created
(process:4102): GLib-GIO-DEBUG: 11:33:53.493: Using cross-namespace EXTERNAL authentication (this will deadlock if server is GDBus < 2.73.3)
(fwupdmgr:4100): GLib-GIO-DEBUG: 11:33:53.498: _g_io_module_get_default: Found default implementation dconf (DConfSettingsBackend) for ?gsettings-backend?
(fwupdmgr:4100): dconf-DEBUG: 11:33:53.498: watch_fast: "/system/proxy/" (establishing: 0, active: 0)
(fwupdmgr:4100): dconf-DEBUG: 11:33:53.498: watch_fast: "/system/proxy/http/" (establishing: 0, active: 0)
(fwupdmgr:4100): dconf-DEBUG: 11:33:53.498: watch_fast: "/system/proxy/https/" (establishing: 0, active: 0)
(fwupdmgr:4100): dconf-DEBUG: 11:33:53.498: watch_fast: "/system/proxy/ftp/" (establishing: 0, active: 0)
(fwupdmgr:4100): dconf-DEBUG: 11:33:53.498: watch_fast: "/system/proxy/socks/" (establishing: 0, active: 0)
(fwupdmgr:4100): dconf-DEBUG: 11:33:53.498: unwatch_fast: "/system/proxy/" (active: 0, establishing: 1)
(fwupdmgr:4100): dconf-DEBUG: 11:33:53.498: unwatch_fast: "/system/proxy/http/" (active: 0, establishing: 1)
(fwupdmgr:4100): dconf-DEBUG: 11:33:53.498: unwatch_fast: "/system/proxy/https/" (active: 0, establishing: 1)
(fwupdmgr:4100): dconf-DEBUG: 11:33:53.498: unwatch_fast: "/system/proxy/ftp/" (active: 0, establishing: 1)
(fwupdmgr:4100): dconf-DEBUG: 11:33:53.498: unwatch_fast: "/system/proxy/socks/" (active: 0, establishing: 1)
(fwupdmgr:4100): GLib-GIO-DEBUG: 11:33:53.499: _g_io_module_get_default: Found default implementation libproxy (GLibproxyResolver) for ?gio-proxy-resolver?
(fwupdmgr:4100): GLib-GIO-DEBUG: 11:33:53.499: Using cross-namespace EXTERNAL authentication (this will deadlock if server is GDBus < 2.73.3)
(fwupdmgr:4100): GLib-GIO-DEBUG: 11:33:53.499: Using cross-namespace EXTERNAL authentication (this will deadlock if server is GDBus < 2.73.3)
(fwupdmgr:4100): dconf-DEBUG: 11:33:53.500: watch_established: "/system/proxy/" (establishing: 0)
(fwupdmgr:4100): dconf-DEBUG: 11:33:53.500: watch_established: "/system/proxy/http/" (establishing: 0)
(fwupdmgr:4100): dconf-DEBUG: 11:33:53.501: watch_established: "/system/proxy/https/" (establishing: 0)
(fwupdmgr:4100): dconf-DEBUG: 11:33:53.501: watch_established: "/system/proxy/ftp/" (establishing: 0)
(fwupdmgr:4100): dconf-DEBUG: 11:33:53.501: watch_established: "/system/proxy/socks/" (establishing: 0)
(fwupdmgr:4100): Fwupd-DEBUG: 11:33:53.502: Emitting ::status-changed() [idle]
Updating lvfs
(fwupdmgr:4100): Fwupd-DEBUG: 11:33:53.505: downloading https://cdn.fwupd.org/downloads/firmware.xml.xz.jcat
(fwupdmgr:4100): Fwupd-DEBUG: 11:33:53.505: Emitting ::status-changed() [downloading]
Downloading?             [     \                                 ](fwupdmgr:4100): Fwupd-DEBUG: 11:33:53.678: download progress: 100%
(fwupdmgr:4100): Fwupd-DEBUG: 11:33:53.678: download progress: 100%
(fwupdmgr:4100): Fwupd-DEBUG: 11:33:53.678: download progress: 100%
(fwupdmgr:4100): Fwupd-DEBUG: 11:33:53.678: download progress: 100%
(fwupdmgr:4100): Fwupd-DEBUG: 11:33:53.678: Emitting ::status-changed() [idle]
Downloading?             [***************************************](fwupdmgr:4100): Fwupd-DEBUG: 11:33:53.679: status-code was 200

Failed to load signature: Need more input

I'm in The Netherlands if that is important.

@travier
Copy link
Contributor

travier commented Sep 25, 2023

I have an issue with similar symptoms since today, on an updated Fedora 39 (Kinoite but should not matter).

I've noticed that if I move the URL back to gz, then it works again, suggesting that there is an issue with the server or the xz metadata:

$ cat /etc/fwupd/remotes.d/lvfs.conf
[fwupd Remote]

# this remote provides metadata and firmware marked as 'stable' from the LVFS
Enabled=true
Title=Linux Vendor Firmware Service
MetadataURI=https://cdn.fwupd.org/downloads/firmware.xml.gz
ReportURI=https://fwupd.org/lvfs/firmware/report
SecurityReportURI=https://fwupd.org/lvfs/hsireports/upload
AutomaticReports=true
AutomaticSecurityReports=true
ApprovalRequired=false
$ sudo fwupdmgr refresh --force
Updating lvfs
Downloading…             [  -                                    ]
Successfully downloaded new metadata: 6 local devices supported

$ cat /etc/fwupd/remotes.d/lvfs.conf
[fwupd Remote]

# this remote provides metadata and firmware marked as 'stable' from the LVFS
Enabled=true
Title=Linux Vendor Firmware Service
MetadataURI=https://cdn.fwupd.org/downloads/firmware.xml.xz
ReportURI=https://fwupd.org/lvfs/firmware/report
SecurityReportURI=https://fwupd.org/lvfs/hsireports/upload
AutomaticReports=true
AutomaticSecurityReports=true
ApprovalRequired=false
$ sudo fwupdmgr refresh --force
Updating lvfs
Downloading…             [  -                                    ]
Failed to load signature: Need more input

@travier
Copy link
Contributor

travier commented Sep 26, 2023

And it works again today (I no longer have the issue).

@rynofinn
Copy link

rynofinn commented Oct 4, 2023

Working with our CDN provider, we've verified that there is no difference between the files hosted in each region and the origin file. Does anyone still see the symptom at this time?

@cwrau
Copy link

cwrau commented Oct 5, 2023

Working with our CDN provider, we've verified that there is no difference between the files hosted in each region and the origin file. Does anyone still see the symptom at this time?

Currently /usr/bin/fwupdmgr refresh -v via systemd service results in exit code 2 with the following in the journal;

Details ```log 2023-10-05T10:05:23.992632+0200 steve systemd[1]: Starting Refresh fwupd metadata and update motd... 2023-10-05T10:05:24.016430+0200 steve fwupdmgr[2566873]: (/usr/bin/fwupdmgr:2566873): GLib-DEBUG: 10:05:24.016: setenv()/putenv() are not thread-safe and should not be used after threads are created 2023-10-05T10:05:24.019615+0200 steve fwupdmgr[2566873]: (/usr/bin/fwupdmgr:2566873): GLib-GIO-DEBUG: 10:05:24.019: _g_io_module_get_default: Found default implementation dconf (DConfSettingsBackend) for ‘gsettings-backend’ 2023-10-05T10:05:24.019615+0200 steve fwupdmgr[2566873]: (/usr/bin/fwupdmgr:2566873): dconf-DEBUG: 10:05:24.019: watch_fast: "/system/proxy/" (establishing: 0, active: 0) 2023-10-05T10:05:24.019824+0200 steve fwupdmgr[2566873]: (/usr/bin/fwupdmgr:2566873): dconf-DEBUG: 10:05:24.019: watch_fast: "/system/proxy/http/" (establishing: 0, active: 0) 2023-10-05T10:05:24.019856+0200 steve fwupdmgr[2566873]: (/usr/bin/fwupdmgr:2566873): dconf-DEBUG: 10:05:24.019: watch_fast: "/system/proxy/https/" (establishing: 0, active: 0) 2023-10-05T10:05:24.019887+0200 steve fwupdmgr[2566873]: (/usr/bin/fwupdmgr:2566873): dconf-DEBUG: 10:05:24.019: watch_fast: "/system/proxy/ftp/" (establishing: 0, active: 0) 2023-10-05T10:05:24.019913+0200 steve fwupdmgr[2566873]: (/usr/bin/fwupdmgr:2566873): dconf-DEBUG: 10:05:24.019: watch_fast: "/system/proxy/socks/" (establishing: 0, active: 0) 2023-10-05T10:05:24.019941+0200 steve fwupdmgr[2566873]: (/usr/bin/fwupdmgr:2566873): dconf-DEBUG: 10:05:24.019: unwatch_fast: "/system/proxy/" (active: 0, establishing: 1) 2023-10-05T10:05:24.019959+0200 steve fwupdmgr[2566873]: (/usr/bin/fwupdmgr:2566873): dconf-DEBUG: 10:05:24.019: unwatch_fast: "/system/proxy/http/" (active: 0, establishing: 1) 2023-10-05T10:05:24.019959+0200 steve fwupdmgr[2566873]: (/usr/bin/fwupdmgr:2566873): dconf-DEBUG: 10:05:24.019: unwatch_fast: "/system/proxy/https/" (active: 0, establishing: 1) 2023-10-05T10:05:24.019959+0200 steve fwupdmgr[2566873]: (/usr/bin/fwupdmgr:2566873): dconf-DEBUG: 10:05:24.019: unwatch_fast: "/system/proxy/ftp/" (active: 0, establishing: 1) 2023-10-05T10:05:24.019959+0200 steve fwupdmgr[2566873]: (/usr/bin/fwupdmgr:2566873): dconf-DEBUG: 10:05:24.019: unwatch_fast: "/system/proxy/socks/" (active: 0, establishing: 1) 2023-10-05T10:05:24.020061+0200 steve fwupdmgr[2566873]: (/usr/bin/fwupdmgr:2566873): dconf-DEBUG: 10:05:24.020: watch_established: "/system/proxy/" (establishing: 0) 2023-10-05T10:05:24.020097+0200 steve fwupdmgr[2566873]: (/usr/bin/fwupdmgr:2566873): dconf-DEBUG: 10:05:24.020: watch_established: "/system/proxy/http/" (establishing: 0) 2023-10-05T10:05:24.020125+0200 steve fwupdmgr[2566873]: (/usr/bin/fwupdmgr:2566873): dconf-DEBUG: 10:05:24.020: watch_established: "/system/proxy/https/" (establishing: 0) 2023-10-05T10:05:24.020155+0200 steve fwupdmgr[2566873]: (/usr/bin/fwupdmgr:2566873): dconf-DEBUG: 10:05:24.020: watch_established: "/system/proxy/ftp/" (establishing: 0) 2023-10-05T10:05:24.020197+0200 steve fwupdmgr[2566873]: (/usr/bin/fwupdmgr:2566873): dconf-DEBUG: 10:05:24.020: watch_established: "/system/proxy/socks/" (establishing: 0) 2023-10-05T10:05:24.020843+0200 steve fwupdmgr[2566873]: (/usr/bin/fwupdmgr:2566873): GLib-GIO-DEBUG: 10:05:24.020: _g_io_module_get_default: Found default implementation local (GLocalVfs) for ‘gio-vfs’ 2023-10-05T10:05:24.020875+0200 steve fwupdmgr[2566873]: (/usr/bin/fwupdmgr:2566873): pxbackend-DEBUG: 10:05:24.020: px_config_sysconfig_set_config_file: Could not read file /etc/sysconfig/proxy 2023-10-05T10:05:24.020914+0200 steve fwupdmgr[2566873]: (/usr/bin/fwupdmgr:2566873): pxbackend-DEBUG: 10:05:24.020: Active config plugins: 2023-10-05T10:05:24.020914+0200 steve fwupdmgr[2566873]: (/usr/bin/fwupdmgr:2566873): pxbackend-DEBUG: 10:05:24.020: - config-env 2023-10-05T10:05:24.020914+0200 steve fwupdmgr[2566873]: (/usr/bin/fwupdmgr:2566873): pxbackend-DEBUG: 10:05:24.020: - config-kde 2023-10-05T10:05:24.020914+0200 steve fwupdmgr[2566873]: (/usr/bin/fwupdmgr:2566873): pxbackend-DEBUG: 10:05:24.020: - config-gnome 2023-10-05T10:05:24.020914+0200 steve fwupdmgr[2566873]: (/usr/bin/fwupdmgr:2566873): pxbackend-DEBUG: 10:05:24.020: - config-sysconfig 2023-10-05T10:05:24.021717+0200 steve fwupdmgr[2566873]: (/usr/bin/fwupdmgr:2566873): GLib-GIO-DEBUG: 10:05:24.021: Failed to initialize portal (GNetworkMonitorPortal) for gio-network-monitor: Not using portals 2023-10-05T10:05:24.022054+0200 steve fwupdmgr[2566873]: (/usr/bin/fwupdmgr:2566873): GLib-GIO-DEBUG: 10:05:24.022: Using cross-namespace EXTERNAL authentication (this will deadlock if server is GDBus < 2.73.3) 2023-10-05T10:05:24.023299+0200 steve fwupdmgr[2566873]: (/usr/bin/fwupdmgr:2566873): GLib-GIO-DEBUG: 10:05:24.023: _g_io_module_get_default: Found default implementation networkmanager (GNetworkMonitorNM) for ‘gio-network-monitor’ 2023-10-05T10:05:24.023299+0200 steve fwupdmgr[2566873]: (/usr/bin/fwupdmgr:2566873): pxbackend-DEBUG: 10:05:24.023: px_manager_on_network_changed: Network connection changed, clearing pac data 2023-10-05T10:05:24.023346+0200 steve fwupdmgr[2566873]: (/usr/bin/fwupdmgr:2566873): pxbackend-DEBUG: 10:05:24.023: px_manager_constructed: Up and running 2023-10-05T10:05:24.023346+0200 steve fwupdmgr[2566873]: (/usr/bin/fwupdmgr:2566873): GLib-GIO-DEBUG: 10:05:24.023: _g_io_module_get_default: Found default implementation libproxy (GLibproxyResolver) for ‘gio-proxy-resolver’ 2023-10-05T10:05:24.026569+0200 steve fwupdmgr[2566873]: (/usr/bin/fwupdmgr:2566873): Fwupd-DEBUG: 10:05:24.026: Emitting ::status-changed() [idle] 2023-10-05T10:05:24.034378+0200 steve systemd[1]: fwupd-refresh.service: Deactivated successfully. 2023-10-05T10:05:24.043320+0200 steve systemd[1]: Finished Refresh fwupd metadata and update motd. ```

Nevermind, exit code 2 is success 😁

Then this looks like it's working now 👍

@PocketSam
Copy link

PocketSam commented Oct 5, 2023

Didn't meet this issue for more then a month. (Germany)
x-served-by: cache-fra-eddf8230108-FRA

@Amr-Ibra
Copy link

Amr-Ibra commented Oct 6, 2023

I'm affected too. From Germany.

@PocketSam
Copy link

Happened again today. :(
server: gunicorn
x-served-by: cache-fra-eddf8230139-FRA

@Lavode
Copy link

Lavode commented Oct 10, 2023

Same here. As before, main.xml.xz.jcat is the direct download from https://fwupd.org, cdn.xml.xz.jcat the one from https://cdn.fwupd.org (with x-served-by: cache-fra-eddf8230081-FRA). This time the last six bytes are missing:

› sha256sum *.jcat
b65851f9e3350d1103336d646ebde45f4f3daaf4ec0f97d9e4b550d1a18b2dda  cdn.xml.xz.jcat
9ff5334db828811fc9771a48beece4437e889207bbca4bd1bb81853ab7dae2d1  main.xml.xz.jcat

~
› du -b *.jcat
2223	cdn.xml.xz.jcat
2229	main.xml.xz.jcat

~
› colordiff <(xxd cdn.xml.xz.jcat) <(xxd main.xml.xz.jcat)
139c139,140
< 000008a0: 6f4a 7fb8 cc7c fdd7 3ff7 f7bf 014c 15    oJ...|..?....L.
---
> 000008a0: 6f4a 7fb8 cc7c fdd7 3ff7 f7bf 014c 15ea  oJ...|..?....L..
> 000008b0: 9ec4 0c00 00                             .....

And it's not just a single endpoint which returns the truncated file. I can repeatedly download it, hitting a different endpoint (at least as per x-served-by) each time, and all of them are truncated:

› curl -v -o cdn.xml.xz.jcat -L https://cdn.fwupd.org/downloads/firmware.xml.xz.jcat 2>&1 | grep -Ei '(x-served-by|connected to)' && sha256sum cdn.xml.xz.jcat
* Connected to cdn.fwupd.org (146.75.118.49) port 443
< x-served-by: cache-fra-eddf8230051-FRA
b65851f9e3350d1103336d646ebde45f4f3daaf4ec0f97d9e4b550d1a18b2dda  cdn.xml.xz.jcat

~
› curl -v -o cdn.xml.xz.jcat -L https://cdn.fwupd.org/downloads/firmware.xml.xz.jcat 2>&1 | grep -Ei '(x-served-by|connected to)' && sha256sum cdn.xml.xz.jcat
* Connected to cdn.fwupd.org (146.75.118.49) port 443
< x-served-by: cache-fra-eddf8230058-FRA
b65851f9e3350d1103336d646ebde45f4f3daaf4ec0f97d9e4b550d1a18b2dda  cdn.xml.xz.jcat

~
› curl -v -o cdn.xml.xz.jcat -L https://cdn.fwupd.org/downloads/firmware.xml.xz.jcat 2>&1 | grep -Ei '(x-served-by|connected to)' && sha256sum cdn.xml.xz.jcat
* Connected to cdn.fwupd.org (146.75.118.49) port 443
< x-served-by: cache-fra-eddf8230126-FRA
b65851f9e3350d1103336d646ebde45f4f3daaf4ec0f97d9e4b550d1a18b2dda  cdn.xml.xz.jcat

~
› curl -v -o cdn.xml.xz.jcat -L https://cdn.fwupd.org/downloads/firmware.xml.xz.jcat 2>&1 | grep -Ei '(x-served-by|connected to)' && sha256sum cdn.xml.xz.jcat
* Connected to cdn.fwupd.org (146.75.118.49) port 443
< x-served-by: cache-fra-eddf8230059-FRA
b65851f9e3350d1103336d646ebde45f4f3daaf4ec0f97d9e4b550d1a18b2dda  cdn.xml.xz.jcat

~
› curl -v -o cdn.xml.xz.jcat -L https://cdn.fwupd.org/downloads/firmware.xml.xz.jcat 2>&1 | grep -Ei '(x-served-by|connected to)' && sha256sum cdn.xml.xz.jcat
* Connected to cdn.fwupd.org (146.75.118.49) port 443
< x-served-by: cache-fra-eddf8230116-FRA
b65851f9e3350d1103336d646ebde45f4f3daaf4ec0f97d9e4b550d1a18b2dda  cdn.xml.xz.jcat

~
› curl -v -o cdn.xml.xz.jcat -L https://cdn.fwupd.org/downloads/firmware.xml.xz.jcat 2>&1 | grep -Ei '(x-served-by|connected to)' && sha256sum cdn.xml.xz.jcat
* Connected to cdn.fwupd.org (146.75.118.49) port 443
< x-served-by: cache-fra-eddf8230114-FRA
b65851f9e3350d1103336d646ebde45f4f3daaf4ec0f97d9e4b550d1a18b2dda  cdn.xml.xz.jcat

~
› curl -v -o cdn.xml.xz.jcat -L https://cdn.fwupd.org/downloads/firmware.xml.xz.jcat 2>&1 | grep -Ei '(x-served-by|connected to)' && sha256sum cdn.xml.xz.jcat
* Connected to cdn.fwupd.org (146.75.118.49) port 443
< x-served-by: cache-fra-eddf8230111-FRA
b65851f9e3350d1103336d646ebde45f4f3daaf4ec0f97d9e4b550d1a18b2dda  cdn.xml.xz.jcat

@leture
Copy link

leture commented Oct 10, 2023

Hi,

also experience this issue under Manjaro.

I switched in /etc/fwupd/remotes.d/lvfs.conf from MetadataURI=https://cdn.fwupd.org/downloads/firmware.xml.xz to MetadataURI=https://fwupd.org/downloads/firmware.xml.xz and this solved the problem.

@davidlt
Copy link

davidlt commented Oct 10, 2023

Happened to me on Fedora 38 too. Removing cdn solved the problem as suggested in a number of replies. It would be great if it could re-try without CDN by default.

@hughsie
Copy link
Member

hughsie commented Oct 10, 2023

It would be great if it could re-try without CDN by default.

That would destroy the main server any time the CDN went wrong. Outgoing bandwidth in AWS is expensive. :/

@hughsie
Copy link
Member

hughsie commented Oct 10, 2023

I guess the hacky workaround is to just to append 6 bytes of "XXXXXX" to the JCAT file -- libjcat seems to parse it just fine with the extra data appended and then if the CDN did truncate the file then ¯_(ツ)_/¯

@hughsie
Copy link
Member

hughsie commented Oct 10, 2023

Can anyone affected go back to the CDN and try again please? There's 6 bytes of junk on the end of https://cdn.fwupd.org/downloads/firmware.xml.xz.jcat now

@Lavode
Copy link

Lavode commented Oct 10, 2023

I guess the hacky workaround is to just to append 6 bytes of "XXXXXX" to the JCAT file -- libjcat seems to parse it just fine with the extra data appended and then if the CDN did truncate the file then ¯_(ツ)_/¯

I doubt that's sustainable. Last time the file was missing four bytes, this time it's six. Next time it might be eight, and you're back to square one. And that's not to mention just /how awful/ of a hack that is to begin with. ;)

That aside - works for me now. Main website & CDN deliver the same file for now - including the IH8CDN postfix. 😁

@hughsie
Copy link
Member

hughsie commented Oct 10, 2023

including the IH8CDN postfix

That was supposed to be an easter egg! :)

@leture
Copy link

leture commented Oct 10, 2023

Can anyone affected go back to the CDN and try again please? There's 6 bytes of junk on the end of https://cdn.fwupd.org/downloads/firmware.xml.xz.jcat now

Switching back to CDN has worked for me.

@rynofinn
Copy link

Thanks for helping us to track this down. If you are affected still, can you post the output of these two commands:
curl -v https://cdn.fwupd.org/downloads/firmware.xml.xz.jcat
curl -v https://fwupd.org/downloads/firmware.xml.xz.jcat

@hughsie
Copy link
Member

hughsie commented Oct 10, 2023

If this hack, cough, workaround, fixed the issue for you please say. If it's still broken, please shout louder!

@lordtaytertot
Copy link

lordtaytertot commented Oct 30, 2023

@hughsie, I had to use @leture's fix: #6032 (comment) to get things working this morning.

@DomiStyle
Copy link

DomiStyle commented Nov 14, 2023

My Discover on Fedora 39 KDE just started throwing this error today, same from the command line.

Checksums are different between CDN and direct download:

f7eeea093ef8edee01413a38ee3d9507  cdn.xml.xz.jcat
9e1820f33cda3403fdcde480bafe828c  direct.xml.xz.jcat

curl output to cdn.fwupd.org: https://gist.github.com/DomiStyle/5814ef8ffe082ee56098138437622274
curl output to fwupd.org: https://gist.github.com/DomiStyle/acf60e69d14b10c570b4f4e8ce5046ea

edit: File endings compared:
Screenshot_20231114_101548
Screenshot_20231114_101559

@senden9
Copy link

senden9 commented Nov 14, 2023

Same problem for me again.

Updating lvfs
Downloading…             [        |                              ]
Failed to load signature: Need more input

According to curl my data package come from cache-vie6359-VIE. For debugging reasons here the full output of curl with and without cdn:

➜  ~ curl -v --output cdn https://cdn.fwupd.org/downloads/firmware.xml.xz.jcat
* processing: https://cdn.fwupd.org/downloads/firmware.xml.xz.jcat
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0*   Trying [2a04:4e42:200::561]:443...
* Connected to cdn.fwupd.org (2a04:4e42:200::561) port 443
* ALPN: offers h2,http/1.1
} [5 bytes data]
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
} [512 bytes data]
*  CAfile: /etc/ssl/certs/ca-certificates.crt
*  CApath: /etc/ssl/certs
{ [5 bytes data]
* TLSv1.3 (IN), TLS handshake, Server hello (2):
{ [104 bytes data]
* TLSv1.2 (IN), TLS handshake, Certificate (11):
{ [2818 bytes data]
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
{ [300 bytes data]
* TLSv1.2 (IN), TLS handshake, Server finished (14):
{ [4 bytes data]
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
} [37 bytes data]
* TLSv1.2 (OUT), TLS change cipher, Change cipher spec (1):
} [1 bytes data]
* TLSv1.2 (OUT), TLS handshake, Finished (20):
} [16 bytes data]
* TLSv1.2 (IN), TLS handshake, Finished (20):
{ [16 bytes data]
* SSL connection using TLSv1.2 / ECDHE-RSA-CHACHA20-POLY1305
* ALPN: server accepted h2
* Server certificate:
*  subject: CN=cdn.fwupd.org
*  start date: Jul  8 19:45:12 2023 GMT
*  expire date: Aug  8 19:45:11 2024 GMT
*  subjectAltName: host "cdn.fwupd.org" matched cert's "cdn.fwupd.org"
*  issuer: C=BE; O=GlobalSign nv-sa; CN=GlobalSign Atlas R3 DV TLS CA 2023 Q2
*  SSL certificate verify ok.
} [5 bytes data]
* using HTTP/2
* h2 [:method: GET]
* h2 [:scheme: https]
* h2 [:authority: cdn.fwupd.org]
* h2 [:path: /downloads/firmware.xml.xz.jcat]
* h2 [user-agent: curl/8.2.1]
* h2 [accept: */*]
* Using Stream ID: 1
} [5 bytes data]
> GET /downloads/firmware.xml.xz.jcat HTTP/2
> Host: cdn.fwupd.org
> User-Agent: curl/8.2.1
> Accept: */*
> 
{ [5 bytes data]
< HTTP/2 200 
< content-type: application/octet-stream
< server: gunicorn
< content-disposition: inline; filename=firmware.xml.xz.jcat
< last-modified: Tue, 14 Nov 2023 04:32:10 GMT
< cache-control: public, max-age=14400
< expires: Tue, 14 Nov 2023 12:32:20 GMT
< etag: "1699936330.383-2228-4227534271"
< accept-ranges: bytes
< date: Tue, 14 Nov 2023 09:24:50 GMT
< via: 1.1 varnish
< age: 3150
< x-served-by: cache-vie6359-VIE
< x-cache: HIT
< x-cache-hits: 67
< content-length: 2228
< 
{ [2228 bytes data]
100  2228  100  2228    0     0  19657      0 --:--:-- --:--:-- --:--:-- 19716
* Connection #0 to host cdn.fwupd.org left intact
➜  ~ curl -v --output main https://fwupd.org/downloads/firmware.xml.xz.jcat
* processing: https://fwupd.org/downloads/firmware.xml.xz.jcat
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0*   Trying [2600:1f14:414:5601::7701]:443...
* Connected to fwupd.org (2600:1f14:414:5601::7701) port 443
* ALPN: offers h2,http/1.1
} [5 bytes data]
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
} [512 bytes data]
*  CAfile: /etc/ssl/certs/ca-certificates.crt
*  CApath: /etc/ssl/certs
{ [5 bytes data]
* TLSv1.3 (IN), TLS handshake, Server hello (2):
{ [104 bytes data]
* TLSv1.2 (IN), TLS handshake, Certificate (11):
{ [4951 bytes data]
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
{ [333 bytes data]
* TLSv1.2 (IN), TLS handshake, Server finished (14):
{ [4 bytes data]
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
} [70 bytes data]
* TLSv1.2 (OUT), TLS change cipher, Change cipher spec (1):
} [1 bytes data]
* TLSv1.2 (OUT), TLS handshake, Finished (20):
} [16 bytes data]
* TLSv1.2 (IN), TLS handshake, Finished (20):
{ [16 bytes data]
* SSL connection using TLSv1.2 / ECDHE-RSA-AES128-GCM-SHA256
* ALPN: server accepted h2
* Server certificate:
*  subject: CN=fwupd.org
*  start date: Jul 21 00:00:00 2023 GMT
*  expire date: Aug 18 23:59:59 2024 GMT
*  subjectAltName: host "fwupd.org" matched cert's "fwupd.org"
*  issuer: C=US; O=Amazon; CN=Amazon RSA 2048 M02
*  SSL certificate verify ok.
{ [5 bytes data]
* using HTTP/2
* h2 [:method: GET]
* h2 [:scheme: https]
* h2 [:authority: fwupd.org]
* h2 [:path: /downloads/firmware.xml.xz.jcat]
* h2 [user-agent: curl/8.2.1]
* h2 [accept: */*]
* Using Stream ID: 1
} [5 bytes data]
> GET /downloads/firmware.xml.xz.jcat HTTP/2
> Host: fwupd.org
> User-Agent: curl/8.2.1
> Accept: */*
> 
{ [5 bytes data]
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0< HTTP/2 200 
< date: Tue, 14 Nov 2023 09:25:59 GMT
< content-type: application/octet-stream
< content-length: 2236
< server: gunicorn
< content-disposition: inline; filename=firmware.xml.xz.jcat
< last-modified: Tue, 14 Nov 2023 08:32:19 GMT
< cache-control: public, max-age=14400
< expires: Tue, 14 Nov 2023 13:25:59 GMT
< etag: "1699950739.429-2236-4227534271"
< 
{ [2236 bytes data]
100  2236  100  2236    0     0   2602      0 --:--:-- --:--:-- --:--:--  2600
* Connection #0 to host fwupd.org left intact

(I would have used spoiler / details tags for collapsible output but this seems to destroy the codeblock formating on Github.)

I confirm the observation of @DomiStyle. Exactly the NULL NULL IH8CDN end is missing in the end of the CDN file:
image

@hughsie
Copy link
Member

hughsie commented Nov 14, 2023

In the code:

    # it seems some CDNs strip off the gzip 8B trailer :/
    if cdn_compat:
        data += b"IH8CDN"

In retrospect 8B stands for 8 BYTE and IH8CDN isn't long enough. Give me 1 h.

@hughsie
Copy link
Member

hughsie commented Nov 14, 2023

Can you both try now please.

@DomiStyle
Copy link

@hughsie Looks good now! Checksums match and fwupdmgr doesn't complain anymore.

@senden9
Copy link

senden9 commented Nov 14, 2023

Thanks for the fast respones @hughsie. I can confirm fwupd it works again. Also at the moment the output of CDN and main are Identical. So Cloudflare does not cut off the end.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Development

No branches or pull requests