Stake Pool metadata fetch error exceeded 512 bytes

[rhyslbw]

OS
Your OS: Ubuntu

Versions
The db-sync version (eg cardano-db-sync --version): 13.0.4
PostgreSQL version: 11.5

Build/Install Method
The method you use to build or install cardano-db-sync: Image from Dockerhub registry

Run method
The method you used to run cardano-db-sync (eg Nix/Docker/systemd/none): Docker

Additional context
Logs and data provided from Preview network

Problem Report
There are a number of pool_offline_fetch_error rows after syncing, such as:

Offline pool data when fetching metadata from https://adavault.com/advpreview.json exceeded 512 bytes.

This particular file is 166 B:

{
  "name": "ADV",
  "ticker": "ADV",
  "description": "ADAvault Preview",
  "homepage": "https://adavault.com/",
  "nonce": "1662388686",
  "extended": "https://adavault.com/extended.json"
}

I've also noticed there are no rows in pool_offline_data that have the extended property in json. Could this be causing the failure?

dbsync_preview=# select fetch_error from pool_offline_fetch_error where fetch_error not like '%Invalid URL%';
                                                                                                                        fetch_error
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
 Hash mismatch from when fetching metadata from https://preview-metadata.panl.org/. Expected 18dc49aa5e66edc78804bd8b9cf15d7e5e24616eeeede309c6cda7724fb77995 but got a2d8a47fc4d3a6e707487b65b3ab790e6fff71c2afd72f0fe17bc5e523024019.
 Hash mismatch from when fetching metadata from https://testforge.adamantium.online/BOC.metadata.json. Expected b3aae9c1f0f102dca23bea880d3f67965e252adf3e9bbdededda9cf35bfd4f93 but got ebee2c8560ee409510ee81de11ee90d81413ca80abe96127fb92a5980cfcaad1.
 HTTP Exception for https://938ae6dc.cardano-metadata.stagebison.net/metadata.json resulted in : InternalException (HandshakeFailed (Error_Protocol ("certificate has unknown CA",True,UnknownCa)))
 Offline pool data when fetching metadata from https://shorturl.at/hilxY exceeded 512 bytes.
 Hash mismatch from when fetching metadata from https://rb.gy/mkohxw. Expected 978f09fa7d780747a0232f3073e703198e594732e23b5292427500f0c4502bf2 but got 614f72a980f99d98326c55fa572651c179012c85291171b7f1e341e2c0f11817.
 Hash mismatch from when fetching metadata from https://dpadapools.com/testnet3previewmeta.json. Expected 031b78f8bc3af4d6516c27a9328aa67188445fb28907830f5e9aef448f126fc8 but got b7c9e34af2473e6bd7cd9d3b45a61f7b6fd56543257052167bef91963834367d.
 Hash mismatch from when fetching metadata from https://test.lidonation.com/metadata.json. Expected ad72fe7c3c1b0be70596ae2fb693392e3040079d3ccd5f8b06127ec3acd75b33 but got b552338f84a5971eadc8b130e9bf0dcc46894a7ea80c688d5f4a95a49118d716.
 Connection failure when fetching metadata from https://foo.bat/poolmeta.json'.
 Offline pool data when fetching metadata from https://shorturl.at/gTY23 exceeded 512 bytes.
 Offline pool data when fetching metadata from https://adavault.com/advpreview.json exceeded 512 bytes.
 Hash mismatch from when fetching metadata from http://seahubs.com/public/zwish/metadata.json. Expected e1abbc95b43686a1de5b29299c2a172033f2962130a56fc0921e2958fa8bde22 but got ae2ce3bded573ecdfd762d8bddb6ad8ce5b4b464a387170a6dc4c8bdbe44a285.
 Offline pool data when fetching metadata from https://shorturl.at/gTY23 exceeded 512 bytes.
 Offline pool data when fetching metadata from https://adavault.com/advpreview.json exceeded 512 bytes.
 Hash mismatch from when fetching metadata from http://seahubs.com/public/zwish/metadata.json. Expected e1abbc95b43686a1de5b29299c2a172033f2962130a56fc0921e2958fa8bde22 but got ae2ce3bded573ecdfd762d8bddb6ad8ce5b4b464a387170a6dc4c8bdbe44a285.
 Hash mismatch from when fetching metadata from https://cutt.ly/tCQuZMK. Expected f8ecea81a26f9d72c3a655462c4c141d3b4f134a771dfd72b17c847412fa2f62 but got 6e60d3eae613b5320acc2d741a8562424f3aba316bfccd46660fcb4b2e0699dd.
 Connection failure when fetching metadata from https://foo.bat/poolmeta.json'.
 Hash mismatch from when fetching metadata from https://cutt.ly/tCQuZMK. Expected e42e3f3f3cf9738335e7df58f838f42550deb387dfbc05e4bfcf7f8e21eb409f but got 6e60d3eae613b5320acc2d741a8562424f3aba316bfccd46660fcb4b2e0699dd.

[erikd]

The extended property could not possibly be the issue, because as you say its under 512 bytes and at the time of that error, its only a ByteString. It has not been decoded to JSON yet.

[HT-Moh]

Hello guys my pool ticker CCYT seem to be impacted by this issue as well.

[erikd]

@HT-Moh And please, cut and paste the actual text rather than a screenshot.

Looking up the latest PoolMetadataRef entry for your pool and using the new debug/test program http-get-json-metadata (not yet on master) I get:

> cabal run -- http-get-json-metadata https://raw.githubusercontent.com/HT-Moh/cm/main/pm.json
Up to date
Warning: This should be 'application/json'
Content-type: text/plain; charset=utf-8
{"description":"We are two cardanians - Moh & Flu - we operate our ADA Single Pool hosted in Switzerland
 with great expertise and security, I develop on cardano ecosystem everyday and with Flu we run our
  educational blog about blockchain technology focused on Cardano.","homepage":
  "https://crypto-catalyst.com/","name":"CCATALYST","ticker":"CCYT"}

[HT-Moh]

@erikd sorry for that.
The links are:
https://raw.githubusercontent.com/HT-Moh/cm/main/pm.json
https://raw.githubusercontent.com/HT-Moh/cm/main/pme.json
I think I need to host this elsewhere and respond with 'application/json' ?

[erikd]

The current JSON and content-tye is fine, but longer term, yes, it would be good to host this in a place that will provide an application/json content type.

[rdlrt]

@erikd @kderme - are there plans to include this in a release (or even dedicated interim tag perhaps) that we can use soon?

Pool explorers relying on Koios are impacted with this issue, and would be nice to get a consistent tag to use for distributed instances behind🙂.

[erikd]

The commit with this fix is on master and the master schema is slightly different from that of the release/13.0.x branch. These changes could possibly be back ported to the release branch, but there are other issues as well (eg #1277) that may require a new release.

[rdlrt]

Right, but it seems (not sure) #1277 might be a lot of work(?) - even from testing pov. I was hoping we could have a cherry-picked tag (@reqlez created one here using entire PR commits for testing), which could be a quick interim solution

[reqlez]

Yea i'm running my "hack" version right now for testing purposes ( https://github.com/cardano-bsd-alliance/cardano-db-sync/releases/tag/13.0.5a ) I hope i did it right lol had to resolve some conflict. Right now on epoch 307 on that DB, syncing from genesis.

So far I only see a single 512bytes+ error, and it's actually legit:

cexplorer=# select fetch_error from pool_offline_fetch_error where fetch_error like '%exceeded 512 bytes%';
fetch_error

Offline pool data when fetching metadata from https://git.io/J3K2M exceeded 512 bytes.
(1 row)