feat: wikipedia_uk_all_maxi_2022-03 and wikipedia_ru_all_maxi_2022-03

[lidel]

Assuming scripts still work, this PR will be updating Russian snapshot to the latest version.

TODO

Russian

• confirm build still works with wikipedia_ru_all_maxi_2022-02.zim
• build wikipedia_ru_all_maxi_2022-03
  • imo worth the wait
  • build in progress https://farm.openzim.org/pipeline/43a68a1e8ab6513c8ebad126
    • Finished crawling, now downloading files [566780/2235800] [25.4%]
  • takes ~4days on average, but could be more: https://farm.openzim.org/recipes/wikipedia_ru_all
• pin to wikipedia.collab.ipfscluster.io
• pin to web3.storage
  • 💢 unable to list queued pins via ipfs pin remote ls → missing on result list Remote pin ends up in "invisible" queued state web3-storage/web3.storage#1102
  • 💢 no alternative (no way to track remote pin of a CID in UI)
• pin to estuary
  • 💢 unable to list queued pins via ipfs pin remote ls → invalid response missing entire results list Invalid pin listing response: {"count":15,"results":null} application-research/estuary#124
  • 💢 no alternative (no way to track remote pin of a CID in UI)
• update DNSLink - https://github.com/protocol/infra/pull/805

Ukrainian

• build wikipedia_uk_all_maxi_2022-03.zim
• pin to wikipedia.collab.ipfscluster.io
• pin to web3.storage
• pin to estuary
• update DNSLink - https://github.com/protocol/infra/pull/798

Belarusian

• build wikipedia__all_maxi_2022-03.zim
  • sadly relative links are broken, needs analysis
  • unsure if worth the effort, according to https://en.wikipedia.org/wiki/Belarusian_Wikipedia Russian is more useful:

    As of March 2014, the most popular editions of Wikipedia within Belarus are the Russian Wikipedia (with 84.0% of all page views) followed by the English Wikipedia (10.2%).[10] The Belarusian Wikipedia has much fewer page views, because of the Russification of Belarus.[citation needed]

[lidel]

• Added successful Ukrainian mirror based on wikipedia_uk_all_maxi_2022-03.zim
• Belarusian build had broken relative paths and I had no bandwidth to look into it atm
• Russian one for 2022-03 crawl is 93.6% done – waiting https://farm.openzim.org/pipeline/43a68a1e8ab6513c8ebad126 (it then needs to download files, which will be another day)

[lidel]

Building wikipedia_ru_all_maxi_2022-03

[lidel]

Done.

@hsanjuan are you able to add these new snapshots to wikipedia.collab.ipfscluster.io?
(listed at https://collab.ipfscluster.io, but i dont remember what was the process for adding new ones)

• bafybeibiqlrnmws6psog7rl5ofeci3ontraitllw6wyyswnhxbwdkmw4ka → wikipedia_uk_all_maxi_2022-03.zim
• bafybeiezqkklnjkqywshh4lg65xblaz2scbbdgzip4vkbrc4gn37horokq → wikipedia_ru_all_maxi_2022-03.zim

(I already updated DNSLinks, now we need to pin it in more places)

[hsanjuan]

• bafybeibiqlrnmws6psog7rl5ofeci3ontraitllw6wyyswnhxbwdkmw4ka → wikipedia_uk_all_maxi_2022-03.zim

• bafybeiezqkklnjkqywshh4lg65xblaz2scbbdgzip4vkbrc4gn37horokq → wikipedia_ru_all_maxi_2022-03.zim

Pinning

[hsanjuan]

The first one was pinned by web3.storage cluster, but the second one seems stuck (8 failed attempts).

[BigLep]

@lidel : I'm giong to put @galargh given his deep dive around this for #123 . Feel free to assign someone else.

[galargh]

Review checklist

• Verified the new README.md instructions by following them in feat: Move mirror creation to the cloud #123
• Verified the new 🇺🇦 ipfs URL by visiting https://bafybeibiqlrnmws6psog7rl5ofeci3ontraitllw6wyyswnhxbwdkmw4ka.ipfs.dweb.link
• Verified the new 🇷🇺 ipfs URL by visiting https://bafybeiezqkklnjkqywshh4lg65xblaz2scbbdgzip4vkbrc4gn37horokq.ipfs.dweb.link

[lidel]

uk and ru are pinned to both web3 and estuary, merging!