Split: Naming - move into transports then refactor

[mitra42]

• Creating naming.js
• integrate into Naming.js
• in "dweb" repo, remove need for bootloader
• Test on dweb.me
• remove register.js and dependencies on dweb-objects from dweb-transport
• Clean up dweb-objects see issue149
• Remove objects from dweb-mirror See Remove remnants of dweb-objects dweb-mirror#250
• move to /metadata from /arc/archive.org/metadata etc (see below)
• Catch https://dweb.me and https://dweb.archive.org and https://archive.org in namer for dweb-mirror as well as dweb:
• Move to https://dweb.archive.org instead of dweb: see below
• Check for documentation of naming in this repo and google docs

[mitra42]

Part of #20

[mitra42]

• Consider Making naming a set of functions that can be applied in turn
  • mapping https://archive.org -> wolk, gun etc

Naming.p_namingcb
Stored in Transports.namingcb
< p_resolveNames array -> array
< top of most of Transports.xxx

[mitra42]

• Should be able to apply different set of naming on Mirror e.g. https://archive.org/metadata/x -> ${DwebArchive.mirror}/metadata/x
  Moved to Naming: localhost #27

[mitra42]

• Moving from /arc/archive.org/xxx to /xxx

• scan all code for /arc/archive.org: DM DA DAC IAUX DTS

• Forward of /arc/archive.org/* to /* replaces more specific forwards

• Check Naming.js (and then rescan DTS for /arc/archive.org)

• DA.ReactSupport.ReactConfig.tabbyrootinsert - hard links to dweb, unused, removed

• DAC.Util.gateway urls - see below

[mitra42]

• remove most references to dweb:
  • Scan all code
    • DM
    • DA
    • DAC
    • IAUX

[mitra42]

• Look for /thumbnail and replace with /services/img

[mitra42]

• Remove references to dweb.me where possible
  • See DAC.upstreamPrefix which specifies dweb.me:/arc/archive.org
  • Scan:, IAUX, DTS
  • Look for use in DM of config.archiveui.upstream
  • Look for use in DM of httpServer (as passed to DM.init
• use dweb.archive.org/ where possible

[mitra42]

• DAC.Util.gateway urls - find where used and how

• scan code turning up in DAC, DM DA

• url_related in AIP.fetch_related & AI.relatedItems

• url_related_local AI.relatedItems

• url_servicesimg in

  • AIP.saveThumbnail - needs hand thinking, prob just use mapping in Naming.js
  • AM.httpUrl via gatewayServer

• urlDownload MFS.seed DA.DetailsPage.DetailsIAWrap x2

• url_default_fl n/a its a field list

• url_playlist_local AI._fetch_metadata

• url_metadata AF.urls

• url_advancedsearch DAC.Util._query via gatewayserver

• gatewayServer()

  • AI._fetch_bookreader needs special handlign as passed this.server
  • DA.ReactSupport.transportStatusAndProps calls gatewayServer/info which will fail on dweb.archive.org
  • _transportsOptions.http.urlBase - which will be wrong if gatewayServer = dweb.archive.org (I think)

[mitra42]

Cleanup

• Scan for TODO-GATEWAY
• look for localhost:4244 should (almost) always be {DwebArchive.mirror} : 4244
• Check edge case task A set of edge cases for testing dweb-archive#120
• Push to dweb.archive.org
• THEN retest all edge cases on dweb.archive.org

[mitra42]

Debugging:

• still heartbeat on https://dweb.me/info - fix when pipeline running
• https://localhost:4244/details/commute fails (offline - but files cached) fails complaining about https://archive.org/metadata/commute
  • why is it going to archive.org ?
• http://localhost:4244/details/ThirdAve1950 fails
  • DM https://www-dweb-metadata.dev.archive.org/metadata/cd_ballad-on-third-avenue_ed-hale 404 see dweb-metadata fails if not torrent dweb-mirror#281
  • console: http://localhost:4244/download/ThirdAve1950/ThirdAve1950.thumbs/ThirdAve1950_000030.jpg 404
  • http://localhost:4244/services/img/78_sax-third-avenue_jimmy-james-collins-h.-driggs-bernie-mattinson_gbia0006619b
• http://localhost:4244/details/ISS039-E-003344

[mitra42]

Debugging on dweb.archive.org (code loaded locally)

• http://localhost:4244/archive/archive.html?mirror=&transport=HTTP&identifier=commute
  • fails reading language files via dweb.archive.org with no cors
  • Unable to fetch metadata for commute Cannot read property 'archive.org' of undefined
  • CORS on dweb-metadata

[mitra42]

Debugging from edge case tests at internetarchive/dweb-archive#120

• http://localhost:4244/details/I_am_too_sexy is_dark - see DM log for failure

[mitra42]

Tests on dweb.archive.org
(W) needs webpack (P) needs push/pipeline (D) deploying (T) needs test

• [W] GUN failing to load scripts
• forwarding to dweb.archive.org
• Getting manage.css via http (from https page) fail.
  • Need a permanent fix to this - had to hack play8.min.js
• Scan console.log - has numerous dweb.me in it - they were old thumbnaillinks
• On "home" advancedsearch for home not working - https://dweb.archive.org/advancedsearch?out... seems to go to https://archive.org/advancedsearch?outpu... which fails (advancedsearch.php) works
• On "home" advancedsearch
• Thumbnail errors see Naming: cors and 403 errors on /services/img #26
• on "commute" RelatedItems getting bad images

[mitra42]

Running through test cases on internetarchive/dweb-archive#120

• @mitra - cant find images/notfound.png :-)
• JourneysInPersiaAndKurdistanVolII and EnglishTeluguDictionary work on dweb.archive.org, retest on localhost

[mitra42]

https://dweb.archive.org/details/@mitra -> notfound.png -> image not found

• why is image in component being set to "/images/Broken_document.png" if that is correct, then should use "notfound" image in /images
• why are collection images going to https://archive.org/archive/images/notfound.png instead of https://archive.org/images/notfound.png OR is it going to dweb.archive.org and getting forwarded ?
• Image on left of person goes to gravatar e.g. https://web.archive.org/web/https://www.gravatar.com/avatar/a94f02232c2eb5444f0dce001e652cc0 - need to understand that.
  Accounts page - Image via Gravatar dweb-archive#172
• https://dweb.archive.org/details/@mitra doesnt have any images on the actual collections
  See Collection with no image should show no image, rather than not-found dweb-archive#171
• ReactSupport._imgUrlOrStream should return URL if single Http url

[mitra42]

• http://localhost:4244/details/prelinger has a SortBy dropdown but https://dweb.archive.org/details/prelingera shows a "Object Object". Testing with http://localhost:4244/archive/archive.html?mirror=&transport=HTTP&identifier=prelinger

[mitra42]

All above done or with own issues