-
-
Notifications
You must be signed in to change notification settings - Fork 54
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Version 1.2 and some guidance on setup/making the embed plugin and make it work consistently #26
Comments
Thanks for the detailed report. There's a couple of issues, some on my end, some on my the Archipelago end :)
|
- update to warcio 1.3.2, better handling of WARCs with incorrect content-length, should fix webrecorder/replayweb.page#23 - loading: dedupResource() uses put() instead of add() to avoid aborting transaction with duplicate add, fixes part of webrecorder/replayweb.page#26 - statuscodes: update to latest http-status-codes, wrap getStatusText() to ensure never throws, returns 'Unknown Status' if unknown
- update to warcio 1.3.2, better handling of WARCs with incorrect content-length, should fix webrecorder/replayweb.page#23 - loading: dedupResource() uses put() instead of add() to avoid aborting transaction with duplicate add, fixes part of webrecorder/replayweb.page#26 - statuscodes: update to latest http-status-codes, wrap getStatusText() to ensure never throws, returns 'Unknown Status' if unknown bump version to 2.2.2
@DiegoPino Released v1.3.0, which should fix the AbortError issue, even without range request support -- it will load slowly, but it should load w/o errors now. |
@ikreymer thanks for your detailed response. We are having some trouble finding the right balance between "security", flexibilty and speed right now. I managed to get HEAD requests for the WACZ implementation working but hitting some resource limits when trying to seek/deliver the range request afterwards (PHP/AWS S3 SDK are playing with my patience on streams and memory usage). I will test V1.3.0 There is still one issue that seems to be affecting is that some CSS/Images are being handled differently (no clue why) on WARC files and end not being served. E.g here https://webarchive.archipelago.nyc/do/db17b0d6-886b-4ee4-bfb9-0edf9ce404b5 In this case missing CSS is consistent for the landing page But for the first example I shared (1 Gbyte WARC) Will do my homework first and get Ranges working without killing the server! Thanks! |
Try updating to 1.3.0 -- The AbortError would cause certain resources to not be loaded, hopefully this is fixed now. Sorry to hear about the difficulties with streaming! I would definitely recommend using the S3 presigned URLs with a reasonable duration (a day?), then you do not need to worry about local cacheing at all! That should work pretty well, you'll just need to configure CORS settings on the bucket, which I can help you with also. Here's the WARC you shared loading from DigitalOcean CDN, it takes some time, but does load: |
I think all the issues mentioned here have now been resolved and the embedding is working! |
Hi @ikreymer @emmadickson
I know you guys are busy with WACZ but wanted to catch up with some issues we have been having on the embed version of replay web on Archipelago with version 1.2
I suspect a lot of this is because we are CDN loading the JS but also because the files we are testing agains are "largish" and also pure WARC. But we may have other issues so open to suggestions.
This URL , a 1 GByte WARC file eventually loads on Safari (slow), I see small pauses made every 1000 records and I get a lot of restarts and failed attempts (inclusive from the Client reloading the whole page) and surprisingly loads faster (still a few minutes() and more resilient on Chrome but without any CSS/Images/assets
?stream=true
) to enable modal during testing. This basically is read 1024 bytes from S3+ pass 1024 back to the HTTP request (I may chunk this larger?) Sadly can not use GET argument that in production because the embed tag fails if the "source" property is not just actual end in a valid file extension (not a big issue, modified it to always stream fro WARC and WACZ files in that first URL i shared). But know I have second thoughts. Is streaming what you need/works better? or is chunking better for your JS? Also, does the stream need to be seek-able?To test a direct download of the stream please test this url (reusing or IIIF endpoint, please dismiss the weird semantic here)
https://webarchive.archipelago.nyc//do/4/iiif/51b281b4-093e-494c-9820-9eeeb03a4c6e/full/full/0/default.warc
e.g wget takes (942.50M 26.7MB/s in 39s ), replay embed 5 minutes of more on Chrome.
And It restarts. Sometimes it works, sometimes not.
E.g Safari
Also should we be worried about the initial message (since running from CDN)
We are loading ui.js via CDN and it works.
FYI we are running NGINX, and file delivery is not directly S3, (access control + some other users may be using Azure or directly filesystem so we wrap things. maybe we need to tune our Binary responses?
Sorry for the "cover it all" issue but I feel its more like a use case sharing and for sure using just WACZ should solve all the issues, but for now I want to be sure its not us/something we can do better
Thanks for the great work!!
The text was updated successfully, but these errors were encountered: