URL rewriting: Youtube JS player fails when we URL-encode `=` and `,` characters

[benoit74]

When rewriting URLs found in documents (from zimit typically), we have a specific logic to compute URLs to use in documents (HTML mostly) for safe operation inside the ZIM.

This logic considers that we should URL-encode any character found in the URL path except the /:

python-scraperlib/src/zimscraperlib/rewriting/url_rewriting.py

Lines 334 to 337 in 295fefc

	return (
	f"{quote(relative_path, safe='/')}"
	f"{'#' + item_fragment if item_fragment else ''}"
	)

This causes an issue to replay Youtube videos because the player JS seems to "interpret" path segments found in its URL.

For instance we could have a JS script at www.youtube.com/s/_/ytembeds/_/js/k=ytembeds.base.en_US.7J4wKKTzgAI.es5.O/am=AAAAgA/d=1/excm=base,root/br=1/rs=AGKMywEpP5H1eAyamKTuJTnAwxE-ZPBAgA/m=root,base.

Currently the URL to load this is rewritten to something like ../../www.youtube.com/s/_/ytembeds/_/js/k%3Dytembeds.base.en_US.7J4wKKTzgAI.es5.O/am%3DAAAAgA/d%3D1/excm%3Dbase,root/br%3D1/rs%3DAGKMywEpP5H1eAyamKTuJTnAwxE-ZPBAgA/m%3Droot%2Cbase.

While this is correct at HTML / HTTP level, it seems to cause the player JS to not be able to parse its own URL anymore and the script fails.

Or at least if I replace safe('/') with safe('/=,') in rewriting logic then the player JS loads successfully.

Question is two-fold:

• shall we always consider = and , to be safe? Only for youtube URLs?
• should we consider other characters to be safe?