Problem with brotli compression

[chipschap]

I noticed a while ago after some updates that some web sites stopped rendering correctly and produced binary-looking buffers. I didn't resolve this. A little later another person mentioned the same thing in the emacs help mailing list. But some others don't have the problem. I tracked it down and it appears to be that emacs-w3m doesn't have a "brotli" handler for brotli web page compression, as used by slashdot, duckduckgo, and others. Strangely this only seems to affect the later versions of the w3m command; earlier versions seem okay although I can't be more specific about which versions.

I added two lines of code that for me and the other user resolve the problem. The patch requires that the "brotli" command be in the command path. I don't check for this. I didn't generate a diff but the following in w3m.el should be hopefully clear enough. This has been tested on Endeavour (Arch), OpenBSD, and Debian systems.

defcustom w3m-decoder-alist should add this line to the list of types

(br "brotli" ("-d"))

defconst w3m-encoding-alist should add this line to the list of types

	     (br . ("br"))

I hope this is of some use.

[yamaoka]

Thank you very much for the information. I've added the brotli entry to w3m-decoder-alist and w3m-encoding-alist.

Though I've never faced with such a page seen like binary, I've installed the brotli executable in my box. ;-)

[Boruch-Baum]

I also have never had the problem, and I frequently use emacs-w3m for the sites that OP mentions. The relevant upstream PR was merged on Feb 11, 2021: tats/w3m#164 My guess is that the reason neither of us has been affected is that brotli compression needs to be mentioned in the HTTP GET request that w3m sends, in place of mentioning, for example, gzip, so the mystery of which versions of w3m have the issue would just be a configuration option in w3m. My w3m config file (~/.w3m/config) includes the configuration option "accept_encoding" which for me is "gzip, compress, bzip, bzip2, deflate". My guess is that OP has "brotli" in that list and prior to gzip there.

…

On 2022-11-07 15:48, yamaoka wrote: Thank you very much for the information. I've added the brotli entry to w3m-decoder-alist and w3m-encoding-alist. Though I've never faced with such a page seen like binary, I've installed the brotli executable in my box. ;-) — Reply to this email directly, [1]view it on GitHub, or [2]unsubscribe. You are receiving this because you are subscribed to this thread. Message ID: ***@***.***> References 1. #112 (comment) 2. https://github.com/notifications/unsubscribe-auth/AAOE3KHKCX47JOW7GP7LDKTWHGIMXANCNFSM6AAAAAARZR2FZU

-- hkp://keys.gnupg.net CA45 09B5 5351 7C11 A9D1 7286 0036 9E45 1595 8BC0

[chipschap]

I am using w3m 0.5.3.git20220409_1-2 and I can't find the option accept_encoding in the documentation. I don't have a w3m config file at all so I suspect w3m accepts brotli but then emacs-w3m couldn't previously deal with it. The problem here dates from about the time you mention --- early in 2021 when you point out that brotli support was added to w3m.

But in any event, having emacs-w3m being capable of handling brotli is a bit of a step forward, I think.

[Boruch-Baum]

Yes, I'm happy that you helped emacs-w3m take the step forward. I was trying to suggest how it is that users who browse slashdot and duckduckgo don't experience the behavior that you reported, and also to guide anyone who wants to add brotli support to their w3m config. For information about the w3m config file locations (user-specific and system-default): man(1) w3m. In debian, you can also perform $ dpkg -L w3m |grep config For information about the 'accept_encoding' option: $ w3m -show-option |less -paccept_encoding or $ w3m -show-option |grep accept_encoding

…

On 2022-11-09 08:51, chipschap wrote: I am using w3m 0.5.3.git20220409_1-2 and I can't find the option accept_encoding in the documentation. I don't have a w3m config file at all so I suspect w3m accepts brotli but then emacs-w3m couldn't previously deal with it. The problem here dates from about the time you mention --- early in 2021 when you point out that brotli support was added to w3m. But in any event, having emacs-w3m being capable of handling brotli is a bit of a step forward, I think. — Reply to this email directly, [1]view it on GitHub, or [2]unsubscribe. You are receiving this because you commented. Message ID: ***@***.***> References 1. #112 (comment) 2. https://github.com/notifications/unsubscribe-auth/AAOE3KDB3JM5NEAU3S3SSU3WHPJB7ANCNFSM6AAAAAARZR2FZU

-- hkp://keys.gnupg.net CA45 09B5 5351 7C11 A9D1 7286 0036 9E45 1595 8BC0

[chipschap]

Yes, got it, and your clarification is very useful. Thank you for that, and it seems we now have the best of both worlds. In my case, I can go back to reading Slashdot with w3m.el instead of eww!

…

On Wed, Nov 9, 2022 at 8:42 AM Boruch Baum ***@***.***> wrote: Yes, I'm happy that you helped emacs-w3m take the step forward. I was trying to suggest how it is that users who browse slashdot and duckduckgo don't experience the behavior that you reported, and also to guide anyone who wants to add brotli support to their w3m config. For information about the w3m config file locations (user-specific and system-default): man(1) w3m. In debian, you can also perform $ dpkg -L w3m |grep config For information about the 'accept_encoding' option: $ w3m -show-option |less -paccept_encoding or $ w3m -show-option |grep accept_encoding On 2022-11-09 08:51, chipschap wrote: > I am using w3m 0.5.3.git20220409_1-2 and I can't find the option > accept_encoding in the documentation. I don't have a w3m config file at > all so I suspect w3m accepts brotli but then emacs-w3m couldn't > previously deal with it. The problem here dates from about the time you > mention --- early in 2021 when you point out that brotli support was > added to w3m. > > But in any event, having emacs-w3m being capable of handling brotli is > a bit of a step forward, I think. > > — > Reply to this email directly, [1]view it on GitHub, or [2]unsubscribe. > You are receiving this because you commented. Message ID: > ***@***.***> > > References > > 1. #112 (comment) > 2. https://github.com/notifications/unsubscribe-auth/AAOE3KDB3JM5NEAU3S3SSU3WHPJB7ANCNFSM6AAAAAARZR2FZU -- hkp://keys.gnupg.net CA45 09B5 5351 7C11 A9D1 7286 0036 9E45 1595 8BC0 — Reply to this email directly, view it on GitHub <#112 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAJELKXHXSRSOWVDGY3CCMDWHPWCRANCNFSM6AAAAAARZR2FZU> . You are receiving this because you authored the thread.Message ID: ***@***.***>

-- Bob Newell Honolulu, Hawai`i Via Linux/Emacs/Gnus/BBDB.