man pages are not readable on macOS

[mobyte0]

(This top portion is edited by @kaushalmodi)

Workaround

Due to the way man handles the man page caching and streaming on macOS systems, PAGER=eless man ls does not work there.

But PAGER=less man ls | eless works! See #27 (comment) for details.

(The original post now follows below.)

---

Mentioned this issue in #18.

 rob  ~/b/eless   master ±  echo $PAGER
eless
 rob  ~/b/eless   master ±  man ls | eless -D
Eless Version c447784 (commit hash of current master~1)
https://github.com/kaushalmodi/eless/tree/master
DEBUG: --> Input from pipe/file
DEBUG:     Output to terminal -->
DEBUG: var : -D
DEBUG: Raw Args                       : -D
DEBUG: Emacs Args                     : -nw
DEBUG: Pipe Contents (up to 10 lines) : This script is not supposed to send output to a pipe
DEBUG: Temp File : /var/folders/24/18pqhyd57wv1c4cn4r7zswt40000gn/T/emacs-stdin-rob.NyrJ0jY
DEBUG: first_line_piped_data = This script is not supposed to send output to a pipe
DEBUG: No man page or info manual detected
DEBUG: Eless Command : emacs_Q_view_mode /var/folders/24/18pqhyd57wv1c4cn4r7zswt40000gn/T/emacs-stdin-rob.NyrJ0jY                          -nw                          --eval '(progn
                                   (set-visited-file-name nil)
                                   (rename-buffer "*Stdin*" :unique))'
DEBUG: Args passed to emacs_Q_view_mode : /var/folders/24/18pqhyd57wv1c4cn4r7zswt40000gn/T/emacs-stdin-rob.NyrJ0jY -nw --eval (progn
                                   (set-visited-file-name nil)
                                   (rename-buffer "*Stdin*" :unique))

This script is not supposed to send output to a pipe displayed in eless.

 rob  ~/b/eless   master ± 

[kaushalmodi]

Hmm, for the same man ls | eless -D (I also have PAGER set to eless), I get:

Eless Version c447784 (commit hash of current master~1)
https://github.com/kaushalmodi/eless/tree/master
DEBUG: --> Input from pipe/file
DEBUG:     Output to terminal -->
DEBUG: var : -D
DEBUG: Raw Args                       : -D
DEBUG: Emacs Args                     : -nw
DEBUG: Pipe Contents (up to 10 lines) : LS(1)                            User Commands                           LS(1)



NAME
       ls - list directory contents

SYNOPSIS
       ls [OPTION]... [FILE]...
DEBUG: Temp File : /tmp/emacs-stdin-kmodi.EZUp5dI
DEBUG: first_line_piped_data = LS(1)                            User Commands                           LS(1)
DEBUG: Man Page = LS(1)
DEBUG: Eless Command : emacs_Q_view_mode                          -nw                          --eval '(progn
                                   (man (downcase "LS(1)"))
                                   ;; Below workaround is only for emacs 24.5.x and older releases
                                   ;; where the man page takes some time to load.
                                   ;; 1-second delay before killing the *scratch* window
                                   ;; seems to be sufficient
                                   (when (version<= emacs-version "24.5.99")
                                      (sit-for 1))
                                   (delete-window))'
DEBUG: Args passed to emacs_Q_view_mode : -nw --eval (progn
                                   (man (downcase "LS(1)"))
                                   ;; Below workaround is only for emacs 24.5.x and older releases
                                   ;; where the man page takes some time to load.
                                   ;; 1-second delay before killing the *scratch* window
                                   ;; seems to be sufficient
                                   (when (version<= emacs-version "24.5.99")
                                      (sit-for 1))
                                   (delete-window))

[kaushalmodi]

So, your Pipe Contents are different.. will need to investigate..

[mobyte0]

Just tested this on bash, it worked, so this seems to be a shell-specific issue. I'll look into this a bit as well.

[kaushalmodi]

Just tested this on bash

That was going to be my next question :)

so this seems to be a shell-specific issue.

Phew! :)

I'll look into this a bit as well.

Thank you!

[mobyte0]

Here is some documentation incase any other fish users want to look at this issue: https://fishshell.com/docs/current/tutorial.html#tut_pipes_and_redirections.

[mobyte0]

So here's what I found: https://stackoverflow.com/a/37531498/7873568

The command man ls ^&1 | eless is functional, but I'd like to just use eless as a pager and just be able to do man ls while eless is set to $PAGER.

[kaushalmodi]

Do you want to hack the eless script yourself to figure this out.. I don't have fish installed.

My best guess is that cat isn't doing the same thing on fish that it's doing on bash and tcsh.

This line -- It's where $piped_data gets assigned for the man page case i.e. "--> Input from pipe/file".

[mobyte0]

Of course the quick and easy way would be to check $SHELL but there must be a more elegant way to do it. I'll take a look.

[kaushalmodi]

Something is very confusing..

You got this printed:

DEBUG:     Output to terminal -->

That indicates that the output_to_pipe_flag variable should be 0.

So this if block is not entered:

if [[ ${output_to_pipe_flag} -eq 1 ]]
then
    echo "This script is not supposed to send output to a pipe"
    exit 1

.. then why is that echo message leaking into the pipeline_data?:

DEBUG: Pipe Contents (up to 10 lines) : This script is not supposed to send output to a pipe

Odd..

[mobyte0]

Did some testing:

 rob  ~/c/c/3310   master   set output_to_pipe_flag 1
 rob  ~/c/c/3310   master   echo $output_to_pipe_flag
1
 rob  ~/c/c/3310   master   if [[ ${output_to_pipe_flag} -eq 1 ]]
                                    then
                                      echo "Test"
                                      exit 1

                                end
Variables may not be used as commands. In fish, please define a function or use 'eval [[ $output_to_pipe_flag -eq 1 ]]'.
fish: if [[ ${output_to_pipe_flag} -eq 1 ]]
         ^
 rob  ~/c/c/3310   master  

I think fish is supposed to properly recognize the shebang #!/usr/bin/env bash but I'm not sure.

[kaushalmodi]

Variables may not be used as commands.

which variable is being used as command (which command?).. We are just checking if a FOO variable is 1.

[kaushalmodi]

Do you want to see if a change like this in the whole script fixes this for you?

diff --git a/eless b/eless
index e973726..78bada8 100755
--- a/eless
+++ b/eless
@@ -492,7 +492,7 @@ function emacs_Q_view_mode {
 
 # Below if condition is reached if you try to do this:
 #   eless foo.txt | grep bar .. Not allowed!
-if [[ ${output_to_pipe_flag} -eq 1 ]]
+if test ${output_to_pipe_flag} -eq 1
 then

If so, please provide a PR, and I will try it out on my side.

[kaushalmodi]

Alternatively, see if

if [ ${output_to_pipe_flag} -eq 1 ]

([/] instead of [[/]])

does the same thing i.e. fixes this issue for you.

[mobyte0]

This worked!

I'll test the single square bracket one now.

[mobyte0]

Single square bracket test was functional as well.

[mobyte0]

However, setting eless to PAGER still results in the original problem. But at least piping works now.

[kaushalmodi]

Thanks to https://stackoverflow.com/a/35661359/1219634 and http://hyperpolyglot.org/unix-shells#arith-conditional-expr.

Though.. the bigger question still remains.. why does fish try to interpret bash syntax when I have the bash shebang at the top!

Do you have bash symlinked to fish? :P

[kaushalmodi]

setting eless to PAGER still results in the original problem

Though, now the -D output should be different, isn't it?

[mobyte0]

Nope, my bash is just plain old GNU bash, lol.

How would you propose I enable debugging when just using something like man ls?

[mobyte0]

Now that I think about it, let me set PAGER to eless -D.

[kaushalmodi]

Yup, you got it .. also: https://eless.scripter.co/#how-to-help-debug :)

[mobyte0]

Got a different output this time inside the eless session.

Eless Version c447784 (commit hash of current master~1)
https://github.com/kaushalmodi/eless/tree/master
This script is not supposed to send output to a pipe

And the debug output:

 rob  ~  man ls
Eless Version c447784 (commit hash of current master~1)
https://github.com/kaushalmodi/eless/tree/master
DEBUG: --> Input from pipe/file
DEBUG:     Output to terminal -->
DEBUG: var : -D
DEBUG: Raw Args                       : -D
DEBUG: Emacs Args                     : -nw
DEBUG: Pipe Contents (up to 10 lines) : LS(1)                                  User Commands                                 LS(1)



NAME
       ls - list directory contents

SYNOPSIS
       ls [OPTION]... [FILE]...
DEBUG: Temp File : /var/folders/24/18pqhyd57wv1c4cn4r7zswt40000gn/T/emacs-stdin-rob.hH4AC5h
DEBUG: first_line_piped_data = LS(1)                                  User Commands                                 LS(1)
DEBUG: Man Page = LS(1)
DEBUG: Eless Command : emacs_Q_view_mode                          -nw                          --eval '(progn
                                   (man (downcase "LS(1)"))
                                   ;; Below workaround is only for emacs 24.5.x and older releases
                                   ;; where the man page takes some time to load.
                                   ;; 1-second delay before killing the *scratch* window
                                   ;; seems to be sufficient
                                   (when (version<= emacs-version "24.5.99")
                                      (sit-for 1))
                                   (delete-window))'
DEBUG: Args passed to emacs_Q_view_mode : -nw --eval (progn
                                   (man (downcase "LS(1)"))
                                   ;; Below workaround is only for emacs 24.5.x and older releases
                                   ;; where the man page takes some time to load.
                                   ;; 1-second delay before killing the *scratch* window
                                   ;; seems to be sufficient
                                   (when (version<= emacs-version "24.5.99")
                                      (sit-for 1))
                                   (delete-window))
 rob  ~ 

[kaushalmodi]

Hmm.. compared to my debug output ( #27 (comment) ), your output now looks the same, except of course the Temp File.

But you still are getting "This script is not supposed to send output to a pipe" from an if block that should not be executed..

Do you have any [[ .. ]] remaining to be replaced with [ .. ] by any chance?

[mobyte0]

Replaced all of the [[ .. ]] with [ .. ], essentially got the same output.

[kaushalmodi]

Still wondering why that if block (containing "This script is not supposed to send output to a pipe") is getting executed..

But in any case, I can only guess.. fish probably doesn't like this:

                  )' 2>/dev/null </dev/tty

[kaushalmodi]

Alright.. so I just cloned the fish-shell repo, built it, ran ./fish.. and.. man ls works just fine (and this is without any modification in eless.. I used the version from the master branch of this repo.)

What is your fish version, do you want to try the version from its master branch?

Proof

[kaushalmodi]

Ah, OK, sorry, I misunderstood. The issue is the @mobyte0 is seeing this error instead of the man page: #27 (comment)

What's perplexing is that that error is printing from the if clause that should never have been accessed for the man ls case.

[sshaw]

Okay. I do have that problem (PAGER=eless man ls)

From the CAT PAGES section of man(1):

Man will try to save the formatted man pages, in order to save formatting time the next time
these pages are needed. Traditionally, formatted versions of pages in DIR/manX are saved in
DIR/catX, but other mappings from man dir to cat dir can be specified in /private/etc/man.conf.
No cat pages are saved when the required cat directory does not exist. No cat pages are saved
when they are formatted for a line length different from 80. No cat pages are saved when
man.conf contains the line NOCACHE.

So I assume the reason you're getting the warning is that the pages formatted by PAGER are sent to disk to be repurposed.

I also get mangled characters. Like this, but only for certain man pages (For me only Perl and Tk man pages, e.g., man Net::FTP).

As you can see from man -d, there's a lot going on before it's output to eless:

/tmp/eless >man -d Net::FTP
Reading config file /private/etc/man.conf
Looked whether there exists a message catalog man, but there is none
(and for English messages none is needed)

found man directory /usr/share/man
found man directory /usr/local/share/man
found man directory /usr/X11/man
found manpath map /bin --> /usr/share/man
found manpath map /sbin --> /usr/share/man
found manpath map /usr/bin --> /usr/share/man
found manpath map /usr/sbin --> /usr/share/man
found manpath map /usr/local/bin --> /usr/local/share/man
found manpath map /usr/local/sbin --> /usr/local/share/man
found manpath map /usr/X11/bin --> /usr/X11/man
found manpath map /usr/bin/X11 --> /usr/X11/man
found manpath map /usr/bin/mh --> /usr/share/man
found man directory /usr/local/gnupg-2.1/share/man

using eless as pager

using /usr/bin/less -is as browser

using /bin/cat to dump HTML pages as textadding /Users/sshaw/perl5/perlbrew/perls/perl-5.20.0/man to manpath
adding /Users/sshaw/.rvm/rubies/ruby-2.4.0/share/man to manpath
adding /usr/share/man to manpath
adding /usr/local/share/man to manpath
adding /opt/X11/share/man to manpath
adding /Library/TeX/texbin/man to manpath
adding /Users/sshaw/.rvm/share/man to manpath
adding /usr/local/gnupg-2.1/share/man to manpath
adding /Applications/Xcode.app/Contents/Developer/usr/share/man to manpath
adding /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/share/man to manpath
no cat page stored because of nonstandard line length
using default preprocessor sequence
found tbl(1) directive

not executing command:
  (cd '/Users/sshaw/perl5/perlbrew/perls/perl-5.20.0/man' && (echo ".ll 10.3i"; echo ".nr LL 10.3i"; /bin/cat '/Users/sshaw/perl5/perlbrew/perls/perl-5.20.0/man/man3/Net::FTP.3') | /usr/bin/tbl | /usr/bin/groff -Wall -mtty-char -Tascii -mandoc -c | (eless || true))

[nasyxx]

Hi, my bash version shows below.

> bash --version
GNU bash, version 3.2.57(1)-release (x86_64-apple-darwin17)
Copyright (C) 2007 Free Software Foundation, Inc.

I also have tried 4.4, but still failed.

Other than that, when I use

PAGER=less man ls | eless

everything is okay, however, when I use

PAGER=most man ls | eless

"This script is not supposed to send output to a pipe" displayed as PAGER=eless man ls did.

[kaushalmodi]

@sshaw @mobyte0 @nasyxx Do you want to try this out:

No cat pages are saved when man.conf contains the line NOCACHE.

Does the issue go away if you disable man page caching?

[kaushalmodi]

@nasyxx

PAGER=less man ls | eless everything is okay

@sshaw @mobyte0 @nasyxx So are these 2 statements correct for the man binary on your systems:

• PAGER=less man ls | eless works
• PAGER=eless man ls does not work

however, when I use PAGER=most man ls | eless, "This script is not supposed to send output to a pipe" displayed as PAGER=eless man ls did.

Alright, things have anyways got complicated.. so let's leave most out of this for now.. may be a separate issue once this gets resolved :)

[nasyxx]

Hi, I added NOCACHE to the end of man.conf,

but it still does not work.

When I run PAGER=eless man Net::FTP, the result is the same as @sshaw 's, mangled characters.

And PAGER=eless man -d Net::FTP shows as below.

...
adding mandatory man directories

adding /Applications/Xcode.app/Contents/Developer/usr/share/man to manpath
adding /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/share/man to manpath
no cat page stored because of nonstandard line length
using default preprocessor sequence
found tbl(1) directive

not executing command:
  (cd '/usr/local/share/man' && (echo ".ll 14.5i"; echo ".nr LL 14.5i"; /bin/cat '/usr/local/share/man/man3/Net::FTP.3') | /usr/bin/tbl | /usr/bin/groff -Wall -mtty-char -Tascii -mandoc -c | (eless || true))

[kaushalmodi]

the result is the same as @sshaw 's, mangled characters.

I fixed the "mangled characters" issue today morning.. please update to the latest master. The mangled characters are actually ascii color codes that the reader should interpret (like man mode in Emacs, or the man binary).

[kaushalmodi]

And I get this on doing PAGER=eless man -d Net::FTP:

adding mandatory man directories

adding /cad/adi/apps/gnu/linux/x86_64/6/share/man to manpath
no cat page stored because of nonstandard line length
using default preprocessor sequence
found tbl(1) directive
man:
not executing command:
  (cd "/usr/share/man" && (echo ".ll 16.3i"; echo ".nr LL 16.3i"; echo ".pl 1100i"; /usr/bin/gunzip -c '/usr/share/man/man3/Net::FTP.3pm.gz'; echo ".\\\""; echo ".pl \n(nlu+10") | /usr/bin/gtbl | /usr/bin/nroff -c -mandoc 2>/dev/null | eless)

That last bit:

... echo ".\\\""; echo ".pl \n(nlu+10") | /usr/bin/gtbl | /usr/bin/nroff -c -mandoc 2>/dev/null | eless

is quite different on my system.

[nasyxx]

Well, now, it does not work again.

[nasyxx]

And on doing with -d (I tried it twice, both for this result.), I get

using default preprocessor sequence
found tbl(1) directive

not executing command:
  (cd '/usr/local/share/man' && (echo ".ll 14.5i"; echo ".nr LL 14.5i"; /bin/cat '/usr/local/share/man/man3/Net::FTP.3') | /usr/bin/tbl | /usr/bin/groff -Wall -mtty-char -Tascii -mandoc -c | (eless || true))

[kaushalmodi]

does not work again

This is like shooting in the dark..

Does PAGER=less man ls | eless also not work? If it works, then I believe we can settle on a workaround for macOS.. Simply create a little bash function or alias that does that. WDYT?

[kaushalmodi]

does not work again

This is like shooting in the dark..

Thinking more about it.. not exactly.. You get that "This script is not .." error when it correctly gets detected as a man page i.e. when this if condition gets executed.

When it was not getting detected (before this fix), it was entering this final else clause. So all man page raw data including the man page color codes showed up without rendering in emacs buffer.

So that part makes sense now.

So based on @sshaw's experience, PAGER=less man ls | eless or PAGER=less man Net::FTP | eless must now work robustly on macOS.

[nasyxx]

Does PAGER=less man ls | eless also not work?

Yes, it works fine.

[kaushalmodi]

OK, so if PAGER=less man ls | eless is working for @mobyte0 too, would an aliasing solution like this work?

If so, please share the alias that works for you.

Of course, if one of you can provide a true fix so that eless works with macOS man, I will accept the PR.

[nasyxx]

the alias that works for you.

I am not sure how to make an alias with parameters in ZSH. So I write it as:

alias eman='elessman(){PAGER=less man $1 | eless}; elessman'

> eman ls  # should work

[kaushalmodi]

@nasyxx Thanks! That almost works on bash.. I have verified this to work on bash (ref):

> alias eman='elessman() { PAGER=less man $1 | eless; }; elessman'
> eman ls

The fix was to add that ; after eless. If this works for you too on zsh, I can mention in that docs that macOS users can use this alias in their bash/zsh profiles if they want to view man pages using eless.

[nasyxx]

add that ; after eless

It still works for me on zsh.

[mobyte0]

Here's a totally gross alias that I've set up to solve the problem (in fish shell):

function man
		 /usr/bin/man $argv[1] | eless
end

So basically, it just takes the man page and pipes it right into eless. I think I'm giving up on wrestling with $PAGER, lol.

[kaushalmodi]

I have updated the documentation in the latest commit on master. Now we have this section: https://eless.scripter.co/#example-eless-config-in-bash

[mobyte0]

Oh, hey, I found an even better solution. Interestingly enough, I realized that in the man pages for man that it lets you specify the pager as an argument. Thanks to this, a universal solution should be possible that would involve just setting man to an alias such as /usr/bin/man -P eless $argv with replacing /usr/bin/man to wherever your man is located.

Hopefully this helps!

[kaushalmodi]

@mobyte0 Very cool! that's definitely less work than the earlier alias. So does this alias work for you:

alias eman='man -P eless'
eman ls

If so, I'll update the documentation.

Also, does this issue need to stay open any more?

---

Update

Or may be the below.. if you want to leave the default PAGER as eless?:

alias eman='PAGER=less man -P eless'

[kaushalmodi]

I believe it is safe to close this issue as PAGER=less man -P eless .. has been confirmed to work well on OSX.

Ref: https://eless.scripter.co/#example-eless-config-in-bash