changes needed to compile and run on illumos (nee OpenSolaris) #3340

Closed
jblachly opened this Issue Aug 29, 2016 · 29 comments

Comments

Projects
None yet
7 participants
@jblachly

jblachly commented Aug 29, 2016

fish-shell will not compile on illumos (and probably not on Solaris) because this family of operating systems lacks the MAP_FILE flag for mmap. FreeBSD also lacks it according to my read of the manpage (although it may be defined somewhere for compatibility), so I am not sure how/if fish-shell builds on FreeBSD.

git commit 3702616

uname -a output:

SunOS smartos-dev 5.11 joyent_20160731T153326Z i86pc i386 i86pc Solaris

fish-shell will not build because the mmap flag MAP_FILE is not defined on Solaris family operating systems. THis flag, which is a dummy, compatibility flag set to zero on all platforms for which it is defined (AFAIK), is included in the mmap call on line 1045 of env_universal_common.cpp.

By including

// Solaris family (e.g. illumos) doesn't have MAP_FILE
#ifndef MAP_FILE
#define MAP_FILE 0
#endif

at the beginning of env_universal_common.h fish-shell will now compile.

I have not made a pull request because I am new to the codebase and am not sure the most appropriate place for this #ifdef.

If this fix can get integrated, it opens up an entirely new pool of potential users. Once fish-shell builds straightforwardly I will push to get fish-shell in the illumos package repos.

Thanks for considering this.

I have to amend the above. Although the above fix will allow fish-shell to build, running it leads to error-city. Looks like I have a lot more work to do before it is usable. Sorry

@krader1961

This comment has been minimized.

Show comment
Hide comment
@krader1961

krader1961 Aug 29, 2016

Contributor

FWIW, It is rather strange that any UNIX derivative does not have a MAP_FILE symbol. Yes, strictly speaking, it is redundant since the default behavior of mmap() is to memory map a file. That's why it is zero on every platform I've ever used in the past 30+ years. Thus making it a no-op when or'd into a mask. You're proposal to add a fallback definition to env_universal_common.h seems fine to me.

Fish has built and run on Solaris in the past. Not sure what the current situation is. If you provide details about the errors you're seeing we can probably provide guidance.

Contributor

krader1961 commented Aug 29, 2016

FWIW, It is rather strange that any UNIX derivative does not have a MAP_FILE symbol. Yes, strictly speaking, it is redundant since the default behavior of mmap() is to memory map a file. That's why it is zero on every platform I've ever used in the past 30+ years. Thus making it a no-op when or'd into a mask. You're proposal to add a fallback definition to env_universal_common.h seems fine to me.

Fish has built and run on Solaris in the past. Not sure what the current situation is. If you provide details about the errors you're seeing we can probably provide guidance.

@krader1961 krader1961 added this to the fish-future milestone Aug 29, 2016

@krader1961 krader1961 changed the title from Won't compile on illumos (nee OpenSolaris) - fix included to changes needed to compile and run on illumos (nee OpenSolaris) Aug 29, 2016

@jblachly

This comment has been minimized.

Show comment
Hide comment
@jblachly

jblachly Aug 29, 2016

@krader1961 Thanks for your response (and changing issue title so I don't have to open a slew of issues).

I note that the getcwd(NULL, 0) call in wutil.cpp passes 0 for size_t size. This causes an EINVAL return on my platform. Reading the man page for linux, it appears that glibc extends the POSIX.1-2001 standard and allows a zero size in which case the buffer is allocated as large as is needed. However, this only works on glibc platforms.

Changing this parameter to PATH_MAX (defined in limits.h on most platforms, but also taken care of explicitly in your wutil.cpp file in case it is not defined by the platform) fixes the issue.

Is it okay if I detail individual issues (and proposed fixes, if I know them) here, or should I open additional issues for better tracking?

@krader1961 Thanks for your response (and changing issue title so I don't have to open a slew of issues).

I note that the getcwd(NULL, 0) call in wutil.cpp passes 0 for size_t size. This causes an EINVAL return on my platform. Reading the man page for linux, it appears that glibc extends the POSIX.1-2001 standard and allows a zero size in which case the buffer is allocated as large as is needed. However, this only works on glibc platforms.

Changing this parameter to PATH_MAX (defined in limits.h on most platforms, but also taken care of explicitly in your wutil.cpp file in case it is not defined by the platform) fixes the issue.

Is it okay if I detail individual issues (and proposed fixes, if I know them) here, or should I open additional issues for better tracking?

@floam

This comment has been minimized.

Show comment
Hide comment
@floam

floam Aug 29, 2016

Member

Maybe first take a look over the autoconf macros commonly used to check for this one and see how it's commonly dealt with in case there's a surprise or it informs - it's something we don't do enough of, frankly - there are a few things it resolves for us we haven't quite "hooked up" to the macros they set for us after pretty great tests.

Member

floam commented Aug 29, 2016

Maybe first take a look over the autoconf macros commonly used to check for this one and see how it's commonly dealt with in case there's a surprise or it informs - it's something we don't do enough of, frankly - there are a few things it resolves for us we haven't quite "hooked up" to the macros they set for us after pretty great tests.

@jblachly

This comment has been minimized.

Show comment
Hide comment
@jblachly

jblachly Aug 29, 2016

@floam I have to say that I never did learn the autconf and other build tools, but I'll give it a go.

FWIW, I am initiating a pull request for illumos-gate to add MAP_FILE to mman.h, but Oracle Solaris 11.3 latest still does not include it AFAIK, so an autoconf detection will still be nice

@floam I have to say that I never did learn the autconf and other build tools, but I'll give it a go.

FWIW, I am initiating a pull request for illumos-gate to add MAP_FILE to mman.h, but Oracle Solaris 11.3 latest still does not include it AFAIK, so an autoconf detection will still be nice

@floam

This comment has been minimized.

Show comment
Hide comment
@floam

floam Aug 29, 2016

Member

FWIW: Austin Group references in 0000850: Standardize MAP_ANON the mmap-anon.m4, mirrored here. I guess we should see if we have are seeing a test as capable as that occurring at all (needs updating?), if it's being used right, or if it needs fixing?

I know I tried to run autoupdate yesterday, and autoscan'd fish as a new project just to compare and it looks like this is all kind of dusty.

Member

floam commented Aug 29, 2016

FWIW: Austin Group references in 0000850: Standardize MAP_ANON the mmap-anon.m4, mirrored here. I guess we should see if we have are seeing a test as capable as that occurring at all (needs updating?), if it's being used right, or if it needs fixing?

I know I tried to run autoupdate yesterday, and autoscan'd fish as a new project just to compare and it looks like this is all kind of dusty.

@krader1961

This comment has been minimized.

Show comment
Hide comment
@krader1961

krader1961 Aug 29, 2016

Contributor

@jblachly Whether or not we tackle all the illumos (nee OpenSolaris) problems in this issue or separate issues depends on whether we expect there to be more than one or two non-trivial problems. The lack of MAP_FILE is a trivial problem with a trivial, obvious, fix. Something like the getcwd() problem is non-trivial.

The getcwd() failure is interesting. Since we're explicitly passing a NULL pointer for the buffer we're telling the implementation to allocate an appropriately sized buffer and thus a size of zero is reasonable. On the other hand I found at least one Open Group standard that says "If buf is a null pointer, the behavior of getcwd() is unspecified." So, like so many things of this nature, the question is how best to support multiple platforms that conform to different standards. Note that the man getcwd man page on macOS Sierra says that the function returns EINVAL if the size argument is zero. Yet it obviously does not since I'm successfully using fish on that OS.

There is also this interesting blog article about their real world experience dealing with APIs like getcwd() and PATH_MAX. On OS X (even the beta macOS Sierra build) PATH_MAX is 1024. Which is absurd. So it's not obvious that replacing that zero with PATH_MAX in the getcwd() call is the right thing to do on all platforms.

Fantastic! Even the fish source code is schizophrenic about PATH_MAX. Just look at how it's used in src/fish_tests.cpp. </face-palm>

Contributor

krader1961 commented Aug 29, 2016

@jblachly Whether or not we tackle all the illumos (nee OpenSolaris) problems in this issue or separate issues depends on whether we expect there to be more than one or two non-trivial problems. The lack of MAP_FILE is a trivial problem with a trivial, obvious, fix. Something like the getcwd() problem is non-trivial.

The getcwd() failure is interesting. Since we're explicitly passing a NULL pointer for the buffer we're telling the implementation to allocate an appropriately sized buffer and thus a size of zero is reasonable. On the other hand I found at least one Open Group standard that says "If buf is a null pointer, the behavior of getcwd() is unspecified." So, like so many things of this nature, the question is how best to support multiple platforms that conform to different standards. Note that the man getcwd man page on macOS Sierra says that the function returns EINVAL if the size argument is zero. Yet it obviously does not since I'm successfully using fish on that OS.

There is also this interesting blog article about their real world experience dealing with APIs like getcwd() and PATH_MAX. On OS X (even the beta macOS Sierra build) PATH_MAX is 1024. Which is absurd. So it's not obvious that replacing that zero with PATH_MAX in the getcwd() call is the right thing to do on all platforms.

Fantastic! Even the fish source code is schizophrenic about PATH_MAX. Just look at how it's used in src/fish_tests.cpp. </face-palm>

@jblachly

This comment has been minimized.

Show comment
Hide comment
@jblachly

jblachly Aug 30, 2016

@krader1961 I will take you up on your offer for me to detail the errors I'm seeing for your guidance. I will be happy to open additional tickets, but also to perform any work on my end to help track these down. Please keep in mind although I can stumble about in C and C++, I do not consider myself a systems programmer. My level of sophistication with debugging in C is stuck at the "include extra verbose debug logging". To that end, I did overload and modify the wperror() function to report the function from which it was called -- can I provide that code here or should I fork and make pull request?

Using the github commit referenced at the top of this issue:

  • First, it fails to compile unless I define MAP_FILE. I am taking care of this at my OS level, but I think this will still be a problem on Solaris.
  • After fixing that it builds, but there is a getcwd error when running:
% ./fish
<E> fish: getcwd() failed with errno 22/Invalid argument
<E> fish: Could not determine current working directory. Is your locale set correctly?
<W> fish: Unable to parse universal variable message: 'SET __fish_init_2_3_0:\x1d'
<E> fish: getcwd() failed with errno 22/Invalid argument
...many, many lines (see later)
<W> fish: Unable to parse universal variable message: 'SET __fish_init_2_3_0:\x1d'
<E> fish: getcwd() failed with errno 22/Invalid argument
@ > <E> fish: getcwd() failed with errno 22/Invalid argument
@ >
  • I have already detailed how to fix this problem (PATH_MAX)
  • After fixing the getcwd problem and recompiling, here is the output (and where I am stuck):
% ./fish
<W> fish: Unable to parse universal variable message: 'SET __fish_init_2_3_0:\x1d'
<W> fish: Unable to parse universal variable message: 'SET __fish_init_2_3_0:\x1d'
<W> fish: Unable to parse universal variable message: 'SET fish_greeting:Welcome\x20to\x20fish\x2c\x20the\x20friendly\x20interactive\x20shell\x0aType\x20\x1b\x5b32mhelp\x1b\x5b30m\x1b\x28B\x1b\x5bm\x20for\x20instructions\x20on\x20how\x20to\x20use\x20fish'
<W> fish: Unable to parse universal variable message: 'SET __fish_init_1_50_0:\x1d'
<W> fish: Unable to parse universal variable message: 'SET fish_color_normal:normal'
<W> fish: Unable to parse universal variable message: 'SET fish_color_command:brblue'
<W> fish: Unable to parse universal variable message: 'SET fish_color_param:cyan'
<W> fish: Unable to parse universal variable message: 'SET fish_color_redirection:normal'
<W> fish: Unable to parse universal variable message: 'SET fish_color_comment:red'
<W> fish: Unable to parse universal variable message: 'SET fish_color_error:red\x1e\x2d\x2dbold'
<W> fish: Unable to parse universal variable message: 'SET fish_color_escape:cyan'
<W> fish: Unable to parse universal variable message: 'SET fish_color_operator:cyan'
<W> fish: Unable to parse universal variable message: 'SET fish_color_end:green'
<W> fish: Unable to parse universal variable message: 'SET fish_color_quote:brown'
<W> fish: Unable to parse universal variable message: 'SET fish_color_autosuggestion:brgrey'
<W> fish: Unable to parse universal variable message: 'SET fish_color_user:green'
<W> fish: Unable to parse universal variable message: 'SET fish_color_host:normal'
<W> fish: Unable to parse universal variable message: 'SET fish_color_valid_path:\x2d\x2dunderline'
<W> fish: Unable to parse universal variable message: 'SET fish_color_cwd:green'
<W> fish: Unable to parse universal variable message: 'SET fish_color_cwd_root:red'
<W> fish: Unable to parse universal variable message: 'SET fish_color_match:cyan'
<W> fish: Unable to parse universal variable message: 'SET fish_color_search_match:\x2d\x2dbackground\x3dpurple'
<W> fish: Unable to parse universal variable message: 'SET fish_color_selection:\x2d\x2dbackground\x3dpurple'
<W> fish: Unable to parse universal variable message: 'SET fish_pager_color_prefix:cyan'
<W> fish: Unable to parse universal variable message: 'SET fish_pager_color_completion:normal'
<W> fish: Unable to parse universal variable message: 'SET fish_pager_color_description:brgrey'
<W> fish: Unable to parse universal variable message: 'SET fish_pager_color_progress:cyan'
<W> fish: Unable to parse universal variable message: 'SET fish_color_history_current:cyan'
There is no fish_key_bindings function called: 'fish_default_key_bindings'
Reverting to default bindings
<W> fish: Unable to parse universal variable message: 'SET fish_key_bindings:fish_default_key_bindings'
<W> fish: Unable to parse universal variable message: 'SET fish_key_bindings:fish_default_key_bindings'
There is no fish_key_bindings function called: ''
Reverting to default bindings

(previous 2 lines repeated 125 times...I think)

The function call stack limit has been exceeded. Do you have an accidental infinite loop?
fish: __fish_reload_key_bindings VARIABLE SET fish_key_bindings
      ^
in event handler: handler for variable 'fish_key_bindings'

There is no fish_key_bindings function called: ''
Reverting to default bindings

(previous 2 lines repeated approximately 9x10^zillion times)

The function call stack limit has been exceeded. Do you have an accidental infinite loop?
fish: __fish_reload_key_bindings VARIABLE SET fish_key_bindings
      ^
in event handler: handler for variable 'fish_key_bindings'

<W> fish: Unable to parse universal variable message: 'SET __fish_init_2_3_0:\x1d'
@ ~/home/james/source/fish-shell>

From here I have a shell and can execute external commands, but command completion routines in path.cpp:path_get_path_core cause a lot of pain:

@ ~/home/james/source/fish-shell> l<W> fish: Error while searching for command '/usr/xpg4/bin/l'
access: No such file or directory
<W> fish: Error while searching for command '/usr/local/sbin/l'
access: No such file or directory
<W> fish: Error while searching for command '/usr/local/bin/l'
access: No such file or directory
<W> fish: Error while searching for command '/opt/local/sbin/l'
access: No such file or directory
<W> fish: Error while searching for command '/opt/local/bin/l'
access: No such file or directory
<W> fish: Error while searching for command '/usr/sbin/l'
access: No such file or directory
<W> fish: Error while searching for command '/usr/bin/l'
access: No such file or directory
<W> fish: Error while searching for command '/sbin/l'
access: No such file or directory
ls
autom4te.cache           config.status            Dockerfile               fish_tests               Makefile                 src
build_tools              config.sub               Doxyfile                 FISH-BUILD-VERSION-FILE  Makefile.in              test
CHANGELOG.md             configure                Doxyfile.help            fish.pc                  obj                      tests
config.guess             configure.ac             Doxyfile.user            fish.pc.in               osx
config.h                 CONTRIBUTING.md          etc                      fish.spec.in             pcre2-10.21
config.h.in              COPYING                  fish                     fish.xcodeproj           po
config.h.in~             debian                   fish_indent              install-sh               README.md
config.log               doc_src                  fish_key_reader          lexicon_filter.in        share
@ ~/home/james/source/fish-shell>

I have done further amateur debugging (really, it's way past time I learned to use gdb) by emitting specific errno's and can make some conclusions about why this may be happening, but I will stop here in case you as the experts have any insights.

Thanks again and let me know if I should break this into separate issues, or if I can do any debugging on your behalf. Kind regards.

@krader1961 I will take you up on your offer for me to detail the errors I'm seeing for your guidance. I will be happy to open additional tickets, but also to perform any work on my end to help track these down. Please keep in mind although I can stumble about in C and C++, I do not consider myself a systems programmer. My level of sophistication with debugging in C is stuck at the "include extra verbose debug logging". To that end, I did overload and modify the wperror() function to report the function from which it was called -- can I provide that code here or should I fork and make pull request?

Using the github commit referenced at the top of this issue:

  • First, it fails to compile unless I define MAP_FILE. I am taking care of this at my OS level, but I think this will still be a problem on Solaris.
  • After fixing that it builds, but there is a getcwd error when running:
% ./fish
<E> fish: getcwd() failed with errno 22/Invalid argument
<E> fish: Could not determine current working directory. Is your locale set correctly?
<W> fish: Unable to parse universal variable message: 'SET __fish_init_2_3_0:\x1d'
<E> fish: getcwd() failed with errno 22/Invalid argument
...many, many lines (see later)
<W> fish: Unable to parse universal variable message: 'SET __fish_init_2_3_0:\x1d'
<E> fish: getcwd() failed with errno 22/Invalid argument
@ > <E> fish: getcwd() failed with errno 22/Invalid argument
@ >
  • I have already detailed how to fix this problem (PATH_MAX)
  • After fixing the getcwd problem and recompiling, here is the output (and where I am stuck):
% ./fish
<W> fish: Unable to parse universal variable message: 'SET __fish_init_2_3_0:\x1d'
<W> fish: Unable to parse universal variable message: 'SET __fish_init_2_3_0:\x1d'
<W> fish: Unable to parse universal variable message: 'SET fish_greeting:Welcome\x20to\x20fish\x2c\x20the\x20friendly\x20interactive\x20shell\x0aType\x20\x1b\x5b32mhelp\x1b\x5b30m\x1b\x28B\x1b\x5bm\x20for\x20instructions\x20on\x20how\x20to\x20use\x20fish'
<W> fish: Unable to parse universal variable message: 'SET __fish_init_1_50_0:\x1d'
<W> fish: Unable to parse universal variable message: 'SET fish_color_normal:normal'
<W> fish: Unable to parse universal variable message: 'SET fish_color_command:brblue'
<W> fish: Unable to parse universal variable message: 'SET fish_color_param:cyan'
<W> fish: Unable to parse universal variable message: 'SET fish_color_redirection:normal'
<W> fish: Unable to parse universal variable message: 'SET fish_color_comment:red'
<W> fish: Unable to parse universal variable message: 'SET fish_color_error:red\x1e\x2d\x2dbold'
<W> fish: Unable to parse universal variable message: 'SET fish_color_escape:cyan'
<W> fish: Unable to parse universal variable message: 'SET fish_color_operator:cyan'
<W> fish: Unable to parse universal variable message: 'SET fish_color_end:green'
<W> fish: Unable to parse universal variable message: 'SET fish_color_quote:brown'
<W> fish: Unable to parse universal variable message: 'SET fish_color_autosuggestion:brgrey'
<W> fish: Unable to parse universal variable message: 'SET fish_color_user:green'
<W> fish: Unable to parse universal variable message: 'SET fish_color_host:normal'
<W> fish: Unable to parse universal variable message: 'SET fish_color_valid_path:\x2d\x2dunderline'
<W> fish: Unable to parse universal variable message: 'SET fish_color_cwd:green'
<W> fish: Unable to parse universal variable message: 'SET fish_color_cwd_root:red'
<W> fish: Unable to parse universal variable message: 'SET fish_color_match:cyan'
<W> fish: Unable to parse universal variable message: 'SET fish_color_search_match:\x2d\x2dbackground\x3dpurple'
<W> fish: Unable to parse universal variable message: 'SET fish_color_selection:\x2d\x2dbackground\x3dpurple'
<W> fish: Unable to parse universal variable message: 'SET fish_pager_color_prefix:cyan'
<W> fish: Unable to parse universal variable message: 'SET fish_pager_color_completion:normal'
<W> fish: Unable to parse universal variable message: 'SET fish_pager_color_description:brgrey'
<W> fish: Unable to parse universal variable message: 'SET fish_pager_color_progress:cyan'
<W> fish: Unable to parse universal variable message: 'SET fish_color_history_current:cyan'
There is no fish_key_bindings function called: 'fish_default_key_bindings'
Reverting to default bindings
<W> fish: Unable to parse universal variable message: 'SET fish_key_bindings:fish_default_key_bindings'
<W> fish: Unable to parse universal variable message: 'SET fish_key_bindings:fish_default_key_bindings'
There is no fish_key_bindings function called: ''
Reverting to default bindings

(previous 2 lines repeated 125 times...I think)

The function call stack limit has been exceeded. Do you have an accidental infinite loop?
fish: __fish_reload_key_bindings VARIABLE SET fish_key_bindings
      ^
in event handler: handler for variable 'fish_key_bindings'

There is no fish_key_bindings function called: ''
Reverting to default bindings

(previous 2 lines repeated approximately 9x10^zillion times)

The function call stack limit has been exceeded. Do you have an accidental infinite loop?
fish: __fish_reload_key_bindings VARIABLE SET fish_key_bindings
      ^
in event handler: handler for variable 'fish_key_bindings'

<W> fish: Unable to parse universal variable message: 'SET __fish_init_2_3_0:\x1d'
@ ~/home/james/source/fish-shell>

From here I have a shell and can execute external commands, but command completion routines in path.cpp:path_get_path_core cause a lot of pain:

@ ~/home/james/source/fish-shell> l<W> fish: Error while searching for command '/usr/xpg4/bin/l'
access: No such file or directory
<W> fish: Error while searching for command '/usr/local/sbin/l'
access: No such file or directory
<W> fish: Error while searching for command '/usr/local/bin/l'
access: No such file or directory
<W> fish: Error while searching for command '/opt/local/sbin/l'
access: No such file or directory
<W> fish: Error while searching for command '/opt/local/bin/l'
access: No such file or directory
<W> fish: Error while searching for command '/usr/sbin/l'
access: No such file or directory
<W> fish: Error while searching for command '/usr/bin/l'
access: No such file or directory
<W> fish: Error while searching for command '/sbin/l'
access: No such file or directory
ls
autom4te.cache           config.status            Dockerfile               fish_tests               Makefile                 src
build_tools              config.sub               Doxyfile                 FISH-BUILD-VERSION-FILE  Makefile.in              test
CHANGELOG.md             configure                Doxyfile.help            fish.pc                  obj                      tests
config.guess             configure.ac             Doxyfile.user            fish.pc.in               osx
config.h                 CONTRIBUTING.md          etc                      fish.spec.in             pcre2-10.21
config.h.in              COPYING                  fish                     fish.xcodeproj           po
config.h.in~             debian                   fish_indent              install-sh               README.md
config.log               doc_src                  fish_key_reader          lexicon_filter.in        share
@ ~/home/james/source/fish-shell>

I have done further amateur debugging (really, it's way past time I learned to use gdb) by emitting specific errno's and can make some conclusions about why this may be happening, but I will stop here in case you as the experts have any insights.

Thanks again and let me know if I should break this into separate issues, or if I can do any debugging on your behalf. Kind regards.

@zanchey zanchey referenced this issue Aug 30, 2016

Closed

Fish gives warning at start on Haiku #3317

2 of 2 tasks complete

zanchey added a commit that referenced this issue Aug 30, 2016

env_universal_common: drop MAP_FILE flag
MAP_FILE is unspecified or ignored on Linux, Solaris, FreeBSD and Haiku;
it is the default on OS X.

Work on #3317 & #3340.
@faho

This comment has been minimized.

Show comment
Hide comment
@faho

faho Aug 30, 2016

Member

The function call stack limit has been exceeded. Do you have an accidental infinite loop?
fish: __fish_reload_key_bindings VARIABLE SET fish_key_bindings

This looks like what happened on #3302, and it would also explain the later issues.

What's your locale? In 3302, it seemed to be related to FreeBSD's unicode handling - which lead to some truly bizarre issues.

Also, if you try fish from git you'll get a less verbose failure for __fish_reload_key_bindings.

Member

faho commented Aug 30, 2016

The function call stack limit has been exceeded. Do you have an accidental infinite loop?
fish: __fish_reload_key_bindings VARIABLE SET fish_key_bindings

This looks like what happened on #3302, and it would also explain the later issues.

What's your locale? In 3302, it seemed to be related to FreeBSD's unicode handling - which lead to some truly bizarre issues.

Also, if you try fish from git you'll get a less verbose failure for __fish_reload_key_bindings.

@jblachly

This comment has been minimized.

Show comment
Hide comment
@jblachly

jblachly Aug 30, 2016

@faho, my fish was compiled directly from git master 3702616 (now only one commit behind). I did run ./fish , but had previously sudo make install'ed ; the errors and warnings are the same even if running from /opt/local/bin.

Will check my locale settings when I get home tonight.

@faho, my fish was compiled directly from git master 3702616 (now only one commit behind). I did run ./fish , but had previously sudo make install'ed ; the errors and warnings are the same even if running from /opt/local/bin.

Will check my locale settings when I get home tonight.

@faho

This comment has been minimized.

Show comment
Hide comment
@faho

faho Aug 30, 2016

Member

From @floam's screencast in #3302, I think I might know why my fix is not working.

Can you confirm that functions -q fish_default_key_bindings; echo $status prints 0, but set -l foo fish_default_key_bindings; functions -q "$foo"; echo $status prints 1? Please enter this exactly as it is, it appears on both FreeBSD and Illumos we have some issues with Unicode that result in variables in quotes being expanded to nothing.

Member

faho commented Aug 30, 2016

From @floam's screencast in #3302, I think I might know why my fix is not working.

Can you confirm that functions -q fish_default_key_bindings; echo $status prints 0, but set -l foo fish_default_key_bindings; functions -q "$foo"; echo $status prints 1? Please enter this exactly as it is, it appears on both FreeBSD and Illumos we have some issues with Unicode that result in variables in quotes being expanded to nothing.

@jblachly

This comment has been minimized.

Show comment
Hide comment
@jblachly

jblachly Aug 31, 2016

(While prepping this message, I discovered that history --clear does not work on my platform)

@faho : Apologies in advance for the lengthy report. I pasted versions without the erroneous "Error while searching for..." failed completion as well, but wanted to also emphasize the problem of failures during command-completion on illumos (which AFAICT may be related in part, but not entirely, to non threadsafe errno ?)

Bottom line: The commands you suggested report 0 and 1 as you expected. Looks like the illumos troubles stem from elsewhere -- unrelated to FreeBSD problems ... sorry :|

Form one (without set foo)

@ ~/home/james/source/fish-shell> f<W> fish: Error while searching for command '/usr/xpg4/bin/f'
access: No such file or directory
<W> fish: Error while searching for command '/usr/local/sbin/f'
access: No such file or directory
<W> fish: Error while searching for command '/usr/local/bin/f'
access: No such file or directory
<W> fish: Error while searching for command '/opt/local/sbin/f'
access: No such file or directory
<W> fish: Error while searching for command '/opt/local/bin/f'
access: No such file or directory
<W> fish: Error while searching for command '/usr/sbin/f'
access: No such file or directory
<W> fish: Error while searching for command '/usr/bin/f'
access: No such file or directory
<W> fish: Error while searching for command '/sbin/f'
access: No such file or directory
fu<W> fish: Error while searching for command '/usr/xpg4/bin/fu'
access: No such file or directory
<W> fish: Error while searching for command '/usr/local/sbin/fu'
access: No such file or directory
<W> fish: Error while searching for command '/usr/local/bin/fu'
access: No such file or directory
<W> fish: Error while searching for command '/opt/local/sbin/fu'
access: No such file or directory
<W> fish: Error while searching for command '/opt/local/bin/fu'
access: No such file or directory
<W> fish: Error while searching for command '/usr/sbin/fu'
access: No such file or directory
<W> fish: Error while searching for command '/usr/bin/fu'
access: No such file or directory
<W> fish: Error while searching for command '/sbin/fu'
access: No such file or directory
functions <W> fish-: Error while searching for command '/usr/xpg4/bin/functions'
access: No such file or directory
<W> fish: Error while searching for command '/usr/local/sbin/functions'
access: No such file or directory
<W> fish: Error while searching for command '/usr/local/bin/functions'
access: No such file or directory
<W> fish: Error while searching for command '/opt/local/sbin/functions'
access: No such file or directory
<W> fish: Error while searching for command '/opt/local/bin/functions'
-access: No such file or directory
<W> fish: Error while searching for command '/usr/sbin/functions'
access: No such file or directory
<W> fish: Error while searching for command '/usr/bin/functions'
access: No such file or directory
<W> fish: Error while searching for command '/sbin/functions'
access: No such file or directory
q fish_default_key_bindings; echo $status
0

(now without errors)

@ ~/home/james/source/fish-shell> functions -q fish_default_key_bindings; echo $status
0

Form two (with set foo)

@ ~/home/james/source/fish-shell> <W> sfish: Error while searching for command '/usr/xpg4/bin/s'
access: No such file or directory
<W> fish: Error while searching for command '/usr/local/sbin/s'
access: No such file or directory
<W> fish: Error while searching for command '/usr/local/bin/s'
access: No such file or directory
<W> fish: Error while searching for command '/opt/local/sbin/s'
access: No such file or directory
<W> fish: Error while searching for command '/opt/local/bin/s'
access: No such file or directory
<W> fish: Error while searching for command '/usr/sbin/s'
access: No such file or directory
<W> fish: Error while searching for command '/usr/bin/s'
access: No such file or directory
<W> fish: Error while searching for command '/sbin/s'
access: No such file or directory
se<W> fish: Error while searching for command '/usr/xpg4/bin/se'
access: No such file or directory
<W> fish: Error while searching for command '/usr/local/sbin/se'
access: No such file or directory
<W> fish: Error while searching for command '/usr/local/bin/se'
access: No such file or directory
<W> fish: Error while searching for command '/opt/local/sbin/se'
access: No such file or directory
<W> fish: Error while searching for command '/opt/local/bin/se'
access: No such file or directory
<W> fish: Error while searching for command '/usr/sbin/se'
access: No such file or directory
<W> fish: Error while searching for command '/usr/bin/se'
access: No such file or directory
<W> fish: Error while searching for command '/sbin/se'
access: No such file or directory
set <W> fish: Error while searching for command '/usr/xpg4/bin/set'
access: -No such file or directory
<W> fish: Error while searching for command '/usr/local/sbin/set'
access: No such file or directory
<W> fish: Error while searching for command '/usr/local/bin/set'
access: No such file or directory
<W> fish: Error while searching for command '/opt/local/sbin/set'
access: No such file or directory
<W> fish: Error while searching for command '/opt/local/bin/set'
access: No such file or directory
<W> fish: Error while searching for command '/usr/sbin/set'
-access: No such file or directory
<W> fish: Error while searching for command '/usr/bin/set'
access: No such file or directory
<W> fish: Error while searching for command '/sbin/set'
access: No such file or directory
l foo fish_default_key_bindings; f<W> fish: Error while searching for command '/usr/xpg4/bin/fu'
access: No such file or directory
u<W> fish: Error while searching for command '/usr/local/sbin/fu'
access: No such file or directory
<W> fish: Error while searching for command '/usr/local/bin/fu'
access: No such file or directory
<W> fish: Error while searching for command '/opt/local/sbin/fu'
access: No such file or directory
<W> fish: Error while searching for command '/opt/local/bin/fu'
access: No such file or directory
<W> fish: Error while searching for command '/usr/sbin/fu'
access: No such file or directory
<W> fish: Error while searching for command '/usr/bin/fu'
access: No such file or directory
<W> fish: Error while searching for command '/sbin/fu'
access: No such file or directory
functions -q "$foo"; echo $status
1

And now succinctly removing the error messages:

@ ~/home/james/source/fish-shell> set -l fish_default_key_bindings; functions -q "$foo"; echo $status
1

jblachly commented Aug 31, 2016

(While prepping this message, I discovered that history --clear does not work on my platform)

@faho : Apologies in advance for the lengthy report. I pasted versions without the erroneous "Error while searching for..." failed completion as well, but wanted to also emphasize the problem of failures during command-completion on illumos (which AFAICT may be related in part, but not entirely, to non threadsafe errno ?)

Bottom line: The commands you suggested report 0 and 1 as you expected. Looks like the illumos troubles stem from elsewhere -- unrelated to FreeBSD problems ... sorry :|

Form one (without set foo)

@ ~/home/james/source/fish-shell> f<W> fish: Error while searching for command '/usr/xpg4/bin/f'
access: No such file or directory
<W> fish: Error while searching for command '/usr/local/sbin/f'
access: No such file or directory
<W> fish: Error while searching for command '/usr/local/bin/f'
access: No such file or directory
<W> fish: Error while searching for command '/opt/local/sbin/f'
access: No such file or directory
<W> fish: Error while searching for command '/opt/local/bin/f'
access: No such file or directory
<W> fish: Error while searching for command '/usr/sbin/f'
access: No such file or directory
<W> fish: Error while searching for command '/usr/bin/f'
access: No such file or directory
<W> fish: Error while searching for command '/sbin/f'
access: No such file or directory
fu<W> fish: Error while searching for command '/usr/xpg4/bin/fu'
access: No such file or directory
<W> fish: Error while searching for command '/usr/local/sbin/fu'
access: No such file or directory
<W> fish: Error while searching for command '/usr/local/bin/fu'
access: No such file or directory
<W> fish: Error while searching for command '/opt/local/sbin/fu'
access: No such file or directory
<W> fish: Error while searching for command '/opt/local/bin/fu'
access: No such file or directory
<W> fish: Error while searching for command '/usr/sbin/fu'
access: No such file or directory
<W> fish: Error while searching for command '/usr/bin/fu'
access: No such file or directory
<W> fish: Error while searching for command '/sbin/fu'
access: No such file or directory
functions <W> fish-: Error while searching for command '/usr/xpg4/bin/functions'
access: No such file or directory
<W> fish: Error while searching for command '/usr/local/sbin/functions'
access: No such file or directory
<W> fish: Error while searching for command '/usr/local/bin/functions'
access: No such file or directory
<W> fish: Error while searching for command '/opt/local/sbin/functions'
access: No such file or directory
<W> fish: Error while searching for command '/opt/local/bin/functions'
-access: No such file or directory
<W> fish: Error while searching for command '/usr/sbin/functions'
access: No such file or directory
<W> fish: Error while searching for command '/usr/bin/functions'
access: No such file or directory
<W> fish: Error while searching for command '/sbin/functions'
access: No such file or directory
q fish_default_key_bindings; echo $status
0

(now without errors)

@ ~/home/james/source/fish-shell> functions -q fish_default_key_bindings; echo $status
0

Form two (with set foo)

@ ~/home/james/source/fish-shell> <W> sfish: Error while searching for command '/usr/xpg4/bin/s'
access: No such file or directory
<W> fish: Error while searching for command '/usr/local/sbin/s'
access: No such file or directory
<W> fish: Error while searching for command '/usr/local/bin/s'
access: No such file or directory
<W> fish: Error while searching for command '/opt/local/sbin/s'
access: No such file or directory
<W> fish: Error while searching for command '/opt/local/bin/s'
access: No such file or directory
<W> fish: Error while searching for command '/usr/sbin/s'
access: No such file or directory
<W> fish: Error while searching for command '/usr/bin/s'
access: No such file or directory
<W> fish: Error while searching for command '/sbin/s'
access: No such file or directory
se<W> fish: Error while searching for command '/usr/xpg4/bin/se'
access: No such file or directory
<W> fish: Error while searching for command '/usr/local/sbin/se'
access: No such file or directory
<W> fish: Error while searching for command '/usr/local/bin/se'
access: No such file or directory
<W> fish: Error while searching for command '/opt/local/sbin/se'
access: No such file or directory
<W> fish: Error while searching for command '/opt/local/bin/se'
access: No such file or directory
<W> fish: Error while searching for command '/usr/sbin/se'
access: No such file or directory
<W> fish: Error while searching for command '/usr/bin/se'
access: No such file or directory
<W> fish: Error while searching for command '/sbin/se'
access: No such file or directory
set <W> fish: Error while searching for command '/usr/xpg4/bin/set'
access: -No such file or directory
<W> fish: Error while searching for command '/usr/local/sbin/set'
access: No such file or directory
<W> fish: Error while searching for command '/usr/local/bin/set'
access: No such file or directory
<W> fish: Error while searching for command '/opt/local/sbin/set'
access: No such file or directory
<W> fish: Error while searching for command '/opt/local/bin/set'
access: No such file or directory
<W> fish: Error while searching for command '/usr/sbin/set'
-access: No such file or directory
<W> fish: Error while searching for command '/usr/bin/set'
access: No such file or directory
<W> fish: Error while searching for command '/sbin/set'
access: No such file or directory
l foo fish_default_key_bindings; f<W> fish: Error while searching for command '/usr/xpg4/bin/fu'
access: No such file or directory
u<W> fish: Error while searching for command '/usr/local/sbin/fu'
access: No such file or directory
<W> fish: Error while searching for command '/usr/local/bin/fu'
access: No such file or directory
<W> fish: Error while searching for command '/opt/local/sbin/fu'
access: No such file or directory
<W> fish: Error while searching for command '/opt/local/bin/fu'
access: No such file or directory
<W> fish: Error while searching for command '/usr/sbin/fu'
access: No such file or directory
<W> fish: Error while searching for command '/usr/bin/fu'
access: No such file or directory
<W> fish: Error while searching for command '/sbin/fu'
access: No such file or directory
functions -q "$foo"; echo $status
1

And now succinctly removing the error messages:

@ ~/home/james/source/fish-shell> set -l fish_default_key_bindings; functions -q "$foo"; echo $status
1
@jblachly

This comment has been minimized.

Show comment
Hide comment
@jblachly

jblachly Aug 31, 2016

So although I doubt this will solve all my fish woes, I found an error in my platform's implementation of wcsncasecmp and have filed an issue.

Let's put debugging for this platform on hold until I get that sorted out. There are definitely other problems that I would like to get fixed; I appreciate everyone's help and I am very much looking forward to trying the fish shell.

So although I doubt this will solve all my fish woes, I found an error in my platform's implementation of wcsncasecmp and have filed an issue.

Let's put debugging for this platform on hold until I get that sorted out. There are definitely other problems that I would like to get fixed; I appreciate everyone's help and I am very much looking forward to trying the fish shell.

@faho

This comment has been minimized.

Show comment
Hide comment
@faho

faho Aug 31, 2016

Member

Bottom line: The commands you suggested report 0 and 1 as you expected. Looks like the illumos troubles stem from elsewhere -- unrelated to FreeBSD problems ... sorry :|

What I expected was just what happens with FreeBSD. They are related.

The issue is that a variable inside quotes expands to nothing. That's just like #3302.

Member

faho commented Aug 31, 2016

Bottom line: The commands you suggested report 0 and 1 as you expected. Looks like the illumos troubles stem from elsewhere -- unrelated to FreeBSD problems ... sorry :|

What I expected was just what happens with FreeBSD. They are related.

The issue is that a variable inside quotes expands to nothing. That's just like #3302.

@jblachly

This comment has been minimized.

Show comment
Hide comment
@jblachly

jblachly Aug 31, 2016

Sorry then, misunderstood your expected results. So if this explains the lack of default keybindings , and the illumos wcsncasecmp error explains my failure to properly evaluate SET statements perhaps I am closer than I think.

Sorry then, misunderstood your expected results. So if this explains the lack of default keybindings , and the illumos wcsncasecmp error explains my failure to properly evaluate SET statements perhaps I am closer than I think.

@jblachly

This comment has been minimized.

Show comment
Hide comment
@jblachly

jblachly Sep 21, 2016

Updates:

I patched illumos to include MAP_FILE as a flag for mmap, although I see that @zanchey made a commit to get rid of that in the interim.

Robert Mustacchi of Joyent verified the wcsncasecmp bug and patched it in illumos

This improves interactive behavior including command suggestion noticeably for the build I was working on at the time I opened this issue at beginning of September. ( 3702616 )

However, the fish-shell current HEAD (master) will not build, so I performed a git bisect. The change to clang (06c658d) is the first commit that fails to build on illumos:

(warnings removed)
  CXX = 'clang++'
  CXXFLAGS = '-g -O2 -fno-exceptions -Wextra -Ipcre2-10.21/src '
  MKDIR_P = '/opt/local/bin/mkdir -p'
  mkdir    obj
  CPPFLAGS = '-DLOCALEDIR="/usr/local/share/locale" -DPREFIX=L"/usr/local" -DDATADIR=L"/usr/local/share" -DSYSCONFDIR=L"/usr/local/etc" -DBINDIR=L"/usr/local/bin" -DDOCDIR=L"/usr/local/share/doc/fish" -iquote. -iquote./src/ -DFISH_BUILD_VERSION="2.3.1-543-g06c658d"'
  CXX      obj/fish.o
... (warnings removed)...
  CXX      obj/builtin_set_color.o
src/builtin_set_color.cpp:144:9: error: use of undeclared identifier 'cur_term'
    if (cur_term == NULL && setupterm(0, STDOUT_FILENO, &errret) == ERR) {
        ^
src/builtin_set_color.cpp:151:10: error: use of undeclared identifier 'exit_attribute_mode'
    if (!exit_attribute_mode) {
         ^
src/builtin_set_color.cpp:163:13: error: use of undeclared identifier 'enter_bold_mode'
        if (enter_bold_mode) writembs(tparm(enter_bold_mode));
            ^
src/builtin_set_color.cpp:163:45: error: use of undeclared identifier 'enter_bold_mode'
        if (enter_bold_mode) writembs(tparm(enter_bold_mode));
                                            ^
./src/output.h:32:39: note: expanded from macro 'writembs'
#define writembs(mbs) writembs_check((mbs), #mbs, __FILE__, __LINE__)
                                      ^
src/builtin_set_color.cpp:167:13: error: use of undeclared identifier 'enter_underline_mode'
        if (enter_underline_mode) writembs(enter_underline_mode);
            ^
src/builtin_set_color.cpp:167:44: error: use of undeclared identifier 'enter_underline_mode'
        if (enter_underline_mode) writembs(enter_underline_mode);
                                           ^
./src/output.h:32:39: note: expanded from macro 'writembs'
#define writembs(mbs) writembs_check((mbs), #mbs, __FILE__, __LINE__)
                                      ^
src/builtin_set_color.cpp:173:28: error: use of undeclared identifier 'exit_attribute_mode'
            writembs(tparm(exit_attribute_mode));
                           ^
./src/output.h:32:39: note: expanded from macro 'writembs'
#define writembs(mbs) writembs_check((mbs), #mbs, __FILE__, __LINE__)
                                      ^
src/builtin_set_color.cpp:180:28: error: use of undeclared identifier 'exit_attribute_mode'
            writembs(tparm(exit_attribute_mode));
                           ^
./src/output.h:32:39: note: expanded from macro 'writembs'
#define writembs(mbs) writembs_check((mbs), #mbs, __FILE__, __LINE__)
                                      ^
8 errors generated.
Makefile:801: recipe for target 'obj/builtin_set_color.o' failed
make: *** [obj/builtin_set_color.o] Error 1

The changes were to Makefile.in and configure.ac. I looked through them briefly but as I know neither clang nor autotools well, I did not see any obvious culprit.

Once again, I really appreciate everyone's help and suggestions. I want to see this port through to the end but need help with a few of the hurdles.

Updates:

I patched illumos to include MAP_FILE as a flag for mmap, although I see that @zanchey made a commit to get rid of that in the interim.

Robert Mustacchi of Joyent verified the wcsncasecmp bug and patched it in illumos

This improves interactive behavior including command suggestion noticeably for the build I was working on at the time I opened this issue at beginning of September. ( 3702616 )

However, the fish-shell current HEAD (master) will not build, so I performed a git bisect. The change to clang (06c658d) is the first commit that fails to build on illumos:

(warnings removed)
  CXX = 'clang++'
  CXXFLAGS = '-g -O2 -fno-exceptions -Wextra -Ipcre2-10.21/src '
  MKDIR_P = '/opt/local/bin/mkdir -p'
  mkdir    obj
  CPPFLAGS = '-DLOCALEDIR="/usr/local/share/locale" -DPREFIX=L"/usr/local" -DDATADIR=L"/usr/local/share" -DSYSCONFDIR=L"/usr/local/etc" -DBINDIR=L"/usr/local/bin" -DDOCDIR=L"/usr/local/share/doc/fish" -iquote. -iquote./src/ -DFISH_BUILD_VERSION="2.3.1-543-g06c658d"'
  CXX      obj/fish.o
... (warnings removed)...
  CXX      obj/builtin_set_color.o
src/builtin_set_color.cpp:144:9: error: use of undeclared identifier 'cur_term'
    if (cur_term == NULL && setupterm(0, STDOUT_FILENO, &errret) == ERR) {
        ^
src/builtin_set_color.cpp:151:10: error: use of undeclared identifier 'exit_attribute_mode'
    if (!exit_attribute_mode) {
         ^
src/builtin_set_color.cpp:163:13: error: use of undeclared identifier 'enter_bold_mode'
        if (enter_bold_mode) writembs(tparm(enter_bold_mode));
            ^
src/builtin_set_color.cpp:163:45: error: use of undeclared identifier 'enter_bold_mode'
        if (enter_bold_mode) writembs(tparm(enter_bold_mode));
                                            ^
./src/output.h:32:39: note: expanded from macro 'writembs'
#define writembs(mbs) writembs_check((mbs), #mbs, __FILE__, __LINE__)
                                      ^
src/builtin_set_color.cpp:167:13: error: use of undeclared identifier 'enter_underline_mode'
        if (enter_underline_mode) writembs(enter_underline_mode);
            ^
src/builtin_set_color.cpp:167:44: error: use of undeclared identifier 'enter_underline_mode'
        if (enter_underline_mode) writembs(enter_underline_mode);
                                           ^
./src/output.h:32:39: note: expanded from macro 'writembs'
#define writembs(mbs) writembs_check((mbs), #mbs, __FILE__, __LINE__)
                                      ^
src/builtin_set_color.cpp:173:28: error: use of undeclared identifier 'exit_attribute_mode'
            writembs(tparm(exit_attribute_mode));
                           ^
./src/output.h:32:39: note: expanded from macro 'writembs'
#define writembs(mbs) writembs_check((mbs), #mbs, __FILE__, __LINE__)
                                      ^
src/builtin_set_color.cpp:180:28: error: use of undeclared identifier 'exit_attribute_mode'
            writembs(tparm(exit_attribute_mode));
                           ^
./src/output.h:32:39: note: expanded from macro 'writembs'
#define writembs(mbs) writembs_check((mbs), #mbs, __FILE__, __LINE__)
                                      ^
8 errors generated.
Makefile:801: recipe for target 'obj/builtin_set_color.o' failed
make: *** [obj/builtin_set_color.o] Error 1

The changes were to Makefile.in and configure.ac. I looked through them briefly but as I know neither clang nor autotools well, I did not see any obvious culprit.

Once again, I really appreciate everyone's help and suggestions. I want to see this port through to the end but need help with a few of the hurdles.

@zanchey

This comment has been minimized.

Show comment
Hide comment
@zanchey

zanchey Sep 21, 2016

Member

It might be worth trying to build with your standard system compiler (GCC?) rather than LLVM; I don't know why the change was made.

Perhaps try:
./configure CXX=g++; make (edited; was CXX=gcc)
and if that's no good then
./configure; make V=1
to get a full verbose log; if you can paste the verbose log into a gist then we can look through it.

Member

zanchey commented Sep 21, 2016

It might be worth trying to build with your standard system compiler (GCC?) rather than LLVM; I don't know why the change was made.

Perhaps try:
./configure CXX=g++; make (edited; was CXX=gcc)
and if that's no good then
./configure; make V=1
to get a full verbose log; if you can paste the verbose log into a gist then we can look through it.

@jblachly

This comment has been minimized.

Show comment
Hide comment
@jblachly

jblachly Sep 26, 2016

@zanchey Compiles with g++. I will try to track down the clang issue eventually but higher priority is fixing what looks like some platform differences in syscalls (getcwd seems problematic for one). Thanks everyone for your help so far.

@zanchey Compiles with g++. I will try to track down the clang issue eventually but higher priority is fixing what looks like some platform differences in syscalls (getcwd seems problematic for one). Thanks everyone for your help so far.

zanchey referenced this issue Oct 2, 2016

Autotools build: teach it clang, utility macros
Teach autotools about clang++.
 - Use AC macros for these utilities in Makefile:
    LN_S, MKDIR_P, AWK, GREP, FGREP.
   This has the effect on OS X with prefixed coreutils installed
   from macports: > make show-LN_S show-MKDIR_P show-AWK show-GREP
    LN_S = 'ln -s'
    MKDIR_P = '/opt/local/bin/gmkdir -p'
    AWK = 'awk'
    GREP = '/opt/local/bin/grep'
    FGREP = '/opt/local/bin/grep -F'
 - Use GNU Make findstrings, wildcard,notdir,
 - SHELL = @shell@ per reccomended practice and in line with
    actual behavior.
 - Add output for string wrangling steps

zanchey added a commit to zanchey/fish-shell that referenced this issue Nov 30, 2016

Check for functions in std namespace
On Solaris, some standard wide character functions are only contained in
the std:: namespace. The configure script now checks for these, enabling
the appropriate `uses` statements in src/common.h.

The checks are handwritten, because Autoconf's AC_CHECK_FUNC macro
always uses C linkage, but the problem only appears under C++ linkage.

Work on #3340.

zanchey added a commit to zanchey/fish-shell that referenced this issue Nov 30, 2016

add flock fallback
Import the flock compatibility wrapper from NetBSD.

Work on #3340.

zanchey added a commit to zanchey/fish-shell that referenced this issue Nov 30, 2016

configure: check that errno is threadsafe
Without `-pthread` specified to the compiler, errno is not threadsafe on
Solaris (as _REENTRANT is undefined, and _POSIX_C_SOURCE may not be set
until after the inclusion of <errno.h>).

Work on #3340.

zanchey added a commit to zanchey/fish-shell that referenced this issue Nov 30, 2016

configure: force use of `-pthread` on Solaris
This fixes a problem with non-threadsafe errno.

Ideally, this would be the use of the AX_PTHREAD macro, but it is GPL 3+
only, which is incompatible with the GPL 2 license of fish. It also
would need extending to cover C++.

For now, fish doesn't build on anything except GCC under Solaris anyway,
so `-pthread` is the right thing to use.

Work on #3340.

zanchey added a commit to zanchey/fish-shell that referenced this issue Nov 30, 2016

Check for functions in std namespace
On Solaris, some standard wide character functions are only contained in
the std:: namespace. The configure script now checks for these, enabling
the appropriate `uses` statements in src/common.h.

The checks are handwritten, because Autoconf's AC_CHECK_FUNC macro
always uses C linkage, but the problem only appears under C++ linkage.

Work on #3340.

zanchey added a commit to zanchey/fish-shell that referenced this issue Nov 30, 2016

add flock fallback
Import the flock compatibility wrapper from NetBSD.

Work on #3340.

zanchey added a commit to zanchey/fish-shell that referenced this issue Nov 30, 2016

configure: check that errno is threadsafe
Without `-pthread` specified to the compiler, errno is not threadsafe on
Solaris (as _REENTRANT is undefined, and _POSIX_C_SOURCE may not be set
until after the inclusion of <errno.h>).

Work on #3340.

zanchey added a commit to zanchey/fish-shell that referenced this issue Nov 30, 2016

configure: force use of `-pthread` on Solaris
This fixes a problem with non-threadsafe errno.

Ideally, this would be the use of the AX_PTHREAD macro, but it is GPL 3+
only, which is incompatible with the GPL 2 license of fish. It also
would need extending to cover C++.

For now, fish doesn't build on anything except GCC under Solaris anyway,
so `-pthread` is the right thing to use.

Work on #3340.
@zanchey

This comment has been minimized.

Show comment
Hide comment
@zanchey

zanchey Dec 3, 2016

Member

Could you try #3608? That produces a working binary for me on Solaris 11.

Member

zanchey commented Dec 3, 2016

Could you try #3608? That produces a working binary for me on Solaris 11.

zanchey added a commit that referenced this issue Dec 3, 2016

Check for functions in std namespace
On Solaris, some standard wide character functions are only contained in
the std:: namespace. The configure script now checks for these, enabling
the appropriate `uses` statements in src/common.h.

The checks are handwritten, because Autoconf's AC_CHECK_FUNC macro
always uses C linkage, but the problem only appears under C++ linkage.

Work on #3340.

zanchey added a commit that referenced this issue Dec 3, 2016

add flock fallback
Import the flock compatibility wrapper from NetBSD.

Work on #3340.

zanchey added a commit that referenced this issue Dec 3, 2016

configure: check that errno is threadsafe
Without `-pthread` specified to the compiler, errno is not threadsafe on
Solaris (as _REENTRANT is undefined, and _POSIX_C_SOURCE may not be set
until after the inclusion of <errno.h>).

Work on #3340.

zanchey added a commit that referenced this issue Dec 3, 2016

configure: force use of `-pthread` on Solaris
This fixes a problem with non-threadsafe errno.

Ideally, this would be the use of the AX_PTHREAD macro, but it is GPL 3+
only, which is incompatible with the GPL 2 license of fish. It also
would need extending to cover C++.

For now, fish doesn't build on anything except GCC under Solaris anyway,
so `-pthread` is the right thing to use.

Work on #3340.
@jblachly

This comment has been minimized.

Show comment
Hide comment
@jblachly

jblachly Dec 29, 2016

@zanchey Accidentally commented on #3608 instead of here.

Bottom line, latest commit compiles great; I included a short patch (below) to fix a getcwd error that I previously mentioned. 👍

Fix that and we are nearly 100% functional on illumos. I now need to track down some kind of ANSI error.

james@fish-build:~/fish-shell$ git diff
diff --git a/src/wutil.cpp b/src/wutil.cpp
index 3d8c914..2cff992 100644
--- a/src/wutil.cpp
+++ b/src/wutil.cpp
@@ -129,7 +129,7 @@ bool wreaddir_for_dirs(DIR *dir, wcstring *out_name) {
 const wcstring wgetcwd() {
     wcstring retval;

-    char *res = getcwd(NULL, 0);
+    char *res = getcwd(NULL, PATH_MAX);
     if (res) {
         retval = str2wcstring(res);
         free(res);

@zanchey Accidentally commented on #3608 instead of here.

Bottom line, latest commit compiles great; I included a short patch (below) to fix a getcwd error that I previously mentioned. 👍

Fix that and we are nearly 100% functional on illumos. I now need to track down some kind of ANSI error.

james@fish-build:~/fish-shell$ git diff
diff --git a/src/wutil.cpp b/src/wutil.cpp
index 3d8c914..2cff992 100644
--- a/src/wutil.cpp
+++ b/src/wutil.cpp
@@ -129,7 +129,7 @@ bool wreaddir_for_dirs(DIR *dir, wcstring *out_name) {
 const wcstring wgetcwd() {
     wcstring retval;

-    char *res = getcwd(NULL, 0);
+    char *res = getcwd(NULL, PATH_MAX);
     if (res) {
         retval = str2wcstring(res);
         free(res);
@krader1961

This comment has been minimized.

Show comment
Hide comment
@krader1961

krader1961 Dec 29, 2016

Contributor

One of the relevant standards for getcwd() says that passing NULL for the buffer isn't required to be supported. And if we have to specify a maximum length we might as well allocate the buffer ourself. Also, looking at our code we are inconsistent about how we invoke getcwd() (do grep PATH_MAX src/*). That should also be fixed by normalizing on a single pattern. Perhaps even creating a fish_getcwd() to ensure we have a single implementation the rest of fish can use.

Contributor

krader1961 commented Dec 29, 2016

One of the relevant standards for getcwd() says that passing NULL for the buffer isn't required to be supported. And if we have to specify a maximum length we might as well allocate the buffer ourself. Also, looking at our code we are inconsistent about how we invoke getcwd() (do grep PATH_MAX src/*). That should also be fixed by normalizing on a single pattern. Perhaps even creating a fish_getcwd() to ensure we have a single implementation the rest of fish can use.

@jblachly

This comment has been minimized.

Show comment
Hide comment
@jblachly

jblachly Dec 29, 2016

Re. the ANSI escape code and unicode character output between commands, e.g.:

james@fish-build:~/fish-shell$ fish
30m30m⏎
james@fish-build ~/fish-shell>
30m⏎
james@fish-build ~/fish-shell> echo "hello world"
30mhello world
30m⏎

I tracked it down with git bisect to the following commit: 01dbfb0

The changes are in reader.cpp ; I will do some further digging to see what's up.

The unicode "return arrow" comes from common.cpp:
omitted_newline_char = can_be_encoded(L'\x23CE') ? L'\x23CE' : L'~';

But I'm not sure why the \e30m reset is being displayed before every line (note that it appears before the 'hello world' output) nor why the unicode return arrow is being displayed every time immediately prior to a prompt. BEL (\a) is also being emitted audibly.

I should also note that apart from this, colors work perfectly well , so hopefully that will make it easier to track down -- in src/reader.cpp ?

Re. the ANSI escape code and unicode character output between commands, e.g.:

james@fish-build:~/fish-shell$ fish
30m30m⏎
james@fish-build ~/fish-shell>
30m⏎
james@fish-build ~/fish-shell> echo "hello world"
30mhello world
30m⏎

I tracked it down with git bisect to the following commit: 01dbfb0

The changes are in reader.cpp ; I will do some further digging to see what's up.

The unicode "return arrow" comes from common.cpp:
omitted_newline_char = can_be_encoded(L'\x23CE') ? L'\x23CE' : L'~';

But I'm not sure why the \e30m reset is being displayed before every line (note that it appears before the 'hello world' output) nor why the unicode return arrow is being displayed every time immediately prior to a prompt. BEL (\a) is also being emitted audibly.

I should also note that apart from this, colors work perfectly well , so hopefully that will make it easier to track down -- in src/reader.cpp ?

@krader1961

This comment has been minimized.

Show comment
Hide comment
@krader1961

krader1961 Dec 29, 2016

Contributor

No need, @jblachly. The problem is that some libc implementations don't allow mixing narrow and wide stdio functions on the same stream. BSD, where I tested that change, does allow you to mix them. A fix for that issue was merged a couple of hours ago.

Contributor

krader1961 commented Dec 29, 2016

No need, @jblachly. The problem is that some libc implementations don't allow mixing narrow and wide stdio functions on the same stream. BSD, where I tested that change, does allow you to mix them. A fix for that issue was merged a couple of hours ago.

@jblachly

This comment has been minimized.

Show comment
Hide comment
@jblachly

jblachly Dec 30, 2016

@krader1961 Unfortunately that commit doesn't fix the behavior I'm seeing over here.

@krader1961 Unfortunately that commit doesn't fix the behavior I'm seeing over here.

@krader1961

This comment has been minimized.

Show comment
Hide comment
@krader1961

krader1961 Dec 30, 2016

Contributor

The \e30m sequence on most terminals sets the foreground color to black; e.g., set_color black or equivalent. Ignore the return arrow as that just means the last line of output from the previous command did not end with a newline. It's just a symptom of the real problem.

Please run script typescript.txt fish then exit and attach the typescript.txt file to this issue.

Also, when you say "that commit" do you mean 1621fa4? If you're running a fish with that fix I don't see how the other changes in commit 01dbfb0 could possibly account for the problem you're seeing.

Contributor

krader1961 commented Dec 30, 2016

The \e30m sequence on most terminals sets the foreground color to black; e.g., set_color black or equivalent. Ignore the return arrow as that just means the last line of output from the previous command did not end with a newline. It's just a symptom of the real problem.

Please run script typescript.txt fish then exit and attach the typescript.txt file to this issue.

Also, when you say "that commit" do you mean 1621fa4? If you're running a fish with that fix I don't see how the other changes in commit 01dbfb0 could possibly account for the problem you're seeing.

@zanchey

This comment has been minimized.

Show comment
Hide comment
@zanchey

zanchey Jan 9, 2017

Member

Perhaps even creating a fish_getcwd() to ensure we have a single implementation the rest of fish can use.

SGTM

Member

zanchey commented Jan 9, 2017

Perhaps even creating a fish_getcwd() to ensure we have a single implementation the rest of fish can use.

SGTM

@jblachly

This comment has been minimized.

Show comment
Hide comment
@jblachly

jblachly Mar 12, 2017

Some commit between Dec 29 and now fixed the problem with the ANSI escape code being displayed.

The only outstanding issues I found on a first pass are:

  1. getcwd (which I patch on my end to pass PATH_MAX instead of 0 as second param) and
  2. make install having some difficulty installing man pages which I haven't tracked down yet.

V. excited to get fish shell working on illumos! Thanks so much for your great work along the way.

Likely, passing PATH_MAX is reasonable solution (and it is used in wgetrealpath anyway). If you want to tackle the NULL issue and allocate the buffer yourself, could we at least pass nonzero for the buflen in the meantime to maximize compatibility?

Some commit between Dec 29 and now fixed the problem with the ANSI escape code being displayed.

The only outstanding issues I found on a first pass are:

  1. getcwd (which I patch on my end to pass PATH_MAX instead of 0 as second param) and
  2. make install having some difficulty installing man pages which I haven't tracked down yet.

V. excited to get fish shell working on illumos! Thanks so much for your great work along the way.

Likely, passing PATH_MAX is reasonable solution (and it is used in wgetrealpath anyway). If you want to tackle the NULL issue and allocate the buffer yourself, could we at least pass nonzero for the buflen in the meantime to maximize compatibility?

@3eka

This comment has been minimized.

Show comment
Hide comment
@3eka

3eka Apr 26, 2017

Hi all,
I just entered into this discussion and wonder what is status?

I am running OpenIndiana (OI) OS (https://www.openindiana.org/) which is built on top of Illumos...

Tried to compile fish-2.5.0 (./configure --prefix=/opt/SFW; that prefix is in PATH) and have entered to few issues:

checking term.h presence... yes
configure: WARNING: term.h: present but cannot be compiled
configure: WARNING: term.h:     check for missing prerequisite headers?
configure: WARNING: term.h: see the Autoconf documentation
configure: WARNING: term.h:     section "Present But Cannot Be Compiled"
configure: WARNING: term.h: proceeding with the compiler's result
configure: WARNING:     ## ----------------------------------------------- ##
configure: WARNING:     ## Report this to fish-users@lists.sourceforge.net ##
configure: WARNING:     ## ----------------------------------------------- ##

Not sure if term.h issue is important?

$ gmake
  FISH_BUILD_VERSION = '2.5.0'
  CXX = '/usr/gcc/5/bin/g++ -std=c++11'
  CXXFLAGS = '-g -O2 -fno-exceptions -Wextra -pthread -Ipcre2-10.21/src '
  MKDIR_P = '/usr/gnu/bin/mkdir -p'
  SED = '/usr/gnu/bin/sed'
  AWK = 'gawk'
  FGREP = '/usr/gnu/bin/grep -F'
  doxygen  user_doc
sh: line 1: ./lexicon_filter: cannot execute [Permission denied]
  prefix = '/opt/SFW'
  sysconfdir = '/opt/SFW/etc'
  docdir = '/opt/SFW/share/doc/fish'
  extra_completionsdir = '/opt/SFW/share/fish/vendor_completions.d'
  extra_functionsdir = '/opt/SFW/share/fish/vendor_functions.d'
  extra_confdir = '/opt/SFW/share/fish/vendor_conf.d'
$ chmod 755 ./lexicon_filter
$ pfexec gmake install
  FISH_BUILD_VERSION = '2.5.0'
  CXX = '/usr/gcc/5/bin/g++ -std=c++11'
  CXXFLAGS = '-g -O2 -fno-exceptions -Wextra -pthread -Ipcre2-10.21/src '
  MKDIR_P = '/usr/gnu/bin/mkdir -p'
  SED = '/usr/gnu/bin/sed'
  AWK = 'gawk'
  FGREP = '/usr/gnu/bin/grep -F'
  doxygen  user_doc
/nobackup/Build/fish-2.5.0/doc.h:6189: warning: Reached end of file while still inside a (nested) comment. Nesting level 1 (probable line reference: 1)
  prefix = '/opt/SFW'
  sysconfdir = '/opt/SFW/etc'
  docdir = '/opt/SFW/share/doc/fish'
  extra_completionsdir = '/opt/SFW/share/fish/vendor_completions.d'
  extra_functionsdir = '/opt/SFW/share/fish/vendor_functions.d'
  extra_confdir = '/opt/SFW/share/fish/vendor_conf.d'
fish has now been built.
Run gmake install to install fish.
  HAVE_GETTEXT = '1'
Installing translations...
  datadir = '/opt/SFW/share'
  mandir = '/opt/SFW/share/man'
 Installing fish
 Installing fish_indent
 Installing fish_key_reader
Creating sysconfdir tree  
Creating datadir tree
Creating placeholder vendor/'extra_' directories
Installing pkgconfig file 
Installing the fish completion library...
Installing fish functions
Installing man pages
Installing helper tools
Installing online user documentation
Installing more man pages
/usr/gnu/bin/install: cannot stat 'share/man/man1/fish.1': No such file or directory
/usr/gnu/bin/install: cannot stat 'share/man/man1/fish_indent.1': No such file or directory
/usr/gnu/bin/install: cannot stat 'share/man/man1/fish_key_reader.1': No such file or directory
Checking for legacy binaries...

fish is now installed on your system.
To run fish, type fish in your terminal.
Even if you are already in fish, you should now start a new fish session.

To set your colors, run fish_config
To scan your man pages for completions, run fish_update_completions
To accept autosuggestions (in grey) as you type, hit ctrl-F or right arrow key.

Have fun! <><

Had to change permissions of lexicon_filter to 775 (that should be easy to fix).
Not sure about failing to build man pages...

When running fish:

$ rm -rf ~/.config/fish/
$ fish
<E> fish: getcwd() failed with errno 22/Invalid argument
<E> fish: Could not determine current working directory. Is your locale set correctly?
<E> fish: getcwd() failed with errno 22/Invalid argument
<E> fish: getcwd() failed with errno 22/Invalid argument

This is really annoying and I wonder if MAX_PATH fix (from top of discussion) is still relevant (or it has to be added to source)?

I have got same results when trying to build from git clone source tree.

This looks very interesting and I would like to have it working on OI...

Thanks and best regards.

3eka commented Apr 26, 2017

Hi all,
I just entered into this discussion and wonder what is status?

I am running OpenIndiana (OI) OS (https://www.openindiana.org/) which is built on top of Illumos...

Tried to compile fish-2.5.0 (./configure --prefix=/opt/SFW; that prefix is in PATH) and have entered to few issues:

checking term.h presence... yes
configure: WARNING: term.h: present but cannot be compiled
configure: WARNING: term.h:     check for missing prerequisite headers?
configure: WARNING: term.h: see the Autoconf documentation
configure: WARNING: term.h:     section "Present But Cannot Be Compiled"
configure: WARNING: term.h: proceeding with the compiler's result
configure: WARNING:     ## ----------------------------------------------- ##
configure: WARNING:     ## Report this to fish-users@lists.sourceforge.net ##
configure: WARNING:     ## ----------------------------------------------- ##

Not sure if term.h issue is important?

$ gmake
  FISH_BUILD_VERSION = '2.5.0'
  CXX = '/usr/gcc/5/bin/g++ -std=c++11'
  CXXFLAGS = '-g -O2 -fno-exceptions -Wextra -pthread -Ipcre2-10.21/src '
  MKDIR_P = '/usr/gnu/bin/mkdir -p'
  SED = '/usr/gnu/bin/sed'
  AWK = 'gawk'
  FGREP = '/usr/gnu/bin/grep -F'
  doxygen  user_doc
sh: line 1: ./lexicon_filter: cannot execute [Permission denied]
  prefix = '/opt/SFW'
  sysconfdir = '/opt/SFW/etc'
  docdir = '/opt/SFW/share/doc/fish'
  extra_completionsdir = '/opt/SFW/share/fish/vendor_completions.d'
  extra_functionsdir = '/opt/SFW/share/fish/vendor_functions.d'
  extra_confdir = '/opt/SFW/share/fish/vendor_conf.d'
$ chmod 755 ./lexicon_filter
$ pfexec gmake install
  FISH_BUILD_VERSION = '2.5.0'
  CXX = '/usr/gcc/5/bin/g++ -std=c++11'
  CXXFLAGS = '-g -O2 -fno-exceptions -Wextra -pthread -Ipcre2-10.21/src '
  MKDIR_P = '/usr/gnu/bin/mkdir -p'
  SED = '/usr/gnu/bin/sed'
  AWK = 'gawk'
  FGREP = '/usr/gnu/bin/grep -F'
  doxygen  user_doc
/nobackup/Build/fish-2.5.0/doc.h:6189: warning: Reached end of file while still inside a (nested) comment. Nesting level 1 (probable line reference: 1)
  prefix = '/opt/SFW'
  sysconfdir = '/opt/SFW/etc'
  docdir = '/opt/SFW/share/doc/fish'
  extra_completionsdir = '/opt/SFW/share/fish/vendor_completions.d'
  extra_functionsdir = '/opt/SFW/share/fish/vendor_functions.d'
  extra_confdir = '/opt/SFW/share/fish/vendor_conf.d'
fish has now been built.
Run gmake install to install fish.
  HAVE_GETTEXT = '1'
Installing translations...
  datadir = '/opt/SFW/share'
  mandir = '/opt/SFW/share/man'
 Installing fish
 Installing fish_indent
 Installing fish_key_reader
Creating sysconfdir tree  
Creating datadir tree
Creating placeholder vendor/'extra_' directories
Installing pkgconfig file 
Installing the fish completion library...
Installing fish functions
Installing man pages
Installing helper tools
Installing online user documentation
Installing more man pages
/usr/gnu/bin/install: cannot stat 'share/man/man1/fish.1': No such file or directory
/usr/gnu/bin/install: cannot stat 'share/man/man1/fish_indent.1': No such file or directory
/usr/gnu/bin/install: cannot stat 'share/man/man1/fish_key_reader.1': No such file or directory
Checking for legacy binaries...

fish is now installed on your system.
To run fish, type fish in your terminal.
Even if you are already in fish, you should now start a new fish session.

To set your colors, run fish_config
To scan your man pages for completions, run fish_update_completions
To accept autosuggestions (in grey) as you type, hit ctrl-F or right arrow key.

Have fun! <><

Had to change permissions of lexicon_filter to 775 (that should be easy to fix).
Not sure about failing to build man pages...

When running fish:

$ rm -rf ~/.config/fish/
$ fish
<E> fish: getcwd() failed with errno 22/Invalid argument
<E> fish: Could not determine current working directory. Is your locale set correctly?
<E> fish: getcwd() failed with errno 22/Invalid argument
<E> fish: getcwd() failed with errno 22/Invalid argument

This is really annoying and I wonder if MAX_PATH fix (from top of discussion) is still relevant (or it has to be added to source)?

I have got same results when trying to build from git clone source tree.

This looks very interesting and I would like to have it working on OI...

Thanks and best regards.

@jblachly

This comment has been minimized.

Show comment
Hide comment
@jblachly

jblachly Apr 27, 2017

@3eka PATH_MAX (see my post above yours) will fix the getcwd problem. It has not been incorporated into tree

@3eka PATH_MAX (see my post above yours) will fix the getcwd problem. It has not been incorporated into tree

nomaed added a commit to nomaed/fish-shell that referenced this issue Jul 17, 2017

env_universal_common: drop MAP_FILE flag
MAP_FILE is unspecified or ignored on Linux, Solaris, FreeBSD and Haiku;
it is the default on OS X.

Work on #3317 & #3340.

@robberer robberer referenced this issue in OpenIndiana/oi-userland Aug 20, 2017

Merged

add fish-2.6.0 #3442

@mqudsi mqudsi closed this in ffebe74 Sep 26, 2017

mqudsi added a commit that referenced this issue Sep 26, 2017

Support building on Solaris 11
Took care of remaining issues preventing fish from building on Solaris.
Mainly caused by some assumptions that certain defines are POSIX when
they are not (`NAME_MAX`).

Moved `NAME_MAX` defines to common.h - for some reason, it was being
defined in a cpp file (`env_universal_common.cpp`) even though it is used
in multiple source files.

Now compiles on Solaris 11 with GNU Make. Still some warnings because
fish was written with GNU getopt in mind and the Solaris version doesn't
use `const char *` but rather just `char *` for getopt values, but it
builds nevertheless.

Assuming this closes #3340

(cherry picked from commit ffebe74)
@mqudsi

This comment has been minimized.

Show comment
Hide comment
@mqudsi

mqudsi Sep 26, 2017

Contributor

The standard is pretty clear that these arguments have to be specified. What we're doing is definitely not portable (even if it's very common).

It seems that fish uses wgetcwd() throughout, which is a wrapper for getcwd() anyway... Makes it really easy to fix.

Contributor

mqudsi commented Sep 26, 2017

The standard is pretty clear that these arguments have to be specified. What we're doing is definitely not portable (even if it's very common).

It seems that fish uses wgetcwd() throughout, which is a wrapper for getcwd() anyway... Makes it really easy to fix.

mqudsi added a commit that referenced this issue Sep 26, 2017

Fix non-standard getcwd() invocation
The POSIX standard specifies that a buffer should be supplied to
getcwd(), not doing so is undefined (or rather, platform-defined)
behavior. This was causing the getcwd errors on illumos (though not seen
on Solaris 11) reported in #3340

Closes #3340

mqudsi added a commit that referenced this issue Sep 26, 2017

Fix non-standard getcwd() invocation
The POSIX standard specifies that a buffer should be supplied to
getcwd(), not doing so is undefined (or rather, platform-defined)
behavior. This was causing the getcwd errors on illumos (though not seen
on Solaris 11) reported in #3340

Closes #3340

(cherry picked from commit b495c68)

@faho faho modified the milestones: fish-future, fish-3.0 Jan 11, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment