Shell linter ignores nonextant and unreadable files

[sckendall2]

Example:

$ ksh -n /no/such; print $?
0
$ touch a.ksh
$ chmod 0 a.ksh
$ ksh -n a.ksh; print $?   
0

I see this behavior in every version of ksh93 including 1.0.10.

So if you misspell the name of your script, then it gets a clean bill of health! The behavior I expect is a sensible error message and a nonzero exit status.

[McDutchie]

Agreed. Thanks for the report.

[McDutchie]

So, this is not a bug as such, it's intentional behaviour, but it's harmful, especially for -n.

What's happening is this. If a file name argument is given and the file is not found or is inaccessible, then ksh tries the argument again as a script as if the -c option had been given.

ksh/src/cmd/ksh93/sh/main.c

Lines 271 to 282 in ffd52e5

	if(sp || errno!=ENOENT)
	{
	errormsg(SH_DICT,ERROR_system(ERROR_NOEXEC),e_open,name);
	UNREACHABLE();
	}
	/* try sh -c 'name "$@"' */
	sh_onoption(SH_CFLAG);
	sh.comdiv = (char*)sh_malloc(strlen(name)+7);
	name = strcopy(sh.comdiv,name);
	if(sh.st.dolc)
	strcopy(name," \"$@\"");
	goto shell_c;

For example, ksh 'echo hi' outputs hi if the file echo hi doesn't exist. I don't think that's particularly desirable at all. You get funny stuff like:

$ touch 'echo hi'  # create empty file
$ ksh 'echo hi'    # execute empty script
$ chmod 0 'echo hi'
$ ksh 'echo hi'    
hi

Unfortunately this behaviour has been in ksh93 from the beginning (1995 or before) so users would probably be upset if we got rid of now, especially on the 1.0 branch. I just might on the dev branch, though. Opinions welcome.

For -n, the same thing happens, but it's much more harmful, as you've observed. In your reproducers, the files /no/such and a.ksh are nonexistent or inaccessible, so ksh treats them as -c scripts, lints them, finds no syntax error and nothing to warn about, so quietly exits with status 0.

The following patch disables that behaviour if -n/noexecis active, so your reproducers give a sensible error message. It doesn't change the non-noexec case.

diff --git a/src/cmd/ksh93/sh/main.c b/src/cmd/ksh93/sh/main.c
index 7d8b4c2d9..a1438f349 100644
--- a/src/cmd/ksh93/sh/main.c
+++ b/src/cmd/ksh93/sh/main.c
@@ -268,9 +268,10 @@ noreturn void sh_main(int ac, char *av[], Shinit_f userinit)
 						if(isdir)
 							errno = EISDIR;
 						 error_info.id = av[0];
-						if(sp || errno!=ENOENT)
+						if(sp || errno!=ENOENT || sh_isoption(SH_NOEXEC))
 						{
-							errormsg(SH_DICT,ERROR_system(ERROR_NOEXEC),e_open,name);
+							int e = errno==ENOENT ? ERROR_NOENT : ERROR_NOEXEC;
+							errormsg(SH_DICT,ERROR_system(e),e_open,name);
 							UNREACHABLE();
 						}
 						/* try sh -c 'name "$@"' */

[sckendall2]

@McDutchie the change you propose--disable that behavior only in noexec mode--sounds good to me.

As to whether to fix the non-noexec behavior on the dev branch, I would say do it. FWIW: In decades of (among other things!) helping developers get their scripts, typically developed on ksh93, to be portable to other KornShell implementations, I've never run into this behavior--even though none of the other KornShells have it. I've seen many, many instances of nonportable ksh93 behaviors, but I haven't seen this one. (Except recently, with -n.)

[McDutchie]

There's even more weirdness with it; reading the source code again, I'm noticing it adds "$@" to the command string.

The effect:

$ ksh print one two three
one two three

The print command is not found on PATH, so this is effectively translated to: ksh -c 'print "$@"' dummy one two three. It looks like an attempt to make ksh more user friendly. But:

$ ksh echo one two three 
echo: echo: cannot execute [Exec format error]

Since the echo command is a file that is found on PATH, the hack is not activated so ksh tries to execute it as a shell script as normal, which fails because it's a binary file.

And of course:

$ ksh '{ print; }' one two three         
ksh: syntax error at line 1: `"$@"' unexpected

This whole thing is just inherently broken. It's also not documented anywhere, as far as I can find – not in the man page, not in the Bolsky & Korn book.

[sckendall2]

There's a weird thing I noticed a long time ago: you can dot a function.

$ function f { print hi; }
$ . f
hi

I mention this because I was thinking maybe it was the same thing, and that you might want to look at the implementation of dot. But it seems not to be the same thing. It must be a separate undocumented behavior.

$ . 'print hi'
-ksh: .: print hi: cannot open [No such file or directory]

I don't think dotting a function is worth worrying about or filing as a separate issue.

[JohnoKing]

There's a weird thing I noticed a long time ago: you can dot a function.

That's intentional functionality:

ksh/src/cmd/ksh93/bltins/misc.c

Lines 265 to 283 in ffd52e5

	/* check for KornShell style function first */
	np = nv_search(script,sh.fun_tree,0);
	if(np && is_afunction(np) && !nv_isattr(np,NV_FPOSIX) && !(sh_isoption(SH_POSIX) && context->bnode==SYSDOT))
	{
	if(!np->nvalue)
	{
	path_search(script,NULL,0);
	if(np->nvalue)
	{
	if(nv_isattr(np,NV_FPOSIX))
	np = 0;
	}
	else
	{
	errormsg(SH_DICT,ERROR_exit(1),e_found,script);
	UNREACHABLE();
	}
	}
	}

ksh/src/cmd/ksh93/bltins/misc.c

Lines 319 to 331 in ffd52e5

	if(np)
	{
	/* execute the function as though it were a dot script */
	sh_exec((Shnode_t*)(nv_funtree(np)),sh_isstate(SH_ERREXIT));
	}
	else
	{
	/* run the dot script */
	buffer = sh_malloc(IOBSIZE+1);
	iop = sfnew(NULL,buffer,IOBSIZE,fd,SFIO_READ);
	sh_offstate(SH_NOFORK);
	sh_eval(iop,sh_isstate(SH_PROFILE)?SH_FUNEVAL:0);
	}

It's always available for source and only available outside of POSIX mode for .. It's also documented in the man page:

ksh/src/cmd/ksh93/sh.1

Lines 6370 to 6385 in ffd52e5

	\(dg \f3\|.\fP \f2name\^\fP \*(OK \f2arg\^\fP .\|.\|. \*(CK
	If
	.I name\^
	is a function defined with the
	.B function
	.I name\^
	reserved word syntax,
	the function is executed in the current environment
	(as if it had been defined with the
	.IB name ()
	syntax).
	Otherwise if
	.I name\^
	refers to a file, the
	file is read in its entirety and the commands are
	executed in the current shell environment.