~/Documents/bin: #!sh, #!bc, ... -> not executable

[personalizedrefrigerator]

Summary

• #!command_name at the top of an executable text file seems to only work if command_name is python or lua.

Expected Behavior

~/Documents/bin/test1:

#!sh
echo "It works"

or

#!bc
print 'a test'

or

#!almake_shell
# ^ Part of almost_make pip package

echo "It doesn't work"

~/Documents/bin/test2:

#!python
print("It works")

$ chmod u+x ~/Documents/bin/test1
$ chmod u+x ~/Documents/bin/test2
$ test2
It works
$ test1
test1: command not found

test1 and test2 should produce the same output.

Thank you for your time!

[holzschu]

Yes, that is currently the case. The offending lines are in iOS_system, file iOS_system.m, lines 2296-2316:
https://github.com/holzschu/ios_system/blob/57e220e540c8da596b1331aeef27844f9ffd2b07/ios_system.m#L2296

I try to read the first line, infer if it is a scripting language and if so add it to the beginning of the command line.
The limited number of scripting languages has not been an issue until now, so thank you for raising that one.

I think I could just take the last word of the first line, if it begins with "#!". It's not going to work with "#!sh", for the time being, but it will work with other shells.

[personalizedrefrigerator]

I have almost no experience with ObjectiveC, so this is probably partially wrong, but maybe,

    ... lua case ... (line 2311)
    } else {
        // Leave scriptName untouched.
        // Ref: https://stackoverflow.com/questions/11325951/how-to-split-a-string-of-words-and-add-to-an-array-objective-c
        NSArray* newArgs = [firstLine componentsSeparatedByString: @" "];
        int numNewArgs = [newArgs count]; // Or is it .length???
        argc += numNewArgs + 1;
        argv = (char**) realloc(argv, sizeof(char*) * (argc));

        // Shift existing arguments
        for (int i = numNewArgs; i <= argc; i++) {
            argv[i] = argv[i - numNewArgs];
        }

        int i = 0;
        for (NSString* newArg in newArgs) {
            argv[i] = strdup(newArg.UTF8String);
            i++;
        }

        argv[i] = realloc(argv[i], locationName.length + 1);
        strcpy(argv[i], locationName.UTF8String);
        break;
    }

    if (scriptName) {
                                    // 2) insert script language at beginning of argument list
                                    argc += 1;
                                    argv = (char **)realloc(argv, sizeof(char*) * (argc + 1));
                                    // Move everything one step up
                                    for (int i = argc; i >= 1; i--) { argv[i] = argv[i-1]; }
                                    argv[1] = realloc(argv[1], locationName.length + 1);
                                    strcpy(argv[1], locationName.UTF8String);
                                    argv[0] = strdup(scriptName); // this one is new
                                    break;
    }

By adding the additional handling in an else, the ability to have /crazy/path/to/a/python/interpreter/python in the #!command here is preserved.

[holzschu]

I've opted for something more extreme:

NSCharacterSet *invertedAlphaNumCharSet = [[NSCharacterSet alphanumericCharacterSet] invertedSet];
NSArray *components = [firstLine componentsSeparatedByCharactersInSet:invertedAlphaNumCharSet];
NSString *scriptNameString = components.lastObject;

The name of the script is the last word of the line, word being defined as "a set of alphanumeric characters". This works with #!/any/path/to/perl, #!bc and also #! /usr/bin/env python. It will fail if the language name contains a hyphen, though.

The only thing it does is add the name of the command in front of the file, so it does not work so well with bc and sh. But it should work with almake_shell.

I'm uploading the new version now; it should be available for TestFlight in 24 to 36h.

[holzschu]

The new build has been submitted for TestFlight, but I just realized: it's not going to be enough for #!almake_shell, since it's not a working command. I need to recurse here if the command does not exist.

[personalizedrefrigerator]

Although

NSCharacterSet *invertedAlphaNumCharSet = [[NSCharacterSet alphanumericCharacterSet] invertedSet];
NSArray *components = [firstLine componentsSeparatedByCharactersInSet:invertedAlphaNumCharSet];
NSString *scriptNameString = components.lastObject;

works for commands like almake_shell and bc, it won't work for commands like #!bc -l...

[holzschu]

Ah, yes. It's going to be difficult to have a regex that matches both /usr/bin/env python and bc -l (or /usr/bin/env bc -l).
This needs more thinking.

[personalizedrefrigerator]

It looks like build 155 broke ipython. See #204.

[holzschu]

Build 157 has improved on the shebang analysis: all arguments are transferred to the beginning of the command line, so "#! bc -f" works now. "#!almake_shell" wont't work, because we don't loop back to the beginning to parse the new command if it's a file, but I think "#!python almake_shell" might work.

"#!sh" of course still won't work. I'm also not fully satisfied with the treatment of "python": to make it work with "python3.9", everything that contains the string "python" is remapped to "python", but that seems like too much, and something that will create issues later.

[personalizedrefrigerator]

#!python -m almost_make.utils.shellUtil.interactiveShell ...
is working!

[holzschu]

That's great!
Does that mean that there could be a way to execute small shell scripts?

[ifuchs]

I have a python script which I have set to executable using chmod. The first line of the script says #!python. However when I try to run it by typing the file name I am getting command not found. Should this work?

[personalizedrefrigerator]

I have a python script which I have set to executable using chmod. The first line of the script says #!python. However when I try to run it by typing the file name I am getting command not found. Should this work?

Is the python script in ~/Documents/bin? Does it show up as executable when you run ls -l path/to/script?

[holzschu]

You're right, I can reproduce this issue, and it should work (I'm also pretty sure it did). I'll have a look.

Sorry, my bad, #! python on the first line does work. What happened to me was that I had a different file with the same name in the current directory (that did not start with #! python). That's the problem with naming all your tests test.py.

[ifuchs]

My .py file is in the documents/bin directory and it is set to executable. It has
#! python on the first line. Nevertheless it is not working. Is there something else?

[ifuchs]

I stand corrected. It does work (there was a spurious blank line at the beginning of the file). However command completion does not work for this file. Should it?

[holzschu]

I think the list for commant completion is computed only once, at the start of the program. It should appear on the next run.

[ifuchs]

I had restarted it and still had the problem but I think I understand why. If I type the first few letters of the command and then hit tab for command completion I am getting a different command and I am not being shown any alternatives. If I type enough characters so that the command name is unambiguous and then hit tab, I get the command I’m looking for. Specifically, my command is wordleai.py and if I type word+tab, I get wordcloud.cli.

[holzschu]

Tab shows the first match, but you can use the arrows to scroll through the other possible matches (or, as you did, type more letters)

[ifuchs]

OK, it works except for one thing. If I type part of the command and then hit tab, the full name is filled in, but when I hit return I always get "command not found" BUT only the first time after starting ashell. Every subsequent attempt works fine.