Add One or More Linux Example and Documentation

[johnW-ret]

To not open a redundant issue, I have reviewed #1, #23, and #28. It appears to me that Linux has worked before and code to support it was added but it is not well tested.

Running Raspberry Pi OS on ARM32, the Python package defaults to the Windows embeddable python-3.7.3-embed-amd64. For Python 3.11.3 downloads, for example, I see that Windows is the only platform with embeddable packages. I see in #1 that Linux packages were discussed, and at some point Linux support was added. I have successfully built from source on the Pi using only SSH and bash, so I imagine it could be done programmatically, but it took a long time.

I wouldn't mind writing a Linux example and documentation myself, but I reviewed Python.Deployment as suggested above, and I can't seem to understand Linux support.

[henon]

Hey, yeah, for non-windows support I am totally relying on community contributions and I always just took people's word that their additions work without testing myself. Furthermore it was a few years back and I can't remember anything about it clearly any more.

You are welcome to PR any improvements. One thing you can do is look at the code with git blame to find out who added what in which PR. That way you can find out who added linux support and invite them to review your PR.

[henon]

I don't know where it is (post deleted??), but I saw in my inbox that @167rgc911 wrote this:

In reading the code it seems *nix support is incomplete, but basics should work.

Have you tried something like.

Python.Deployment.Installer.PythonDirectoryName = Path.GetFullPath("/home/user/uservenv/");
Runtime.Runtime.PythonDLL = "libpython3.10.so";
This works for me (Alpine Linux inside WSL).

Python version:

    3.10.11 (main, Apr  6 2023, 01:16:54) [GCC 12.2.1 20220924]

Current working directory:

    /home/user/uservenv/work/Python.Included/examples/Python.Deployment.EmbeddedResource

PythonPath:

    /usr/lib/python310.zip:/usr/lib/python3.10:/usr/lib/python3.10/lib-dynload

1.0
-0.9589242746631385
-0.6752620891999122
float64
int32
[ 6. 10. 12.]
"/home/user/uservenv" is a Python venv.

don't Python.Deployment.Installer.SetupPython()

don't Python.Deployment.Installer.TryInstallPip() the Downloader will fail saying missing SSL libs. the logic looks for a Scripts\pip.exe. we can fool this by creating a directory Scripts that contains a pip.exe and pip. Both are a symlink to ../bin/pip.

don't Python.Deployment.Installer.PipInstallModule() as there seems to be quoting issue that makes RunCommand fail.

[henon]

Some of the issues (i.e. the qoting issue) seem very easy to solve, if anybody with a linux setup would PR to give something back to the community I'd gladly accept those contributions.

[167rgc911]

i wasn't sure what the OP wanted to do .. embedding might be hard for Linux.

i have some test code for the last issue .. but it is not working. i thought it was a qouting problem but it seems it is a process problem.

i'm a C#/dotnet noob (Linux and Python user though) so it might take some time to submit a PR.

[henon]

We may have to detect the operating system we are on to be able to go about certain things differently on linux. For instance, to detect the mono runtime one can do this:

    public static class PlatformDectection
    {
        private static readonly Lazy<bool> IsRunningOnMonoValue = new Lazy<bool>(() =>
        {
            return Type.GetType("Mono.Runtime") != null;
        });

        public static bool IsRunningOnMono()
        {
            return IsRunningOnMonoValue.Value;
        }
    }

[167rgc911]

Alpine (musl distro) doesn't have mono. MS provides dotnet6 and dotnet7 packages.

when i first checked. it was failing with something:
/bin/bash -c /path/to/pip spacy

quoting the commandline args still gave the same issue.
/bin/bash -c "/path/to/pip spacy"

                if (RuntimeInformation.IsOSPlatform(OSPlatform.Linux))
                {
                    // Unix/Linux/macOS specific command execution
                    filename = "/bin/bash";
                    //args = $"-c {command}";
                    args = $@"-c ""{command}""";
                }

my next guess was that ProcessStartInfo was not correctly starting/finishing/abruptly stopping. i stopped redirecting stderror and got something like this.

<rgc@:Python.Deployment.EmbeddedResource:114>dotnet run --roll-forward Major
> /bin/bash --verbose -c "/home/rgc/i/p/v/Scripts/pip install spacy "
/home/rgc/i/p/v/Scripts/pip install spacy
RunCommand: Error with command: '/home/rgc/i/p/v/Scripts/pip install spacy '
Cannot mix synchronous and asynchronous operation on process stream.
### Python version:
        3.10.11 (main, Apr  6 2023, 01:16:54) [GCC 12.2.1 20220924]
### Current working directory:
        /home/rgc/i/p/Python.Included/examples/Python.Deployment.EmbeddedResource
### PythonPath:
        /usr/lib/python310.zip:/usr/lib/python3.10:/usr/lib/python3.10/lib-dynload
ERROR: Pipe to stdout was broken
Exception ignored in: <_io.TextIOWrapper name='<stdout>' mode='w' encoding='utf-8'>
BrokenPipeError: [Errno 32] Broken pipe
1.0
-0.9589242746631385
-0.6752620891999122
float64
int32
[ 6. 10. 12.]

[henon]

Maybe this helps? https://stackoverflow.com/questions/11460606/i-am-trying-to-read-the-output-of-a-process-in-c-sharp-but-i-get-this-message-c

[167rgc911]

                // The documentation for Process.StandardOutput says to read before you wait otherwise you can deadlock!
                //string output = process.StandardOutput.ReadToEnd();
                //Log(output);

this allowed pip in Linux to finish successfully.

BeginOutputReadLine is async
ReadToEnd is synchronous

[henon]

Awesome, can you PR? I guess it will work on windows also with the async version. I can test