Python 2.7 support on Ubuntu 18.04

[neurolabusc]

This issue is related to this post, e.g. issues with Python 2.7 on Linux but not MacOS.

I downloaded the latest Python-4-Lazarus (with 64 commits) and opened on my Ubuntu 18.04 system with Lazarus 2.1.0 SVN 62040M GTK2 and FPC 3.0.4. I opened the demo_lazarus project and attempted to compile. Once I fix Py_s1 the demo runs fine if I set
cPyLibraryLinux = '/usr/lib/x86_64-linux-gnu/libpython3.6m.so';
however, if I set this to
cPyLibraryLinux = '/usr/lib/x86_64-linux-gnu/libpython2.7.so';
(which also exists on my computer), the program crashes. I added a conditional for Python3 in the PythonEngine.pas:

if IsPython3000 then
    PyUnicode_FromWideChar    :=Import(AnsiString...

The program seems to run (though at launch it reports Error: could not find symbol "PyUnicodeUCS2_AsWideChar"), and I can run a basic Python script, but when I attempt to close the program it crashes.

The crash is as follows...

Error: could not find symbol "PyUnicodeUCS2_AsWideChar"
[FORMS.PP] ExceptionOccurred
Sender=EAccessViolation
Exception=Access violation
Stack trace:
$0000000000000000
$0000000000796514 line 4665 of Sources/Core/PythonEngine.pas
$00000000004A484D
$0000000000552F19 line 5130 of include/control.inc
$0000000000540FB8 line 6645 of include/wincontrol.inc
$00000000005557EF line 40 of include/customcontrol.inc
$0000000000447CBF line 316 of include/scrollingwincontrol.inc
$0000000000449231 line 131 of include/customform.inc
$0000000000430BD0
$0000000000444BF6 line 1937 of forms.pp
$00000000004352B9
Exception at 0000000000000000: EAccessViolation:
Access violation.

[Alexey-T]

Found the reason. Set in PythonEngine component on form, prop APIVersion to value <1011, eg 1010

[Alexey-T]

@pyscripter

function  TPythonInterface.GetUnicodeTypeSuffix : String;
begin
  if (fMajorVersion > 3) or ((fMajorVersion = 3) and (fMinorVersion >= 3)) then
    Result := ''
  else if APIVersion >= 1011 then
    Result := 'UCS2'
  else
    Result := '';
end;

so for APIVersion=1013 we get Py import with "UCS2" and this crashes with py 2.7.

[Alexey-T]

We have another stopper
https://bugs.python.org/issue38398
:(
So my fix don't help.

[pyscripter]

On non-windows systems the suffix needs to be ucs4.

[pyscripter]

Try the following:

function  TPythonInterface.GetUnicodeTypeSuffix : String;
begin
  if (fMajorVersion > 3) or ((fMajorVersion = 3) and (fMinorVersion >= 3)) then
    Result := ''
  {$IFDEF WINDOWS}
  else if APIVersion >= 1011 then
    Result := 'UCS2'
  {$ELSE}
  else if APIVersion >= 1011 then
    Result := 'UCS4'
  {$ENDIF}
  else
    Result := '';
end;

However there is another python bug which seems to be in Ubuntu only:
https://bugs.python.org/issue38398.
This will prevent python 2 being used in Ubuntu. It should work in other versions of Linux though.

[pyscripter]

Can you test whether the above works in other Linux systems or Darwin?

[Alexey-T]

@pyscripter Sorry, i have Lazarus on Ubuntu only... maybe later.

[pyscripter]

Can you try

>>> import ctypes
>>> hasattr(ctypes.pythonapi, "PyUnicode_FromWideChar")
>>> hasattr(ctypes.pythonapi, "PyUnicodeUSC4_FromWideChar")
>>> hasattr(ctypes.pythonapi, "PyUnicodeUSC2_FromWideChar")

on your Ubuntu 2.7?

[pyscripter]

Maybe it should be reported to the Ubuntu bug tracker as suggested by Victor Stinner.

[Alexey-T]

Ubuntu 18.04

user@user-Notebook:~$ python
Python 2.7.15+ (default, Jul  9 2019, 16:51:35) 
[GCC 7.4.0] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import ctypes
>>> hasattr(ctypes.pythonapi, "PyUnicode_FromWideChar")
False
>>> hasattr(ctypes.pythonapi, "PyUnicodeUSC4_FromWideChar")
False
>>> hasattr(ctypes.pythonapi, "PyUnicodeUSC2_FromWideChar")
False

[neurolabusc]

The Python 2.7 that comes with MacOS 10.14 works fine with Python-4-Lazarus

>python
Python 2.7.16 (default, Jun 19 2019, 07:40:37) 
[GCC 4.2.1 Compatible Apple LLVM 10.0.1 (clang-1001.0.46.4)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import ctypes
>>> hasattr(ctypes.pythonapi, "PyUnicode_FromWideChar")
False
>>> import ctypes
>>> hasattr(ctypes.pythonapi, "PyUnicode_FromWideChar")
False
>>> hasattr(ctypes.pythonapi, "PyUnicodeUSC4_FromWideChar")
False
>>> hasattr(ctypes.pythonapi, "PyUnicodeUSC2_FromWideChar")
False

[Alexey-T]

But the last test shows that no funcs are present, it must not work

[pyscripter]

@neurolabusc @Alexey-T
Sorry guys. It should be UCS4 in the above code.

>>> import ctypes
>>> hasattr(ctypes.pythonapi, "PyUnicodeUCS4_FromWideChar")

[neurolabusc]

The Python 2.7 that comes with MacOS 10.14 works fine with Python-4-Lazarus

>python
Python 2.7.16 (default, Jun 19 2019, 07:40:37) 
[GCC 4.2.1 Compatible Apple LLVM 10.0.1 (clang-1001.0.46.4)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import ctypes
>>> hasattr(ctypes.pythonapi, "PyUnicodeUCS4_FromWideChar")
False
>>> hasattr(ctypes.pythonapi, "PyUnicodeUCS2_FromWideChar")
True

[neurolabusc]

Ubuntu 18.04

$ python
Python 2.7.15+ (default, Jul  9 2019, 16:51:35) 
[GCC 7.4.0] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import ctypes
>>> hasattr(ctypes.pythonapi, "PyUnicodeUCS4_FromWideChar")
True
>>> hasattr(ctypes.pythonapi, "PyUnicodeUCS2_FromWideChar")
False
>>> hasattr(ctypes.pythonapi, "PyUnicode_FromWideChar")
False

For Python3

$ python3
Python 3.5.0 (default, Oct  8 2019, 16:28:33) 
[GCC 7.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import ctypes
>>> hasattr(ctypes.pythonapi, "PyUnicodeUCS4_FromWideChar")
False
>>> hasattr(ctypes.pythonapi, "PyUnicodeUCS2_FromWideChar")
False
>>> hasattr(ctypes.pythonapi, "PyUnicode_FromWideChar")
True

[neurolabusc]

@Alexey-T this seems to do the trick on my Ubuntu 18.04. Essentially your code but MacOS (Darwin) should be treated like Windows (UCS2) not like Linux (UCS4)

function  TPythonInterface.GetUnicodeTypeSuffix : String;
begin
  if (fMajorVersion > 3) or ((fMajorVersion = 3) and (fMinorVersion >= 3)) then
    Result := ''
    {$IFDEF Linux}
    else if APIVersion >= 1011 then
      Result := 'UCS4'
    {$ELSE} //Windows or Darwin
    else if APIVersion >= 1011 then
      Result := 'UCS2'
    {$ENDIF}
  else
    Result := '';
end; 

[pyscripter]

@neurolabusc
In my Ubuntu virtual machine I get False, False, False. Strange.

Darwin 2.7 appears to be UCS2 indeed.
conda-forge/python-feedstock#224

Here is my version:

function  TPythonInterface.GetUnicodeTypeSuffix : String;
begin
  if (fMajorVersion > 3) or ((fMajorVersion = 3) and (fMinorVersion >= 3)) then
    Result := ''
  else if APIVersion >= 1011 then
    Result :=
       {$IFDEF WINDOWS} 'UCS2' {$ELSEIF DARWIN} 'UCS2' {$ELSE} 'UCS4' {$ENDIF}
  else
    Result := '';
end;

[Alexey-T]

Ubuntu 18.04

user@user-Notebook:~$ python
Python 2.7.15+ (default, Jul  9 2019, 16:51:35) 
[GCC 7.4.0] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import ctypes
>>> hasattr(ctypes.pythonapi, "PyUnicodeUCS4_FromWideChar")
True
>>> hasattr(ctypes.pythonapi, "PyUnicodeUCS2_FromWideChar")
False
>>> 

[Alexey-T]

Fixed again.

[Alexey-T]

Seems the issue is fixed?