Can't use with External Dlls

[jesseahlquist]

I am trying to use this package to interface with 32 bit dll's for a Texas Instruments API, but the server fails to start with the message:

"
raise PyInstallerImportError(name) from base_error
pyimod03_ctypes.install..PyInstallerImportError: Failed to load dynlib/dll 'C:\Users\ahlquist3\Anaconda3\envs\MainEnv\Lib\site-packages\msl\examples\loadlib\PortabilityLayer.dll'. Most likely this dynlib/dll was not found when the application was frozen.

Cannot start the 32-bit server.
"

From numerous stack exchange, this appears to be due to issues with ctypes.cdll() and the suggestions say to refreze the executable with a spec file pointing to the corresponding .dlls needed.

However, when running freeze_server32.main(spec='path-to-spec') it fails with the following info:

2563 INFO: PyInstaller: 4.8
2563 INFO: Python: 3.9.12 (conda)
2585 INFO: Platform: Windows-10-10.0.19045-SP0
2585 INFO: UPX is not available.
2585 INFO: Removing temporary files and cleaning cache in C:\Users\ahlquist3\AppData\Local\pyinstaller
script 'C:\Users\ahlquist3\Anaconda3\Lib\site-packages\msl\loadlib\server32-windows.py' not found
1
b''
Traceback (most recent call last):

File ~\Anaconda3\Lib\site-packages\msl\loadlib\NewServer.py:10 in
freeze_server32.main(spec='C:\Users\ahlquist3\Anaconda3\Lib\site-packages\msl\loadlib\server32-alt.spec')

File ~\Anaconda3\lib\site-packages\msl\loadlib\freeze_server32.py:191 in main
check_call(cmd)

File ~\Anaconda3\lib\subprocess.py:373 in check_call
raise CalledProcessError(retcode, cmd)

CalledProcessError: Command '['C:\Users\ahlquist3\Anaconda3\python.exe', '-m', 'PyInstaller', '--distpath', 'C:\Users\ahlquist3\Anaconda3\Lib\site-packages\msl\loadlib', '--workpath', 'C:\Users\AHLQUI~1\AppData\Local\Temp\tmpciuv22kn', '--noconfirm', '--clean', 'C:\Users\ahlquist3\Anaconda3\Lib\site-packages\msl\loadlib\server32-alt.spec']' returned non-zero exit status 1.

I am eager to find a way around all of these issues - any help is immensely appreciated!

Relevant files for the DLPR200API Installer.exe can be downloaded at: https://dlinnovations.com/resources/product-downloads/ - This includes the PortabilityLayer.dll and other .dll, .lib and .h files

[jborbely]

Your issue is that not all dependencies of PortabilityLayer.dll are available on your PATH environment variable.

The PortabilityLayer.dll library depends on

1. the USBlib.dll library (which is saved in the same default installation directory C:\Program Files (x86)\Texas Instruments-DLP\DLP LightCommander API SW\Lib)
2. Microsoft Visual C++ 2008 Redistributable. Download it from here. Make sure you install vcredist_x86.exe.

Here's a script that I used to successfully load the library. Specifying the append_environ_path keyword argument in the Client64 class is important.

from msl.loadlib import Client64, Server32


class DLPR200API(Server32):

    def __init__(self, host, port):
        super().__init__(path='PortabilityLayer.dll', libtype='cdll', host=host, port=port)

    def get_compile_mode(self):
        return self.lib.DLP_FlashCompile_GetCompileMode()


class DLPR200(Client64):

    def __init__(self):
        super().__init__(
            module32=__file__,
            append_environ_path=r'C:\Program Files (x86)\Texas Instruments-DLP\DLP LightCommander API SW\Lib'
        )

    def get_compile_mode(self):
        return self.request32('get_compile_mode')


if __name__ == '__main__':
    dlpr = DLPR200()
    print(dlpr.get_compile_mode())

I'm not sure if the PortabilityLayer.dll library also requires .NET Framework 3.5. After performing step 2) above, run the example script. If it doesn't work follow these instructions to enable .NET Framework 3.5.

[jesseahlquist]

Thank you for these tips! I'm no longer running into the frozen server issue, but I'm still unsure if it is connect to the library correctly.

Running the script above results in a return value of large positive and negative numbers (for example: -1766304000), but the documentation indicates this return value should be either a "0" or "1".

To test further, I added other API methods to the Client64 and Server32 classes.

I've tried including ctypes to accept the return variables, but haven't seemed to have success with this.

• Calling dlpr.get_version_string() results in the error traceback below (start of trace in bold).
• Moreover, when trying to pass a ctypes.POINTER(ctypes.POINTER(ctypes.c_char)) with dlpr.DLP_Misc_GetMCUversionString it results in a pickle error

`from` msl.loadlib import Client64, Server32


class DLPR200API(Server32):

    def __init__(self, host, port):
        super().__init__(path='PortabilityLayer.dll', libtype='cdll', host=host, port=port)

    def get_version_string(self):
        return self.lib.DLP_Misc_GetVersionString()
    
    def get_compile_mode(self):
        return self.lib.DLP_FlashCompile_GetCompileMode()
    
    def get_num_usb(self):
        return self.lib.DLP_Misc_GetTotalNumberOfUSBDevicesConnected()
    
    def get_mcu_version(self,vOutP):
        return self.lib.DLP_Status_GetMCUversionString(vOutP)
    
    def init_api(self):
        return self.lib.DLP_Misc_InitAPI()
    
    def run_batch_command(self,cmdString):
        return self.lib.RunBatchCommand(cmdString)

class DLPR200(Client64):

    def __init__(self):
        super().__init__(
            module32=__file__,
            append_environ_path=r'C:\Program Files (x86)\Texas Instruments-DLP\DLP LightCommander API SW\Lib'
        )
    
    def get_version_string(self):
        return self.request32('get_version_string')
    
    def get_compile_mode(self):
        return self.request32('get_compile_mode')
    
    def get_num_usb(self):
        return self.request32('get_num_usb')
    
    def get_mcu_version(self,vOutP):
        return self.request32('get_mcu_version',vOutP)
    
    def init_api(self):
        return self.request32('init_api')
    
    def run_batch_command(self,cmdString):
       return self.request32('run_batch_command',cmdString)
    

if __name__ == '__main__':
      
    dlpr = DLPR200()
    print(dlpr.get_compile_mode())
    print(dlpr.get_version_string())`

Traceback (most recent call last):
File "C:\Users\ahlquist3\OneDrive - LLNL_xPuSL\General\Programming\BioPrinter, TOP DOWN, 10-3-22, BDM Folder\C\Users\scada-95\Documents\BioPuSL Labview\External Libraries\PythonIPC\TestUsage2.py", line 65, in
print(dlpr.get_version_string())
^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\ahlquist3\OneDrive - LLNL_xPuSL\General\Programming\BioPrinter, TOP DOWN, 10-3-22, BDM Folder\C\Users\scada-95\Documents\BioPuSL Labview\External Libraries\PythonIPC\TestUsage2.py", line 43, in get_version_string
return self.request32('get_version_string')
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\ahlquist3\Anaconda3\envs\MainEnv\Lib\site-packages\msl\loadlib\client64.py", line 211, in request32
return self._client.request32(name, *args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\ahlquist3\Anaconda3\envs\MainEnv\Lib\site-packages\msl\loadlib\client64.py", line 408, in request32
response = self._conn.getresponse()
^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\ahlquist3\Anaconda3\envs\MainEnv\Lib\http\client.py", line 1428, in getresponse
response.begin()
File "C:\Users\ahlquist3\Anaconda3\envs\MainEnv\Lib\http\client.py", line 331, in begin
version, status, reason = self._read_status()
^^^^^^^^^^^^^^^^^^^
File "C:\Users\ahlquist3\Anaconda3\envs\MainEnv\Lib\http\client.py", line 292, in _read_status
line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\ahlquist3\Anaconda3\envs\MainEnv\Lib\socket.py", line 707, in readinto
return self._sock.recv_into(b)
^^^^^^^^^^^^^^^^^^^^^^^
ConnectionResetError: [WinError 10054] An existing connection was forcibly closed by the remote host
Exception ignored in: <function Client64.del at 0x000001A5A37B4720>
Traceback (most recent call last):
File "C:\Users\ahlquist3\Anaconda3\envs\MainEnv\Lib\site-packages\msl\loadlib\client64.py", line 155, in del
File "C:\Users\ahlquist3\Anaconda3\envs\MainEnv\Lib\site-packages\msl\loadlib\client64.py", line 385, in cleanup
File "C:\Users\ahlquist3\Anaconda3\envs\MainEnv\Lib\site-packages\msl\loadlib\client64.py", line 432, in shutdown_server32
File "C:\Users\ahlquist3\Anaconda3\envs\MainEnv\Lib\http\client.py", line 1336, in request
File "C:\Users\ahlquist3\Anaconda3\envs\MainEnv\Lib\http\client.py", line 1382, in _send_request
File "C:\Users\ahlquist3\Anaconda3\envs\MainEnv\Lib\http\client.py", line 1331, in endheaders
File "C:\Users\ahlquist3\Anaconda3\envs\MainEnv\Lib\http\client.py", line 1091, in _send_output
File "C:\Users\ahlquist3\Anaconda3\envs\MainEnv\Lib\http\client.py", line 1035, in send
File "C:\Users\ahlquist3\Anaconda3\envs\MainEnv\Lib\http\client.py", line 1001, in connect
File "C:\Users\ahlquist3\Anaconda3\envs\MainEnv\Lib\socket.py", line 852, in create_connection
File "C:\Users\ahlquist3\Anaconda3\envs\MainEnv\Lib\socket.py", line 837, in create_connection
ConnectionRefusedError: [WinError 10061] No connection could be made because the target machine actively refused it

C:\Users\ahlquist3\OneDrive - LLNL_xPuSL\General\Programming\BioPrinter, TOP DOWN, 10-3-22, BDM Folder\C\Users\scada-95\Documents\BioPuSL Labview\External Libraries\PythonIPC>

[jborbely]

Running the script above results in a return value of large positive and negative numbers (for example: -1766304000), but the documentation indicates this return value should be either a "0" or "1".

Unfortunately, there is nothing I can do about that. The fact that you can call a function in the library means msl-loadlib did its job correctly. Whether the company provided a DLL that does what the documentation says it does is not within my ability to help debug. Contact the manufacturer. My example script was a starting point to show you how to fix your original error.

I've tried including ctypes to accept the return variables, but haven't seemed to have success with this.

You must define the argtypes (see example) and restype (see example) for the functions in the library. By default, ctypes assumes all arguments are integers and the return type is an integer. However, the header file defines Byte as typedef Byte (__cdecl *ProgressCallback)(Double complete);

Calling dlpr.get_version_string() results in the error traceback below (start of trace in bold).

I suspect that since you didn't define argtypes and restype for the DLP_Misc_GetVersionString function that the server is crashing because the library is accessing memory registers that it shouldn't access.

The header file defines PORTABILITYLAYER_API Status LC(DLP_Misc_GetVersionString)(StringBuilder ver, Int32 cbSize); so you would do something like

def get_version_string(self):
    self.lib.DLP_Misc_GetVersionString.argtypes = [ctypes.c_char_p, ctypes.c_int32]
    self.lib.DLP_Misc_GetVersionString.restype = ctypes.c_int  # fine for an enum

    version = ctypes.create_string_buffer(32)  # hopefully 32 characters is enough, if not increase value
    self.lib.DLP_Misc_GetVersionString(version, len(version))
    return version.value  # must return the value attribute, not the ctypes string-buffer object

Moreover, when trying to pass a ctypes.POINTER(ctypes.POINTER(ctypes.c_char)) with dlpr.DLP_Misc_GetMCUversionString it results in a pickle error

You cannot pass pointers between a 32-bit process (the server) and the 64-bit process (your client) because 32-bit and 64-bit programs cannot share memory space (that's the reason for writing msl-loadlib). See the above get_version_string example that shows how to pass a c_char_p and return the value of the string (not the ctypes object).