"pip install pydnp3" on windows10

[pzg250]

when I use pip install pydnp3 to install pydnp3 package on windows10, there is an error that shows errors as below. Any suggestion? Thx

---

pip install pydnp3
Collecting pydnp3
Using cached https://files.pythonhosted.org/packages/3d/7b/9da4115503bdce211323caaa12ad7a5128470366a62cdc9c3877549d0be5/pydnp3-0.1.0.tar.gz
Requirement already satisfied: pybind11>=2.2 in c:\program files\python37\lib\site-packages (from pydnp3) (2.2.3)
Installing collected packages: pydnp3
Running setup.py install for pydnp3 ... error
Complete output from command "c:\program files\python37\python.exe" -u -c "import setuptools, tokenize;file='C:\Users\ADMINI1\AppData\Local\Temp\pip-install-phu88ytc\pydnp3\setup.py';f=getattr(tokenize, 'open', open)(file);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, file, 'exec'))" install --record C:\Users\ADMINI1\AppData\Local\Temp\pip-record-w5bs56zh\install-record.txt --single-version-externally-managed --compile:
c:\program files\python37\lib\distutils\dist.py:274: UserWarning: Unknown distribution option: 'long_description_content_type'
warnings.warn(msg)
running install
running build
running build_ext
-- Building for: NMake Makefiles
CMake Error at CMakeLists.txt:3 (project):
Generator

    NMake Makefiles

  does not support platform specification, but platform

    x64

  was specified.


CMake Error: CMAKE_C_COMPILER not set, after EnableLanguage
CMake Error: CMAKE_CXX_COMPILER not set, after EnableLanguage
-- Configuring incomplete, errors occurred!
See also "C:/Users/Administrator/AppData/Local/Temp/pip-install-phu88ytc/pydnp3/build/temp.win-amd64-3.7/Release/CMakeFiles/CMakeOutput.log".
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "C:\Users\ADMINI~1\AppData\Local\Temp\pip-install-phu88ytc\pydnp3\setup.py", line 89, in <module>
    zip_safe=False,
  File "c:\program files\python37\lib\site-packages\setuptools\__init__.py", line 129, in setup
    return distutils.core.setup(**attrs)
  File "c:\program files\python37\lib\distutils\core.py", line 148, in setup
    dist.run_commands()
  File "c:\program files\python37\lib\distutils\dist.py", line 966, in run_commands
    self.run_command(cmd)
  File "c:\program files\python37\lib\distutils\dist.py", line 985, in run_command
    cmd_obj.run()
  File "c:\program files\python37\lib\site-packages\setuptools\command\install.py", line 61, in run
    return orig.install.run(self)
  File "c:\program files\python37\lib\distutils\command\install.py", line 545, in run
    self.run_command('build')
  File "c:\program files\python37\lib\distutils\cmd.py", line 313, in run_command
    self.distribution.run_command(command)
  File "c:\program files\python37\lib\distutils\dist.py", line 985, in run_command
    cmd_obj.run()
  File "c:\program files\python37\lib\distutils\command\build.py", line 135, in run
    self.run_command(cmd_name)
  File "c:\program files\python37\lib\distutils\cmd.py", line 313, in run_command
    self.distribution.run_command(command)
  File "c:\program files\python37\lib\distutils\dist.py", line 985, in run_command
    cmd_obj.run()
  File "C:\Users\ADMINI~1\AppData\Local\Temp\pip-install-phu88ytc\pydnp3\setup.py", line 34, in run
    self.build_extension(ext)
  File "C:\Users\ADMINI~1\AppData\Local\Temp\pip-install-phu88ytc\pydnp3\setup.py", line 59, in build_extension
    subprocess.check_call(['cmake', ext.sourcedir] + cmake_args, cwd=self.build_temp, env=env)
  File "c:\program files\python37\lib\subprocess.py", line 328, in check_call
    raise CalledProcessError(retcode, cmd)

---

[anhnguyen-cpi]

The "pip install pydnp3" is currently built for ubuntu 16.04 and MacOS. Can you try to follow the readme and build it from source to see if you have the same issue?

I noticed that Garret made some change to enable building on Windows and with alternative versions of Python, you can check it out to see if it fixes the issue: https://github.com/smartergridsolutions/pydnp3

[pzg250]

@anhnguyen-cpi Thank u very much for your comments. It has the same error when I try to build the dnp3 from source. I will try Garrent's version.

[ujjavaljoshi]

Hi, Thanks for the new release for Windows. I have encountered below issue of heap size while compiling on Windows.

 6>cl : Command line warning D9002: ignoring unknown option '-fvisibility=h
   idden' [C:\pydnp3\build\temp.win-amd64-3.5\Release\pydnp3.vcxproj]
     pydnp3.cpp
     Please define _WIN32_WINNT or _WIN32_WINDOWS appropriately. For exampl
     e:
     - add -D_WIN32_WINNT=0x0501 to the compiler command line; or
     - add _WIN32_WINNT=0x0501 to your project's Preprocessor Definitions.
     Assuming _WIN32_WINNT=0x0501 (i.e. Windows XP target).
 6>c:\pydnp3\deps\dnp3\cpp\libs\include\openpal\util\comparisons.h(55): war
   ning C4146: unary minus operator applied to unsigned type, result still
   unsigned [C:\pydnp3\build\temp.win-amd64-3.5\Release\pydnp3.vcxproj]
     c:\pydnp3\src\openpal\util\comparisons.h(74): note: see reference to f
     unction template instantiation 'bool openpal::FloatEqual<T>(T,T,T)' be
     ing compiled
             with
             [
                 T=uint32_t
             ]
     c:\pydnp3\src\openpal\util\comparisons.h(80): note: see reference to f
     unction template instantiation 'void declareComparisions<uint32_t>(pyb
     ind11::module &)' being compiled
     c:\pydnp3\deps\pybind11\include\pybind11\cast.h(1501): note: see refer
     ence to class template instantiation 'pybind11::detail::descr<8>' bein
     g compiled
     c:\pydnp3\deps\pybind11\include\pybind11\cast.h(1499): note: see refer
     ence to class template instantiation 'pybind11::detail::descr<5>' bein
     g compiled
     c:\pydnp3\deps\pybind11\include\pybind11\cast.h(1054): note: see refer
     ence to class template instantiation 'pybind11::detail::descr<7>' bein
     g compiled
 6>c:\pydnp3\deps\pybind11\include\pybind11\cast.h(1874): fatal error C1060
   : compiler is out of heap space [C:\pydnp3\build\temp.win-amd64-3.5\Rele
   ase\pydnp3.vcxproj]
 6>Done Building Project "C:\pydnp3\build\temp.win-amd64-3.5\Release\pydnp3
   .vcxproj" (default targets) -- FAILED.
 1>Done Building Project "C:\pydnp3\build\temp.win-amd64-3.5\Release\ALL_BU
   ILD.vcxproj" (default targets) -- FAILED.

Build FAILED.

[bbarcklay]

The currently build seems to require at least 8Gb free memory. We encounter a similar problem on Linux when running with less than 8Gb. How much memory is available on your build machine?

[ujjavaljoshi]

@bbarcklay-cpi . The current machine has less than 6Gb memory. I will try with different machine and let you know the result.

Thanks for the feedback.

[garretfick]

I have moved my changes for building on Windows to a different repo. The work breaks things into multiple compilation units so you need ~3 GB memory instead of much more

https://github.com/garretfick/pydnp3/

You want the branch feature/COMPILE_WINDOWS

[garretfick]

I have also created a pull request so hopefully this can make it into the main repo
#13

[Kudrat9]

Hi, I'm trying to build pydnp3 on Windows and I used this repo:

https://github.com/garretfick/pydnp3/

However, I encounter several errors (same as building the previous repo). I have attached the output I see after running 'python setup.py install'.
log.txt

[anhnguyen-cpi]

@Kudrat9 Have you tried to build the pydnp3 from https://github.com/garretfick/pydnp3/tree/feature/COMPILE_WINDOWS using ~3GB memory as @garretfick mentioned?

[Kudrat9]

Yes, I did. But it generated the same errors. Also, I'm running it on my host machine. So, memory is not an issue. I'm running on Windows 10 and I tried Python 2.7 and 3.6.

[garretfick]

I'll take a look but might take a day as I'm travelling.

[garretfick]

I was able to build the project myself, without problems.

My steps were to open the project with Visual Studio 2019 and use it's cmake capabilities to build the project. That was successful (albeit with a number of warnings).

[Kudrat9]

I'm trying to build the project in VS 2017 now. I'm getting the error "Cannot open include file: asio.hpp: No such file or directory" even though I have defined ASIO_HOME as an environment variable.

[garretfick]

The asio.hpp error happens if you have not cloned the submodule in the dnp3 dependency.

…

On Thu, Jun 20, 2019 at 12:28 Kudrat9 ***@***.***> wrote: I'm trying to build the project in VS 2017 now. I'm getting the error "Cannot open include file: asio.hpp: No such file or directory" even though I have defined ASIO_HOME as an environment variable. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#3?email_source=notifications&email_token=AB442KYH5FUE4U4BYDFUY73P3OV2LA5CNFSM4FPRC2SKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODYF5ZCQ#issuecomment-504093834>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AB442K4TAY6LEDYVZCX7CULP3OV2LANCNFSM4FPRC2SA> .

[Kudrat9]

I was able to fix the asio error. But then I got the same errors as before. So, I installed VS 2019. When the cmake runs, it works fine.

But when I do the build, it gives me errors.

Most of them seem like syntax errors. Can you tell me which compiler did you use? I tried clang and msvc_x64. But I get the same result. Also, how much RAM is on your system? I just want to make sure it's not a memory issue since I faced that while installing pydnp3 on my linux VM.

[garretfick]

This is my cmake configure output:

1> CMake generation started.
1> Command line: C:\PROGRAM FILES (X86)\MICROSOFT VISUAL STUDIO\2019\ENTERPRISE\COMMON7\IDE\COMMONEXTENSIONS\MICROSOFT\CMAKE\CMake\bin\cmake.exe  -G "Ninja" -DCMAKE_INSTALL_PREFIX:PATH="C:\Users\gfick\CMakeBuilds\6740193d-fe50-1334-9dc0-121c5c224dce\install\x64-Debug (default)" -DCMAKE_CXX_COMPILER:FILEPATH="C:/Program Files (x86)/Microsoft Visual Studio/2019/Enterprise/VC/Tools/MSVC/14.20.27508/bin/HostX64/x64/cl.exe" -DCMAKE_C_COMPILER:FILEPATH="C:/Program Files (x86)/Microsoft Visual Studio/2019/Enterprise/VC/Tools/MSVC/14.20.27508/bin/HostX64/x64/cl.exe"  -DCMAKE_BUILD_TYPE="Debug" -DCMAKE_MAKE_PROGRAM="C:\PROGRAM FILES (X86)\MICROSOFT VISUAL STUDIO\2019\ENTERPRISE\COMMON7\IDE\COMMONEXTENSIONS\MICROSOFT\CMAKE\Ninja\ninja.exe" "C:\Users\gfick\Dev\pydnp3"
1> Working directory: C:\Users\gfick\CMakeBuilds\6740193d-fe50-1334-9dc0-121c5c224dce\build\x64-Debug (default)
1> [CMake] -- The C compiler identification is MSVC 19.20.27508.1
1> [CMake] -- The CXX compiler identification is MSVC 19.20.27508.1
1> [CMake] -- Check for working C compiler: C:/Program Files (x86)/Microsoft Visual Studio/2019/Enterprise/VC/Tools/MSVC/14.20.27508/bin/HostX64/x64/cl.exe
1> [CMake] -- Check for working C compiler: C:/Program Files (x86)/Microsoft Visual Studio/2019/Enterprise/VC/Tools/MSVC/14.20.27508/bin/HostX64/x64/cl.exe -- works
1> [CMake] -- Detecting C compiler ABI info
1> [CMake] -- Detecting C compiler ABI info - done
1> [CMake] -- Detecting C compile features
1> [CMake] -- Detecting C compile features - done
1> [CMake] -- Check for working CXX compiler: C:/Program Files (x86)/Microsoft Visual Studio/2019/Enterprise/VC/Tools/MSVC/14.20.27508/bin/HostX64/x64/cl.exe
1> [CMake] -- Check for working CXX compiler: C:/Program Files (x86)/Microsoft Visual Studio/2019/Enterprise/VC/Tools/MSVC/14.20.27508/bin/HostX64/x64/cl.exe -- works
1> [CMake] -- Detecting CXX compiler ABI info
1> [CMake] -- Detecting CXX compiler ABI info - done
1> [CMake] -- Detecting CXX compile features
1> [CMake] -- Detecting CXX compile features - done
1> [CMake] CMake build is: Debug
1> [CMake] -- Found PythonInterp: C:/Users/gfick/AppData/Local/Programs/Python/Python37/python.exe (found version "3.7.3") 
1> [CMake] -- Found PythonLibs: C:/Users/gfick/AppData/Local/Programs/Python/Python37/libs/Python37.lib
1> [CMake] -- pybind11 v2.3.dev0
1> [CMake] CMake build is: Debug
1> [CMake] ASIO has been checked out as a git submodule: C:/Users/gfick/Dev/pydnp3/deps/dnp3/deps/asio/asio/include
1> [CMake] -- Performing Test HAS_MSVC_GL_LTCG
1> [CMake] -- Performing Test HAS_MSVC_GL_LTCG - Failed
1> [CMake] -- LTO disabled (not supported by the compiler and/or linker)
1> [CMake] -- Configuring done
1> [CMake] -- Generating done
1> [CMake] -- Build files have been written to: C:/Users/gfick/CMakeBuilds/6740193d-fe50-1334-9dc0-121c5c224dce/build/x64-Debug (default)
1> [CMake] 
1> Starting CMake target info extraction ...
1> CMake server connection made.
1> Extracted includes paths.
1> Extracted CMake variables.
1> Extracted source files and headers.
1> Extracted global settings.
1> Extracted code model.
1> Extracted CTest info.
1> Collating data ...
1> Tests found, but none that match the current build type of Debug.
1> Target info extraction done.
1> CMake generation finished.

Look like your compiler version is newer and I think you are using msvc_x64.

I do see that in my configure, ASIO is checked out as a submodule, whereas yours is not, and the errors seem to be related to ASIO. Can you give that a try and see what happens?

I'm not sure if pydnp3 on any platform allows ASIO to not be checked out as a submodule. Perhaps @anhnguyen-cpi can shed light on that case.

[Kudrat9]

I looked into it but found this:

https://groups.google.com/forum/#!topic/automatak-dnp3/hZxt_6hirhY

It's just a warning, and the ASIO_HOME tells where asio is located. So, I don't suppose that is an issue.

[garretfick]

I'm not convinced either way with regard to ASIO.

Are you able to show the first error from Visual Studio or the complete compile output. I see the screen shot has 11 errors, but there are only 10 in the screen shot you provided.

It is also hard to tell which file has the problem without the full output. pydnp3 has very similar file names to opendnp3, so you usually need to see the full path.

[garretfick]

On a separate thread - how did you get ASIO? What version are you using?

[Kudrat9]

The 11th error was a memory issue. But later, I compiled on a bigger system and it was gone. And these 10 errors remained.

I downloaded ASIO from here (and added to the deps folder in dnp3):
https://think-async.com/Asio/Download.html

[garretfick]

Gut feeling I have is that you are using a different version of ASIO.

The version I have for ASIO has the following in serial_port.hpp:

namespace asio {

/// Typedef for the typical usage of a serial port.
typedef basic_serial_port<> serial_port;

} // namespace asio

Downloading the latest from the site you linked, I see the following:

namespace asio {

#if defined(ASIO_ENABLE_OLD_SERVICES)
// Typedef for the typical usage of a serial port.
typedef basic_serial_port<> serial_port;
#else // defined(ASIO_ENABLE_OLD_SERVICES)
...

So, if ASIO_ENABLE_OLD_SERVICES is not defined, then I think you would get error C2039 from your screen shot.

[Kudrat9]

Oh ok, I'll try using the other version (1.10.8) and see if that works.

[garretfick]

Finally found the version number I have - I think it is 1.10.8. Try downloading that version of ASIO then try again.

I do see that the opendnp3 docs say use 1.12.2. I think pydnp3 isn't on the latest version of opendnp3.

[garretfick]

You also asked about my system - my system has 16 GB, but part of that is normally reserved for Hyper-V, so usually at most 14 GB.

[garretfick]

@Kudrat9 did that solve the problem for you?

[Kudrat9]

I got rid of the ASIO warning after replacing the library with the older version. But I get this error when I build pydnp3.

I ran the build a couple times. Whenever I clean the previous build, I see the number of files being cleaned is different. Which is giving me the feeling its still having some memory issue (despite the fact that I ran it on 16 GB system). Any ideas?

[Kudrat9]

This is the error output, which shows same errors as before.

[garretfick]

The bad news

This looks like an issue with opendnp3 (not pydnp3). It has been fixed in opendp3, but I've been unable to find a way automatically solve it here.

The good news
You can "easily" fix this by adding #include <string> to deps\dnp3\cpp\libs\include\asiodnp3\IMasterOperations.h

I need to investigate a bit more about why this isn't happening for Linux and whether I can just update the opendnp3 version, but I hope that should un-block you for now.

[garretfick]

@Kudrat9 I also pushed an update that eliminates the warning about visibility=hidden

[garretfick]

I have pushed another update that somewhat solves this automatically.

I have fixed setup.py so that it works. It detects if IMasterOperations.h needs a patch, and if so, applies the patch. You can run python setup.py install and the build should succeed.

Note that if you run via cmake directly (or Visual Studio), then the patch is not automatically applied and you need to do it manually as described above.

Hopefully this works :)

[Kudrat9]

@garretfick Thanks! This worked. I was able to build it both using VS 2019 and python setup.py install.

I will install the new update to get rid of the warnings.
Thanks for the help! :)

[garretfick]

Great. Hopefully @anhnguyen-cpi 's testing passes and then this can be merged and then fix the underlying issue by updating to a newer version of OpenDNP3.

[benhuynh-eaton]

hi I am currently trying to install the pydnp3 library. My setup configurations are Python 3.8, Visual Studio 2019 and Cmake 3.2.7. I am currently getting this error

\AppData\Local\Programs\Python\Python38-32\lib\subprocess.py", line 364, in check_call
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['cmake', 'C:\Users\E0757436\Desktop\dnp3\pydnp3', '-DCMAKE_LIBRARY_OUTPUT_DIRECTORY=C:\Users\E0757436\Desktop\dnp3\pydnp3\build\lib.win32-cpython-38', '-DPYTHON_EXECUTABLE=C:\Users\E0757436\AppData\Local\Programs\Python\Python38-32\python.exe', '-DCMAKE_LIBRARY_OUTPUT_DIRECTORY_RELEASE=C:\Users\E0757436\Desktop\dnp3\pydnp3\build\lib.win32-cpython-38']' returned non-zero exit status 1.

I was wondering if anyone had a fix for this?

[benhuynh-eaton]

Should say that I have tried upgrading setup tool, and I am running Windows 10