Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

problem of installing Windows #306

Closed
chenqingguo opened this issue Apr 22, 2020 · 110 comments
Closed

problem of installing Windows #306

chenqingguo opened this issue Apr 22, 2020 · 110 comments

Comments

@chenqingguo
Copy link

I install x64 libpq then I build libpqxx using cmake and visual studio 2017.
I got this error MSB4126 Debug |x64 is invalid . I need to build x64 libpqxx.
How to do?
Thanks,chenqingguo.

@tt4g
Copy link
Contributor

tt4g commented Apr 22, 2020

What command did you run?

I can build with the following command on x64 Native Tools Command Prompt for VS 2019:

mkdir build
cd build
cmake .. -DCMAKE_BUILD_TYPE=Debug -G "Visual Studio 16 2019" -A x64 -DPostgreSQL_LIBRARY="<PATH_TO_POSTGRES>/lib/libpq.lib" -DPostgreSQL_INCLUDE_DIR="<PATH_TO_POSTGRES>\include" -DPostgreSQL_TYPE_INCLUDE_DIR="<PATH_TO_POSTGRES>\include"
cmake --build . --config Debug --target -j 8

@tt4g
Copy link
Contributor

tt4g commented Apr 22, 2020

NOTE -A option is important. When this option is not specified, even if you execute CMake on 64bit OS, the generated project file is for 32bit OS.

Visual Studio 15 2017 — CMake 3.17.1 Documentation:

For each toolset that comes with this version of Visual Studio, there are variants that are themselves compiled for 32-bit (x86) and 64-bit (x64) hosts (independent of the architecture they target). By default this generator uses the 32-bit variant even on a 64-bit host. One may explicitly request use of either the 32-bit or 64-bit host tools by adding either host=x86 or host=x64 to the toolset specification. See the CMAKE_GENERATOR_TOOLSET variable for details.

@chenqingguo
Copy link
Author

Well. Here is my command
mkdir build cd build cmake .. cmake—build. —target INSTALL
it works after add -A x64 .
Do I need to copy the include and lib to my program? I thought make install and add find_package in my cmakelists is all what should do.
Thanks

@tt4g
Copy link
Contributor

tt4g commented Apr 22, 2020

Do I need to copy the include and lib to my program?

No, specify lib and include only if CMake cannot find the PostgreSQL library.
I have specified it because my machine has a custom install of libpq.

@GordonLElliott
Copy link

I also had (and fixed) issues with Visual Studio 2019 (version 16). (And thanks for some of the comments on istall, I will integrate those into my process.)

I had bugs appear. I just installed the new version of VS 2019 in last few days, so I suppose the latest. Compiling for 64 bit (-A flag) and 2019, etc., and test in debug mode. Neither the test suite, nor the actual library, would run correctly until I made 2 modifications:

Here is what I did to fix:

IN strconv.cxx, starting line 478:

case 'N':
case 'n':
    // Accept "NaN," "nan," etc.
    ok =    // [GLE] Note text[3] does not exist and triggers failure.
      (text.length() == 3 and (text[1] == 'A' or text[1] == 'a') and
       (text[2] == 'N' or text[2] == 'n') /*and (text[3] == '\0')*/);

IN test_main.hxx, starting line 134. Note that test_main.hxx does not appear in the project directory, is one directory level above runner.cxx in either of the 'runner' or 'unit_runner' sub-projects. It is included in the respective runner.cxx file, and I followed the link. This is a common file, and only needs to be changed once for both sub-projects:

// [GLE] add argc and make test_name null without an an argv[1].
int main(int argc, char const *argv[])
{
  char const *const test_name = argc>1? argv[1]: NULL;

THESE ARE THE ONLY MODIFICATIONS that I needed to successfully run the entire testing suite for pqxx with Visual Studio 2019.

@jtv
Copy link
Owner

jtv commented Apr 22, 2020

For the record, @GordonLElliott filed these problems as #307, and the fixes have been merged into master and the 7.1 working branch (see #274).

@jtv
Copy link
Owner

jtv commented Apr 23, 2020

@chenqingguo is your problem solved now?

@jtv
Copy link
Owner

jtv commented Apr 23, 2020

I have just released 7.0.6, which contains the fixes mentioned here.

@GordonLElliott
Copy link

GordonLElliott commented Apr 23, 2020

<...>

I can build with the following command on x64 Native Tools Command Prompt for VS 2019:

mkdir build
cd build
cmake .. -DCMAKE_BUILD_TYPE=Debug -G "Visual Studio 16 2019" -A x64 -DPostgreSQL_LIBRARY="<PATH_TO_POSTGRES>/lib/libpq.lib" -DPostgreSQL_INCLUDE_DIR="<PATH_TO_POSTGRES>\include" -DPostgreSQL_TYPE_INCLUDE_DIR="<PATH_TO_POSTGRES>\include"
cmake --build . --config Debug --target -j 8

For my purposes, I coplied the cmake commands into a batch file (which also set up the MS variables). This was very helpful--I was completely new to Cmake. (Started with an old version installed and 3 versions of PostgreSQL including an X86, so Cmake found every possible way to mess up that it could find. The above would force correct versions of everything, I think, or at least fail so I knew what to fix. Especially thanks for this and all the othr work--and consider more detail in the notes on build for Microsoft. (Especially that second line using cmake --build. The original notes suggested some sort of 'make' which doesn't even exist on today's Microsoft. My version found a 16 bit copy somehow!!!)

OH--then after that of course cmake supplies a native Visual Studio 2019 solution file. The user can also directly open that and configure for debug and release there. I personally configured for Unicode characer set, which worked fine--I didn't find yet how to integrate that request into the cmake build. User's may not realize that the solution file is built.

I will comment that finding the final library in a subdirectory of 'src' is weird (that is where it goes on my build). I would prefer it went in a 'lib' directory at same level not inside of 'src'.

@tt4g
Copy link
Contributor

tt4g commented Apr 23, 2020

I will comment that finding the final library in a subdirectory of 'src' is weird (that is where it goes on my build). I would prefer it went in a 'lib' directory at same level not inside of 'src'.

@GordonLElliott In the src directory are build artifacts.
Just as Visual Studio outputs default artifacts to bin\Debug, placing build artifacts in the lib directory is considered another task.

For CMake, moving from the src directory to the lib directory is done when you run the install target generated by CMake: https://cmake.org/cmake/help/latest/manual/cmake.1.html#install-a-project

@jtv
Copy link
Owner

jtv commented Apr 24, 2020

Thanks both, I think I have some work to do on the build docs. I'm still not particularly familiar with CMake myself, so I just learned a few things I didn't know!

@joeyave
Copy link

joeyave commented Apr 24, 2020

I have built pqxx for 64 win.
Now when I include it to the project, I have this error:

pqxx.lib(connection.cxx.obj) : error LNK2019: unresolved external symbol __imp_WSAPoll referenced in function "void __cdecl `anonymous namespace'::wait_fd(int,bool,struct timeval *)" (?wait_fd@?A0x909bfe52@@YAXH_NPEAUtimeval@@@z)
project6_part2.exe : fatal error LNK1120: 1 unresolved externals

@jtv
Copy link
Owner

jtv commented Apr 24, 2020

Sounds like you need to link to the socket library... Is it called ws2_32 on Windows? It may take more.

@jtv
Copy link
Owner

jtv commented Apr 24, 2020

In the configure build, I link with ws2_32, wsock32, and/or winsock, as available, in that order. The CMake build links only to those first two — I think the last one may be an older thing. So perhaps your application needs to link with ws2_32 and wsock32 as well.

@joeyave
Copy link

joeyave commented Apr 24, 2020

wsock32

it helped. Thank you. But now I've got abort message. My code:

#include <iostream>
#include <pqxx/pqxx>

int main() {

    pqxx::connection C;

    return 0;
}

@tt4g
Copy link
Contributor

tt4g commented Apr 24, 2020

@joeyave Maybe throw exception from pqxx::connection constructor.
You should add try-catch statement.

@chenqingguo
Copy link
Author

@jtv yes, buildiing succeed, but I think make install in windows is invalid.
I'm learning how to use pqxx, I want to insert 'std::vector<>' into array real[] of pg database.
Here is my test code,

******
pqxx::stream_to inserter(transaction,"test");
for(int i=1; i<99;i++)
{
tmpData tmpdata;
tmpdata.id=to_string(i);
tmpdata.feature=vector<float>(0.1*i,500);
inserter<<std::make_tuple(tmpdata.id,tmpdata.feature);
}
inserter.complete();
transaction.commit();

the result of id in database is right, but the array is incorret. The array is {} in my table--test.
Is there an approprite way to insert vector using pqxx?

@tt4g
Copy link
Contributor

tt4g commented Apr 24, 2020

@chenqingguo

make install in windows is invalid.

Since CMake 3.15, use cmake --install <dir> instead: https://cmake.org/cmake/help/latest/manual/cmake.1.html#install-a-project
Another way is to use cmake --build . --target install: https://stackoverflow.com/questions/34040522/is-there-a-cmake-install-switch/34040747

@jtv
Copy link
Owner

jtv commented Apr 24, 2020

Good point. When you construct a connection, you connect to the database. That could be failing because the database is not running, or because it's on a different port, or because your default user does not exist, or you need to provide a password, or because the database does not exist...

Catch the exception and print its what() message.

@GordonLElliott
Copy link

GordonLElliott commented Apr 24, 2020

OH--H****. Fail is I'm running install without admin privilages--will try that next!!!!!!!!!!!!!!!!!!!!!!!!!

The cmake --install
appears to fail, and also triggering the "INSTALL" in the VS 2019 IDE also fails. Just references the post build batch as whole, and error 1, no detailed explanation.

FOR ALL SHOWING a method to do install -- please tell if you have actually tested this method successfully! (As opposed to just finding cmake instruction; which is tremendously valuable of course because it is the only way we will figure it out.)

I saw install location variables somewhere in the original materials. I syspect that something is wrong in all that setup so that the GUI based install does not work out.

By the way, the "PACKAGE" utility in the GUI successfully creates a gzip file--I have no idea yet what it does.

HEY! The "PACKAGE" builds a gzip of a the install needed. But this step executes in my VS gui, while the "INSTALL" step fails. I can extract the gzip that it creates as a binary install....... I will try that and report. At least this shows that the essence of "INSTALL" is already working, just some specific failure in the project as automatically constructed.

This gzip contains the pqxx library, and clean copies of the include directory!

@GordonLElliott
Copy link

GordonLElliott commented Apr 24, 2020

NOTE: See messages below--successfully installed (need to be administrator), but it put in the Program files (X86) directory. Otherwise very nice.

IN CASE these notes were useful....

I am still trying to do an "install" (don't know how yet) on X64 build, so the copies needed for regular usage are in the weird locations (as mentioned above).

But the build package also includes examples in the form of the tests (runner, unit_runner) which demonstrate linking for X64 in the weird locations of the build. Can use that until get an install to a sensible location for all.

Specifically for person with linking in x64, here are my libraries (obviously change to correct locations on your system):

In properties--linker--additional dependencies:

C:\pqxx_7\build64\src\Debug\pqxx.lib;C:\Program Files\PostgreSQL\12\lib\libpq.lib;wsock32.lib; .....

where ..... is the original on that line. Obviously I built at top level C drive in my case so I could use for the time being, you can figure out what I did. (Someone comment on missing socket library???) This compiles. I have not tested yet my dummy application which does not have connect string so I will need to set up environment variables, which can be done in the VS debugger by the way for testing.

THERE SHOULD be better practice than this method, for example set library path separately and only library names in the list of libraries. (That way only fix the paths when change compiler setup, but it is the sema library wherever it lands.) This example is just to get the list which then does work.

(Also here is what I set to refer to the not-yet-installed library include files, once again copying from the test cases):
IN properties--C/C++--general--Additional Include Directories (here the full line)

C:\Program Files\PostgreSQL\12\include;C:\Program Files\PostgreSQL\12\include\server;C:\pqxx_7\build64\include;C:\pqxx_7\libpqxx-master\include;%(AdditionalIncludeDirectories)

@GordonLElliott
Copy link

And I need to configure my build for Unicode, not multi-byte character set. (Did I already mention?)

I have successfully done that manually. First cmake to build the VS solution. Then open with VS and change the pqxx, runner, and unit_runner properties to use Unicode. This works successfully and the test suite works completely. However it is a manual intervention and as yet not "installed" (since I don't know yet how to do that and the install job in the VS solution fails). All references I have found on the Internet suggest this must be burried in the cmake setup files, and if done then possibly give a define option to trigger automatically. For now even if I had an install step that worked, I would have to stop for the manual intervention to set Unicode character set, so I could not automate as a continuous build--install process with a single batch file either.

@GordonLElliott
Copy link

GordonLElliott commented Apr 24, 2020

Everyone ignore my statement that INSTALL project does not work.

It works (sort of) if you run with administrative permissions. Either in a batch file (I presume) with admin privilages, or in separate step. I will describe how to run INSTALL from VS2019 IDE (presumably works others).

The trick is that even if you are logged in as admin, the VS2019 IDE does not run as administrator. That makes sense, you don't want compiling events (which may have been written by others) to trigger operating system changes--that could introduce viruses etc when unscrupulous persons are involved.

So after build steps have worked (and in my case use the VS IDE to change to Unicode character set and rebuild, or any other changes you require). THEN exit the VS IDE (if open).

Then go to the start menu and find your VS IDE, and right click and follow more options to "run as administrator".

DO NOT do any other tasks as admin -- I recommend -- just the install.

Right click on the INSTALL project, and build.

The problem:

Even though I built as X64, it installed the library and include files in the C:\Program Files (x86)\libpqxx directory for Win32 programs.

OTHER than that--this is a very nice install, clean of almost everything supurflous.

The methods using a command line (say in batch file) to trigger this may have options to override. Or one could take the post build event, make a batch file, and edit to include additional directive to select the correct location--I have not figured that out yet.

ALSO note that the PACKAGE project builds a gzip of the same files. Then you could manually or by batch file (as administrator) expand the gzip into the correct location, either program files (and not the (x86) directory) or if you chose a different install location. Once again someone may figur out how to set this into the INSTALL project--probably a -D or something.

I will be testing usage of this and report....

@tt4g
Copy link
Contributor

tt4g commented Apr 24, 2020

@GordonLElliott You can change install location with --prefix option. e.g: cmake --install <dir>--prefix <install-prefix>
See cmake --install document: https://cmake.org/cmake/help/latest/manual/cmake.1.html#install-a-project

Another way: cmake -DCMAKE_INSTALL_PREFIX=<instal-prefix>

https://cmake.org/cmake/help/latest/variable/CMAKE_INSTALL_PREFIX.html#variable:CMAKE_INSTALL_PREFIX

CMake is a cross-platform tool. It is the user's responsibility to perform the installation with administrator privileges, when writing during installation requires administrator privileges.
Due to security issues, CMake functionality cannot ignore permissions.
This is also true for Linux and MacOS.

@GordonLElliott
Copy link

GordonLElliott commented Apr 24, 2020

Reporting results latest version (7.0.6)

I just downloaded the latest, applied my (slightly updated) batch file, and used the INSTALL project (as described above using administrator priviledges). Then tested the library with a modified Employee database example program.

Another way: cmake -DCMAKE_INSTALL_PREFIX=<instal-prefix>

I added to my original build batch file the suggested install prefix (thanks tt4g) which worked perfectly. I used that method rather than a cmake --install method since I wanted the built up project (VS 2019 solution) to have the target within itself, so that could be run from the IDE. This worked perfectly, and the INSTALL project installed in the desired directory.

Furthermore I tested with both null and empty string ('') data in database for conversion to both int and float types, and found that the throw reports were completely consistent and no debugging traps occured. So all the new changes work.

Here is my batch file to build (but not install) the project from just raw data:

rem GLE build pqxx7...
rem
rem   ** First delete all subdirectories (namely build and any old libpqxx-master)
rem      Copy new clean copy of libpqxx-master directory in to this directory. Cmake line
rem      must properly refer to this location (Files should be just under libpqxx-master)  **
rem
call "C:\Program Files (x86)\Microsoft Visual Studio\2019\Professional\VC\Auxiliary\Build\vcvarsall.bat" x64
md build64
cd build64
cmake ../libpqxx-master -DCMAKE_BUILD_TYPE=Debug -G "Visual Studio 16 2019" -A x64 -DPostgreSQL_LIBRARY="C:\Program Files\PostgreSQL\12/lib/libpq.lib" -DPostgreSQL_INCLUDE_DIR="C:\Program Files\PostgreSQL\12\include" -DPostgreSQL_TYPE_INCLUDE_DIR="C:\Program Files\PostgreSQL\12\include" -DCMAKE_INSTALL_PREFIX="C:\pqxx_7.0.6_new"
if %errorlevel% neq 0 goto :error
cmake --build . --config Debug --target -j 8
if %errorlevel% neq 0 goto :error

goto :OK
:error
echo Failed with error #%errorlevel%.
pause
exit /b %errorlevel%
:OK
pause

I'm sure there are better ways to do this, but it demonstrates all the features. Of course change all directories to be correct for your machine. I just click on the batch file in a directory with only the batch file and latest libpqxx-master directory, and it builds the Solution for Visual Studio 2019. Then building the INSTALL project (with IDE opened as administrator) installs correctly.

Thanks again to everyone who contributed ideas, too numerous to name. I started with a cmake that referred to the wrong version of PostgreSQL, built the wrong kind of program using the wrong version of Visual Studio and put it in the wrong place. (And crashed in the debugger without running the verification testing.) Now (with extra thanks to jtv) I have a clean install that passes all test!

@jtv
Copy link
Owner

jtv commented Apr 25, 2020

Thanks for figuring this all out and documenting it!

@GordonLElliott
Copy link

GordonLElliott commented Apr 25, 2020

I simplified the batch file, removing some flags that were not in the documentation or help, or otherwise the system complained about and didn't work.

rem Build pqxx from CMake files. (Place libpqxx-master directory with files in parallel with batch file)
rem Output will be placed in directory build64. Run Visual Studio as Administrator to build INSTALL as Release.
call "C:\Program Files (x86)\Microsoft Visual Studio\2019\Professional\VC\Auxiliary\Build\vcvarsall.bat" x64
if %errorlevel% neq 0 goto :error
cmake -S libpqxx-master -B build64 -G "Visual Studio 16 2019" -A x64 -DPostgreSQL_LIBRARY="C:\Program Files\PostgreSQL\12/lib/libpq.lib" -DPostgreSQL_INCLUDE_DIR="C:\Program Files\PostgreSQL\12\include" -DPostgreSQL_TYPE_INCLUDE_DIR="C:\Program Files\PostgreSQL\12\include" -DCMAKE_INSTALL_PREFIX="C:\Program Files\libpqxx"
if %errorlevel% neq 0 goto :error
rem change Release to Debug to pre-build a debug copy. IDE opens as Debug no matter what.
cmake --build build64 --config Release
if %errorlevel% neq 0 goto :error
pause
goto :EOF
:error
echo Failed with error #%errorlevel%.
pause
exit /b %errorlevel%

This will build a Release -- or change the one occurrence of "Release" to "Debug" will build a debug copy. Note that the libpqxx.sln solution will be set for "Debug" no matter what (we have not found a switch that changed the setting on the solution, the build config only afects which was pre-compiled in the batch file not the solution itself.

Then as before use the IDE run as Administrator to actually install, or possibly set up a batch file to trigger the INSTALL project in the solution. The INSTALL project will be configured to install in the C:\Program Files\libpqxx directory.

New Install Problem:

Now the problem: The install for Debug and Release put the pqxx.lib file in the same location. Yes, I could make completely different projects for Debug and for Release, with complete duplicate copies of the include files and documentation.

Furthermore the install does not copy the pqxx.pdb file along with the pqxx.lib in the Debug install. So if you were to recompile the Debug pqxx project without installing, then you can't link to the debug .lib file any more because it actually referes back to the .pdb file in your compile directory. (I tested this and it failed--but if I install the .pdb with the .lib then it does not fail just by rebuilding the pqxx project.)

By the way, the source files become an important part of the debug installation, where they were compiled. If you step into a user program compiled against the debug pqxx.lib you can step into the source debugging with the files in their location where they were compiled.

See: https://docs.microsoft.com/en-us/visualstudio/debugger/specify-symbol-dot-pdb-and-source-files-in-the-visual-studio-debugger?view=vs-2019

https://www.wintellect.com/pdb-files-what-every-developer-must-know/

Method I would recommend is either maintain separate locations for release and debug install, or my manual method: I copy the generated pqxx.lib and pqxx.pdb into a lib\Debug directory alongside of the release lib\pqxx.lib file. (Could make a batch file for that install approach I suppose--still a missing aspect of the install methodology.) The install method has already done the heavy lifting of separating out the include, document, and possibly release library, so this is a minor step.

@tt4g
Copy link
Contributor

tt4g commented Apr 26, 2020

@GordonLElliott I think only some developers install debug-built libraries.
They can freely change the installation location with CMAKE_INSTALL_PREFIX.

@GordonLElliott
Copy link

GordonLElliott commented Apr 26, 2020

@GordonLElliott I think only some developers install debug-built libraries.
They can freely change the installation location with CMAKE_INSTALL_PREFIX.

As I mentioned, that results in duplicate include and document directories as well, and does not resolve the lack of the pqxx.pdb in the install directory.

The IDE build produces a nice directory (though weird location) build64\src\Debug . I suppose I could make a batch file to copy the Debug directory to the install lib diretory, after building both Debug and Release configurations and installing the Relase first to set up the entire structure and heavly lifting.

@tt4g
Copy link
Contributor

tt4g commented Apr 26, 2020

The IDE build produces a nice directory (though weird location) build64\src\Debug . I suppose I could make a batch file to copy the Debug directory to the install lib diretory, after building both Debug and Release configurations and installing the Relase first to set up the entire structure and heavly lifting.

The build directories are separated because you are using Visual Studio as a generator with CMake.
MinGW and gcc should not create Debug or Release directories.
As a result, batch files will not work cross-platform.

CMake scripts are currently maintained by contributors.
After adding the batch file, someone must maintain it.
Furthermore, CMake also has RelWithDebInfo and MinSizeRel as build type.
Don't forget these things for full support.

I can't decide whether or not to support it, as all support adds work to the project.

@tt4g
Copy link
Contributor

tt4g commented May 5, 2020

I think that the -S -B form of the construction step is important

By the way, this build method is called CMake out-of-source build.
Building in the source directory is called CMake in-source build.

@GordonLElliott
Copy link

GordonLElliott commented May 5, 2020

Possible documentation change for Windows build:
[Edit 5/8-more compatible with CMake documentation: use "binary" for intermediate]
[Eidt 5/9-refer to top-level directory with source, binary, and install subdirectories--more consistent with common terminology]

On Microsoft Windows

There used to be custom Visual C++ project files and Makefiles in libpqxx, but these are no longer supported.

Instead, use the CMake build.

One problem specific to Windows is that apparently it doesn't let you free memory in a DLL that was allocated in the main program or in another DLL, or vice versa. This can cause trouble when setting your own notice handlers to process error or warning output. Recommended practice is to build libpqxx as a static library, not a DLL.

CMake is available for several compilers for Windows, including Visual Studio.
[Insert relevant comments other compilers]
Only Visual Studio 2017 and later support the C++ 17 standard required by libpqxx 7.

Cmake Build for Visual Studio

An easy way to build a static libpqxx library on Windows for Visual Studio is an "out-of-source build" using CMake, in which source and build files are subdirectories of a top-level directory. (Check that your version of CMake is up to date.) In this method you put the input source files subdirectory (libpqxx-master) into the top-level directory. Then in this top-level directory execute the following two commands:

cmake -S libpqxx-master -B binary -DCMAKE_INSTALL_PREFIX="libpqxx"

This constructs a Visual Studio solution with multiple projects and CMake build information, in the 'binary' subdirectory, but does not build those projects. The CMAKE_INSTALL_PREFIX parameter establishes the final install directory in advance as a third subdirectory 'libpqxx', of the build directory. This cannot be located in the protected Program Files directory without running CMake as administrator. Leaving off the CMAKE_INSTALL_PREFIX parameter usually causes CMake to default to C:\Program Files (x86)\libpqxx in the next step. This is often the wrong install subdirectory, requires administrative privileges or will fail, and those privileges are not granted just by logging in as admin.

Then

cmake --build binary --target INSTALL

will build the projects, including copying the final installation files into the libpqxx subdirectory of your build directory, as previously established. That libpqxx subdirectory can be manually copied into the C:\Program Files directory if desired, to give it a standardized location that may be expected on Windows. A manual copy wil ask for administrative rights.

A version of PostgreSQL must be installed on the build computer, at least the libraries must be installed. See the notes on additional flags below if the first cmake command cannot locate the correct PostgreSQL library.

One way to automate this build process is to copy the following lines and paste into a good programming editor, which will fix any Windows new line problems. Then save as build.bat batch script file, into your top-level directory. Click on the batch file to automate the build process:

REM Automated build of libpqxx placing install files in libpqxx subdirectory.
cmake -S libpqxx-master -B binary -DCMAKE_INSTALL_PREFIX="libpqxx"
if %errorlevel%==0  cmake --build binary --target INSTALL
pause

Any changes can be made by clicking the batch build file again and only necessary changes will be compiled, as is normal for a "make" process.

Flags that can be added on the first cmake line include:
-A x64 or -A Win32 These will select the target architecture as 64 or 32 bit.
-G "Visual Studio 15 2017" may be used to select the 2017 version of Visual Studio, or explicitly select -G "Visual Studio 16 2019" (default if installed). The architecture and generator parameter values must be exactly as shown. Use
-DPostgreSQL_ROOT="C:\Program Files\PostgreSQL\12" to select the version of PostgreSQL at the specified location. To compile for Win32, the "C:\Program Files (x86)\PostgreSQL\10" version of libpq will probably be required from the x86 directory (and include -A Win32).

The second (build) cmake command can also have:
--config Release to switch the install from a Debug to a Release configuration.

A slightly more complex sequence of commands will build (completely separate) Debug and Release installations:

REM Automated build of libpqxx, both Debug and Release
cmake -S libpqxx-master -B binary
if %errorlevel%==0  cmake --build binary --config Debug
if %errorlevel%==0  cmake --build binary --config Release
if %errorlevel%==0  cmake --install binary --config Debug --prefix "libpqxx/Debug"
if %errorlevel%==0  cmake --install binary --config Release --prefix "libpqxx/Release"
pause

@GordonLElliott
Copy link

GordonLElliott commented May 14, 2020

I've discovered another improvement to make programming easier on Windows with Visual Studio. This is called a "property sheet". Rather than having to make entries for the libraries and directories needed, the "property sheet" has those pre-loaded one time.

The file here should be saved as libpqxx.props, in the base directory where the lib, include, etc subdirectories are placed for the installation:

<?xml version="1.0" encoding="utf-8"?>
<!--
    This is a property sheet to be included in MSVS projects of the applications
    using libpqxx. Use "View|Property Manager" and choose "Add Existing
    Property Sheet..." from the context menu to add it from the IDE.
  -->
<Project ToolsVersion="4.0" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
  <ImportGroup Label="PropertySheets" />
  <PropertyGroup Label="UserMacros" />
  <PropertyGroup />
  <ItemDefinitionGroup>
    <ClCompile>
      <AdditionalIncludeDirectories>$(MSBuildThisFileDirectory)include;%(AdditionalIncludeDirectories)</AdditionalIncludeDirectories>
    </ClCompile>
    <Link>
      <AdditionalLibraryDirectories>$(MSBuildThisFileDirectory)lib;%(AdditionalLibraryDirectories)</AdditionalLibraryDirectories>
      <AdditionalDependencies>pqxx.lib;C:\Program Files\PostgreSQL\12\lib\libpq.lib;wsock32.lib;ws2_32.lib;%(AdditionalDependencies)</AdditionalDependencies>
    </Link>
  </ItemDefinitionGroup>
  <ItemGroup />
</Project>

Now notice that the cross-reference to the PostgreSQL library is hard coded, in my case here to version 12. To auto-generate this file, I am sure that a CMake procedure can be developed, I will look into that in the future. For now one can hand edit.

As to the location of the libpqxx files themselves, the loading of the property sheet from a specific location allows the "MSBuildThisFileDirectory" macro to be filled, so this file does not need to be adjusted for any of the other locations as long as the relative subdirectory structure is maintained.

To use, the Visual Studio user select the "Property Manager" to view the project. The various configurations show up including combinations of X64 and X86 and Debug and Release. Use "Add Existing Property Sheet..." by right clicking over the respective configuration, and select the file as saved above.

Now still have to declare compiler standard ISO C++ 17!

Also note that one has to match the library build configuration (Debug, Release) to the project configuration. (I have that batch file that keeps separate libraries, working on integrating that....) For now also could keep separate complete Debug and Release install directories (see above post), and then place the property sheet in each root directory for respective configuration--and insert the respective property sheet from the respective configuration in VS.

This is much easier than having to set up all those settings each time a new project is started.

@jtv
Copy link
Owner

jtv commented May 14, 2020

Great! You've done so much work on this, I think it's definitely worth a repository of its own, where you can work freely. I could link to it from the README.

@GordonLElliott
Copy link

GordonLElliott commented May 14, 2020

Great! You've done so much work on this, I think it's definitely worth a repository of its own, where you can work freely. I could link to it from the README.

Great, Thanks! I have started a junk repository with (incomplete) versions just for getting used to all that. And working on a new version of the batch file. And the property sheet can also do a "Post Build" step that copies a list of DLL's, (which need to go with the property sheet location) so that it becomes simple to build a new app. (Batch to compile pqxx, property sheet for new app, and gather proper list of DLL's from PostgreSQL. MIGHT even copy the PostgreSQL library there, and that fixes the problem of knowing the location within the property sheet. Anyway you see several things to do before ready for prime time. The new version of the pqxx compile batch file uses CMake for virtually every step rather than "robocopy" so it will be more consistent with all operating systems and not dependent upon updates except for CMake itself.)

Thanks again

@GordonLElliott
Copy link

GordonLElliott commented May 29, 2020

Progress (or un-progress) on batch file to build and "install" libpqxx

I have a batch file which does several things very successfully -- and then a most spectacular failure!

The successes:

  1. Configure, Build, and organize "installation" files for libpqxx, with both Debug and Release copy.
  2. Allow configuration of various parameters by commented variables, including Multibyte vs Unicode character set, and selecting compilers and platform within the Visual Studio offerings.
  3. Easy to edit to specify the final "installation" location.
  4. Gather DLLs from PostgreSQL that are needed by the libpq Postgres library. Selected by Postgres version (though have not yet researched the list for each version--in progress--among 10,11,12--tested and working with version 12).
  5. Property Page (as it is called) to make creating a new project in Visual Studio that uses libpqxx super easy--just insert property page from the install location at it will automatically connect all include, library files as well as (optionally) copy the DLL files required by Postgres.
  6. I know how to automatically trigger request to become administrator to install library, property pages, and documents in "Program Files" directory, to be comparable to the PostgreSQL installation--but see below.

Now the spectacular failure:

I was using a copy tool from Windows called 'robocopy' to copy the files. There was an option to make target directory look just like the source directory--including deleting items not intended.

I used that method, but made a simple mistake, and it was triggered as administrator. Apparently I tried to make my entire "Program Files" directory look like the install of the library--empasis on entire. Having some trouble getting the admin level part working, I finally got it to start--and noticed the files being listed didn't look like mine. Turns out it had started making my entire program files directory look like the library (only) and started with deleting the A's. Luckily they were all Adobe files on a subscription and I restored them with relative ease.

But this points out the difficulty of having an "installation" that occurs in the privileged file directories. (The same sort of mistake could wipe out program source, other areas, without security issues--a tool is a tool and needs to be used correctly and used incorrectly can be dangerous--a fact of life.)

So not quite ready, my next approach will be a script that can be run at admin level, but display the script for the user before starting so user can examine its simplicity, then simplify the script that is run to use only something called "xcopy" on Windows, with target of a simply copy of the install to a location in the Program Files directory. Removal of files will become a responsibility of user by using file explorer to delete the entries--not a complete installation method with uninstall.

@GordonLElliott
Copy link

GordonLElliott commented May 30, 2020

Easy PQXX Build for Windows Visual Studio

My long awaited batch build system is now online:

https://github.com/GordonLElliott/Easy-PQXX-Build-for-Windows-Visual-Studio

See the repository for documentation. Major features:

  1. Automated batch file to build and install in the selected location
    (typically C:\Program Files\libpqxx), asking for administrator
    privileges if needed to copy the files.
  2. Configure for Unicode or MultiByte character sets. (For example Unicode
    character set may be needed to interface wxWidgets in the same
    application.)
  3. Keep named installations for different configurations.
  4. One library installation with both Debug and Release compiles that
    automatically switch in Visual Studio.
  5. Property Sheets so your Visual Studio application (with both Debug and
    Release compiles) can be instantly configured.
  6. Gather DLLs from PostgreSQL library into your application automatically.

Additionally various configuration solving selections can be made, for
example if you have more than one PostgreSQL library or want to compile for
more than one version of Visual Studio.

I repaired the possible danger issue by switching to XCopy which is more bug free and has fewer options to cause problems. It basically just copies the directory to the target in the Program FIles subdirectory for libpqxx.

JTV, don't reference it yet, but if you know some users who would be willing to try it out in Windows first, I would appreciate that. Needs work on docu,mentation before ready for prime time.

@GordonLElliott
Copy link

GordonLElliott commented Jun 3, 2020

@jtv said:

That's massive! I'm guessing this could help a lot of people, but maintaining it is going to be difficult as circumstances change. Maybe you could put this up in your own repo? Something like "easy libpqxx builds in Visual Studio" perhaps.

and

Great! You've done so much work on this, I think it's definitely worth a repository of its own, where you can work freely. I could link to it from the README.

Jtv, reading the license for libpqxx, it appears I might need permission from you to name and promote my batch file for installing PQXX .

Thanks
Gordon

@GordonLElliott
Copy link

GordonLElliott commented Jun 3, 2020

Jtv, I went over the documentation in BUILDING-cmake.md, specifically trying on Windows. There really are several problems, much better to use CMake for each of the 3 steps. In BUILDING-cmake.md:

If you just wawnt to get it built and installed quickly, run cmake from the root of the libpqxx source tree. This configures your build.

The rest depends on your system. To compile:

On Windows with Visual Studio, try: msbuild libpqxx.sln
On Unix-like systems, including Linux and macOS, try: make
Your system may use some other command.

Then

How to compile depends on your CMake "generator":

With Unix Makefiles, run make
With Ninja, run ninja
With Visual Studio, run msbuild libpqxx.sln
...or whatever works in your situation.

Missing elements include:

  1. In order to run the msbuild, you have to have set up the environment, like using a command window started from the visual studio menus.
  2. The build works (after running it correctly...), but seriously one needs the "install" step to organize the files--otherwise they are scattered within the source and build locations. As to the build itself, the CMake command cmake --build build is much easier than figuring out all the MSBuild stuff (build is current build directory where the CMake was pointed).
  3. At least on Windows, the "install" step really should be done by calling CMake again, say with
cmake --install build --config Debug   --prefix "libpqxx/Debug"

Sure, you can invoke the MSBuild on the INSTALL project. But you had to configure it correctly in the very first CMake command. Then all it does is call CMake in an internal batch file.

Now my Easy-PQXX Build for Windows Visual Studio README gives meticulously tested alternatives and explanations. (Even if one does not use the batch files just the command lines as explained.)

jtv added a commit that referenced this issue Jun 5, 2020
@jtv
Copy link
Owner

jtv commented Jun 5, 2020

@GordonLElliott only just catching up. Ouch — you had your own little Gitlab moment there! Hope your system has recovered. I just added a link to your project in my own README, saying that you need testers and feedback.

Next, I'll want to update my own documentation for the CMake build to use cmake commands instead of make/ninja/msbuild.

@GordonLElliott
Copy link

GordonLElliott commented Jun 5, 2020

@GordonLElliott only just catching up. Ouch — you had your own little Gitlab moment there! Hope your system has recovered. ......

Yes, took a day at least. First investigate what happened. Then all night to just recover/reinstall files.

But then had to decide what to do. I didn't do what I initially said I was going to do, which was in essence to try to put some protective code into the mechanism used to call into administrator privileges. That method was to use a very cryptic language--only on Microsoft Windows--called "Visual Basic Script". But that was so cryptic that no one would be able to read it and see what was happening. So I dicided to keep that at minimum, and now what is done in that "VBS" is just recursively call the batch file itself with an argument, but running as administrator.

So what I do (as administrator in the batch file) is make sure that there is only one action path if it has any command line argument whatsoever. Then use only simple bug-free commands that only copy files, when on that path, and you can read the batch file and verify that. As I get time I may put some effort into clarifying that aspect.

OH--also thanks for the reference, hope we get someone to do some testing. I have one user but we run such nearly identical environment that it is hardly a test.

AND if you have any interest for the documentation, feel free to copy anything I said or in the project without reference to me.

@GordonLElliott
Copy link

GordonLElliott commented Jun 5, 2020

A minor issue: In BUILDING-cmake.md you have the line

-DBUILD_DOC=on to include documentation in the build.

That flag (at least on Windows CMake build) is going to use Doxygen to rebuild the documentation itself.

Either on or off, it is going to include the documentation into the installed files. (There may be another flag to remove from the installed files--like @abrownsword was wishing.)

@jtv
Copy link
Owner

jtv commented Jun 6, 2020

Sounds like solid thinking on the scripting. I hope it takes off.

I think it's time to close this ticket. The original issue was solved, and what we've got is like a discussion forum. And we completely drowned out @chenqingguo who asked a follow-up question which we totally missed. I'm sorry about that, @chenqingguo!

@jtv jtv closed this as completed Jun 6, 2020
jtv added a commit that referenced this issue Jun 13, 2020
@fahmidshibib
Copy link

Hey, can someone tell me where you got the wsock32.lib file?

@tt4g
Copy link
Contributor

tt4g commented Oct 29, 2020

Hey, can someone tell me where you got the wsock32.lib file?

@fahmidshibib It's installed on Windows machine.

@fahmidshibib
Copy link

Hey, can someone tell me where you got the wsock32.lib file?

@fahmidshibib It's installed on Windows machine.

Hi, I am trying to locate it and trying to add it to my project so that I can remove these errors (even though I am not sure if that lib file is the issue in the first place): (Click on picture for better resolution)
image

I trying to run this piece of code in order to test my connection to postgresql
image

Please help me resolve this. If this is due to wsock32.lib how do I add it? I looked up the internet and found out that I needed to install WindowsSDK. I did that and I found WSock32.Lib in the installation location "C:\Program Files (x86)\Windows Kits\10\Lib\10.0.19041.0\um\x64"

I added this in Properties>Linker>General>Additional Library Directories

Still ending up with these errors.

Thanks for your help!

@tt4g
Copy link
Contributor

tt4g commented Oct 29, 2020

It looks like the linker does not link pqxx.
Check the build options.

@fahmidshibib
Copy link

It looks like the linker does not link pqxx.
Check the build options.

Hi, can you guide me on how to do that please? I have never built and installed something in this way before. What I did till now is:
Ran the following commands in VS2015 x64 Native Tools Command Prompt

C:\project_files\ProjectServer\ACTOOLS-1854\code\AscendServices\libpqxx-6.4.6\build>cmake .. -DCMAKE_BUILD_TYPE=Debug -G "Visual Studio 14 2015" -A x64 -DPostgreSQL_LIBRARY="C:/Program Files/PostgreSQL/13/lib/libpq.lib" -DPostgreSQL_INCLUDE_DIR="C:\Program Files\PostgreSQL\13\include" -DPostgreSQL_TYPE_INCLUDE_DIR="C:\Program Files\PostgreSQL\13\include"

C:\project_files\ProjectServer\ACTOOLS-1854\code\AscendServices\libpqxx-6.4.6\build>cmake --build . --config Debug --target -j 8

C:\project_files\ProjectServer\ACTOOLS-1854\code\AscendServices\libpqxx-6.4.6>cmake --install build --config Debug --prefix "libpqxx/Debug"

And ended up with a file called libpqxx with the following contents:
image

In the project properties VC++ Directories I added the include and lib folders.

Background:
I am trying to connect to postgresql from an application that runs on VS2015, so I had to make some adjustments in the cmake commands to reflect that and used the libpqxx-6.4.6 (I dont know if I needed to do these but I still did just to be safe)

Thanks!

@tt4g
Copy link
Contributor

tt4g commented Oct 29, 2020

Has pqxx.lib in the lib directory been added to the link target?

@fahmidshibib
Copy link

Has pqxx.lib in the lib directory been added to the link target?

No. I just added it to Linkers>Additional Library Directories and Linkers>Inputs>Additional Dependencies. I guess it made some improvements. Now I am getting this
image

I this located in the Postgres installation location? Because I have not included anything from the Postgres Installation folder yet, only the libpqxx stuff.

Thanks!

@tt4g
Copy link
Contributor

tt4g commented Oct 29, 2020

Shared libraries are loaded at application execution time.
If the library is not found, an error is reported because the symbols are not found.

You can fix the problem by adding PostgreSQL bin directory and pqxx bin directory to the PATH environment variable.

See the official documentation for details on where Windows looks for libraries.

@fahmidshibib
Copy link

Shared libraries are loaded at application execution time.
If the library is not found, an error is reported because the symbols are not found.

You can fix the problem by adding PostgreSQL bin directory and pqxx bin directory to the PATH environment variable.

See the official documentation for details on where Windows looks for libraries.

Hi, I tried adding the bin folders from PostgreSQL and libpqxx but I still get that error. This is how I added the path
image

Do I have restart my laptop or something to make it work after updating the PATH environment variables?

Thanks!

@tt4g
Copy link
Contributor

tt4g commented Oct 29, 2020

If you are running an application from VSCode, you may need to restart VSCode to have it reload the updated environment variables.

@fahmidshibib
Copy link

If you are running an application from VSCode, you may need to restart VSCode to have it reload the updated environment variables.

Hey thanks for your help, restarting VS helped. However, I was trying to run a simple insertion statement through pqxx and ran into this exception.
image

Can you please help me understand what this exception is due to?
My test code looks like this:

string connectionstring("host=localhost port=5432 dbname=server_logs user=postgres password=password");
	pqxx::connection conpq(connectionstring);
	std::cout << "Connected to " << conpq.dbname() << std::endl;
	pqxx::work W{ conpq };

	string sql = "INSERT INTO EVEN_LOG VALUES (CURRENT_TIMESTAMP, 'test','test','test','test','test','AscendServices Initiated...');";

	W.exec(sql);
	cout << "PostGres Insertion Successful" << endl;
	conpq.disconnect();

Note: I don't think the sql command is wrong as I ran it directly on Postgres and it passed.

Thanks!

@tt4g
Copy link
Contributor

tt4g commented Nov 1, 2020

The error code represents an "undefined table name".

libpqxx/src/result.cxx

Lines 235 to 236 in e63674f

if (equal(code, "42P01"))
throw undefined_table{Err, Query, code};

Check to see if the table is in the server_logs database with the following command:

$ psql -U postgres -h localhost -p 5432 server_logs
\d EVEN_LOG

@jtv
Copy link
Owner

jtv commented Nov 1, 2020

It can also help to surround your code with try / catch and in the case of an exception, print its error string (as returned by its what() method). That'll usually tell you something more.

In this case, your query references EVEN_LOG... Which I assume should have been EVENT_LOG.

@GordonLElliott
Copy link

Ahhhh... Always listen to @jtv !

@fahmidshibib said it worked directly in Postgres, which I assume meant it worked in PSQL or similar tool.

But an easy transcription error or accidental character deletion after copy and paste could change working command referencing table EVENT_LOG to the string given, if indeed the actual table was named EVENT_LOG. That would explain why an "EVEN_LOG" table (no "T") was missing in the failed example, even though without transcription error it worked in PSQL! The error messages given are pointing the way, "undefined table at...".

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

7 participants