Fix to allow building pljava with Microsoft Visual C

[kenolson]

Code changes to allow compilation and linking with Microsoft
Visual C. Maven build process conditionalized to to detect Visual C
and adjust options appropriately. See msvc-build-notes.txt for
full details.

[thallgren]

This PR has been overlooked. Sorry about that. It needs a rebase before it can go in.

[kenolson]

This pull request has been rebased and is ready to be merged, if you are comfortable with it. It builds fine with postgresql versions through 9.3. Beginning with 9.4, the postgres.exe Windows executable no longer exports all symbols and therefore the fields log_min_messages, client_min_messages and session_timezone that are referenced by pl/java are undefined. I've created a stub file that supplies defaults for log_min_messages and client_min_messages and sets session_timezone to NULL - this works for my application but is not a general solution. I'll create a separate pull request for this stub file since it is independent of the Visual C build changes.

[thallgren]

Looks good. I had just a minor remark regarding the naming of non mvcs specific properties.

[kenolson]

pom.xml updated to remove the msvc prefix except for the one msvc setting for rint definitions.

[jcflack]

Hello Ken (I hope you'll still get mail on this closed PR),

I have just been working on the build process some more, and I discovered I have questions about your patch that didn't come to me until now:

1. I am all ignorance about MS VC, and I was curious what the MSVC_RINT define is needed for (I don't think you mentioned it in msvc-build-notes.txt). Can you briefly explain? It seems you make it HAVE_RINT=1 in some cases and msvc.rint in others, and msvc.rint also ends up on the command line in non-MSVC builds. I assume there's no reason it has to be there at all in the non-MSVC case, right? At the moment it is the cause of a bunch of

   [WARNING] <command-line>:0:5: warning: missing whitespace after the macro name [enabled by default]
   

   warnings in the linux/gcc build, which just makes it harder to scan through the warnings for any real issues.

2. Why, in the compiler-msvc profile, does jvm.lib.dir have to be derived from an environment variable instead of from a Java system property as in the default case? Does the Java on Windows not expose a property for that?

3. (related) Your build notes are emphatic about needing to have a JAVA6_HOME and a JAVA_HOME in the environment and the same java be found on the PATH. Did you make any progress finding why that's so complicated? I don't have a Windows environment to try here, but when I look at the cmd file to start Maven, it seems to rely only on JAVA_HOME to find what java.exe to start, and the rest of everything should be happening all within that JVM with all the properties set accordingly, shouldn't it? What seemed to be going wrong?

4. Was there a naming strategy you had in mind for some of the new properties to have prefix pgsql. and others postgres., and pkglibdir no prefix at all (and the ones prefixed postgres. seem to have to do with the JVM library locations)? Otherwise, would you mind if I changed some of that around?

5. If I do reorganize some of that, will you be able to check it in MSVC again to be sure I haven't broken it? Are there times that would be better or worse for that?

Thank you,
-Chap

[kenolson]

Creating the MS VC build was my first experience with Maven and the learning curve was “interesting”. Also this work was done some time back and only merged recently after a number of other PRs that added support for building with Java 7 (PR #35) which I didn’t subsequently take into account. So, with that background let’s see about your questions (numbering does not match).

1. I think the use of “msvc.rint” represents remnants of some Maven syntax experimentation. What is really needed on the command line is either “HAVE_RINT=1” or nothing. Newer versions of MSVC have the “rint” function and this is conveyed to Postgresql via the HAVE_RINT definition. Earlier versions of Postgresql did not have this definition in pg_config.h because they expected earlier versions of MSVC to be used. Looking at build.xml now, I think we can replace “msvc.rint” with “”. And in pom.xml eliminate the property “msvc.rint” and in the compiler defines block use ${$MSVC_RINT}.
2. The use of JAVA6_HOME was to differentiate between the Java used to compile/build versus the Java used to run Maven prior to PR Add support for Jdk7 #35. It is now unnecessary and the same jvm.lib.dir value may be used for all compilers. (My system’s default Java was Java 7 at the time I started this effort.)
3. The postgres.lib.name and postgres.lib.path properties for MSVC are pointing to postgres.lib – the file defining the exported symbols from the postgresql server executable. MSVC needs this reference in the linker options. I did not know a way to readily conditionalize this portion of the maven pom, and so pointed these to an “innocuous” location for the gcc link (which happened to be the jvm, since it was already referenced in the link and would not introduce an additional dependency). If you have a better way to conditionalize this, that would be great.
4. The pgsql properties point to postgresql compile time include directories. Perhaps the differentiation between compile and runtime is overkill.
5. The pkglibdir property mirrors the symbol that is required to compile Backend.c. Which upon reviewing that code now makes me question what the “correct” value for that property is. Should it be $PGSQL_PKGLIBDIR?

I have no vested interest in any of the naming conventions and am happy for you to reorganize any of this to make it clearer or more consistent. I’d be happy to pull down your branch and test any changes. I can usually turn this around with a few days.

Ken

[jcflack]

Ok, that's encouraging. I think I can streamline some of that. Completely agree about the Maven learning curve. I might have something later in the week.

-Chap

[jcflack]

Hi Ken,

There's a branch in my repo where I think I have a way to do this that relies more on the profile and less on injecting properties into things. I am not feeling cocky enough to make it a pull request yet, but I will be eager to hear how close (or not close) it comes to building in your environment.

A possible issue I can foresee is that it is adding the MS-specific include paths late in the list. If it's necessary for them to be earlier, I did find a way to do that too, it's just (even) more verbose.

It will also mention libjvm twice (with a right path, I hope, and a wrong one), but as long as that won't hurt anything, it's probably not worth the further added verbosity to do anything about it.

I am hoping that msvc-build-notes.txt can be simplified to just say "set JAVA_HOME to the JVM you want and run mvn" but I haven't touched it for this commit.

Do you happen to know what makes the ecpg, pgtypes, and pq libraries have to be mentioned here? I did a test (linux) build without them and it loaded into my backend and ran a few examples with no trouble, which is more what I expected because it gets loaded into a running backend where all that stuff is already resolved. But maybe it's platform dependent? Does anything go wrong in your environment if you delete those three lib refs from the POM?

-Chap

[kenolson]

I was able to test compilation and linking with your branch - there is one change needed in the pom.xml file. The directory reference for the jvm.lib file needed to use JAVA_HOME rather than java.home. With this change the link worked fine (got undefined references to JNI methods with the original).
${JAVA_HOME}/lib

I like the streamlined build - much cleaner!

I haven't had a chance to look at the extra libraries you mention - no experience with them. A search across sources implies that Backend.c may use pq library in versions prior to 9.3 but I haven't tested this.

[jcflack]

That's great, I was expecting it to need more tweaking. I am still curious about the java.home vs. JAVA_HOME business ... could you post what each one is expanding to in your environment? (They should be in the output from maven, if you redirect it to a file it shouldn't be hard to pull them out.)

I'm suspecting that they might refer to slightly different places in the tree so the relative paths following them have to be slightly different. I find myself with a mild preference for relying on the property set by the Java runtime itself rather than the one found in the environment (you don't know where that's been!), if we can make it work by putting the right relative path after it.

About the three libraries, I'd be curious what happens for you if you just delete those 15 lines right out of the POM for experiment's sake. For me, nothing at all seems to miss them. I would expect them to make a difference if we were building a main module that had to pull those other things in when invoked, but pljava.so is a thing that gets pulled in to a process where that other stuff has all been resolved already. At least that seems to be the case in my environment, and now I'm filled with curiosity to know what environment that isn't true for and why.

I guess the commit that put them in was named "Updated to Java7 and clang for Mac OS X Mavericks" so maybe that's my big hint. :) Maybe they can go in a separate OS X profile. I guess they don't really hurt anything, they just add that special kind of befuddlement you get when trying to understand a file and wondering "am I crazy for thinking these shouldn't be needed here?"

-Chap

[jcflack]

(They should be in the output from maven, if you redirect it to a file it shouldn't be hard to pull them out.)

... output from maven -X I mean. I've been running it with -X so much lately I'd forgotten it wasn't normal. :)

[jcflack]

Wait, hold the phone. I've updated my branch. Turns out there's a nar plugin config option that's supposed to automatically add the right <lib> spec to link the JVM. It works for me. Does it work for you?

No more worrying about sun.misc.boot.library.path java.home JAVA_HOME ....

[kenolson]

Your latest change with the nar option to add the spec works for me also - compiles and builds the windows dll just fine.
Resulting dll works fine on windows without the ecpg, pgtypes and pq entries.

I found some notes from a colleague who built pljava for mac (in Septemeber 2014 from tada/pljava/master) in which he says "By default the output of the native build is to create a "shared" library. On Linux platforms this usually means a ".so", on Windows this will likely mean a ".so" file. On Mac by default this creates a ".dylib" file which is not what we want. Instead we need to change the output type to "plugin" which will create a ".bundle" file which we can rename to a ".so" later." So this seems to confirm your hypothesis that these entries may be specific to the mac os build.

[jcflack]

Hi Ken,

Would you have any time to do a test MSVC build with another branch in my repo? I'm trying to simplify the pom and build.xml a bit more.

... let me know if it becomes a chore, but for right now without a proper buildfarm, it's very helpful for me to know there's someone with that environment to try out a build.

Thanks!
-Chap

[kenolson]

I should be able to get to it sometime this week, and will update you with what I find.

Ken

From: Chapman Flack [mailto:notifications@github.com]
Sent: Monday, November 09, 2015 7:35 PM
To: tada/pljava
Cc: Kenneth Olson
Subject: Re: [pljava] Fix to allow building pljava with Microsoft Visual C (#29)

Hi Ken,

Would you have any time to do a test MSVC build with another branch in my repohttps://github.com/jcflack/pljava/tree/simplify/master/libdirmacro? I'm trying to simplify the pom and build.xml a bit more.

... let me know if it becomes a chore, but for right now without a proper buildfarm, it's very helpful for me to know there's someone with that environment to try out a build.

Thanks!
-Chap

—
Reply to this email directly or view it on GitHubhttps://github.com//pull/29#issuecomment-155245314.