New pcre2 release - should we rebuild against it?

[tannerhelland]

A quick check shows that pcre2 has updated since the last build provided in this repo:

http://www.pcre.org/news.txt

Would new VB6-compatible binaries be useful/desirable?

[jpbro]

Thanks again Tanner.

At first blush, my reaction is "Yeah!", but I know you are a busy man, and I don't want to create extra work for you.

As a one-off would you be willing to record the steps you take to setup the build environment and compile PCRE2? Sorry for my ignorance here - I know that's more work for you in the short-term, but it might be less in long-term. Of course, if you'd rather just do the build yourself, I totally understand and I really appreciate your contributions to this project!

[tannerhelland]

I'd love to turn over the full DLL build process to you. Getting more VB devs involved in building 3rd-party libraries is ideal! IMO it's one of the best things we can do to modernize the language.

Let me jot down a "quick and dirty" instruction list. This should be enough to get a new pcre2 built, one that works with your existing code. You won't want to put it into this repo yet (we'll need to go over more specific details before doing that), but since the first step is just downloading a bunch of crap, it may take a few days just to prep the build environment.

Sorry in advance for how terse this is going to be. Feel free to throw out questions if you hit any roadblocks, as it's entirely possible that I'll forget a crucial step somewhere.

Things to download:

1. Latest pcre2 source code
2. Latest VS 2017 Community Edition (this takes forever to download and update, sorry)
3. Latest CMake

CMake is the real lifesaver here, and if you can get comfortable with it, you're 90% of the way toward building most modern 3rd-party libraries.

Once everything is downloaded, load CMake (cmake-gui) from your Start Menu. Point the source code path to the base pcre2 source folder (the one with CMakeLists.txt in it), and set the build folder to wherever you want the VS solution to be built. Then hit "Configure" and leave all the default settings (the default VS 2017 toolchain should already be selected).

Next you'll need to select the appropriate pcre2 build settings. These settings are project-specific and they're created by the project's maintainers, so there's no real pattern to them. Pcre2 is a little tough in this regard as it presents a lot of build options. You may have to review the project's docs to see what's relevant/useful for your particular case. Generally speaking, you'll want to disable "static builds" and enable any "shared/dynamic" builds since for VB6, we're typically building dlls only. IIRC, pcre2 also has some Unicode-specific bits that need to be set to make VB6 interop easier (like passing everything as UTF-16, so we can just cheat and use StrPtr everywhere).

Once you have your settings how you want (and don't worry - you can always revisit them later and generate a new project), hit "Generate" to build a corresponding VS solution. When it's done, navigate to your build folder in Windows Explorer and you should see a fresh batch of VS files. Find the main solution (something like "pcre2.sln") and open it in VS 2017.

Take yourself out to a nice lunch while VS loads. It'll probably take awhile.

Once the solution has opened, select the project you want to build (something like "pcre2") from the right-hand treeview, and at the top of the VS window, change its build type from "Debug" to "Release". Then go to Project > Properties. We'll spend a lot more time here in a future discussion, but for now, let's just do the bare minimum required to get a functional DLL built.

Specifically, you need to go to the C/C++ > Advanced panel and manually change "Calling Convention" from _cdecl to _stdcall.

Now you should hypothetically be able to build a functional DLL. ("Hypothetical" is the keyword here.) If it builds successfully, great! Test out the new DLL and see if it works. If it doesn't build, we'll have to troubleshoot. (And by "troubleshoot", I mean invoke the almighty Google. Every project is different.)

That is a very quick and dirty "getting started" guide, but maybe it'll get you part of the way there.

Thank you for offering to take on the build! You can probably figure out neat features and/or build options that I missed (pcre2 is complicated), so this could very well result in a much better version of the project than I was able to provide. Yay for that!

[dragokas]

Video-instruction: https://github.com/dragokas/PCRE

Please, do not follow directly by my instruction. Not all functions from my final Dll working correctly. It looks like I touched on extra settings when configuring VS project.

P.S. Thank you tannerhelland for detailed explanation.

[tannerhelland]

@dragokas thank you for the follow-up! Do you know what functions in your DLL are not working? If they are broken in your copy, they may be broken in mine, too. I am happy to investigate to see if we can fix it.

[jpbro]

This is great, thank you very much @tannerhelland and @dragokas! I really appreciate your efforts and I'll give it a try myself this weekend.

[dragokas]

tannerhelland, I don't remember the concrete function, but in jpbro's earlier build that uses dll of pcre, you can see error in IDE (also, I guess it repeated in my wrapper). Just try replace dll in wrapper by this dll

P.S. tannerhelland, your VS pre-built environment has no such bug.

[tannerhelland]

Thank you, @dragokas. I appreciate the follow-up information. Hope you are doing well!

@jpbro - good luck with everything!

[jpbro]

I finally got around to trying to build the latest PCRE2, and everything seems to have worked thanks to @tannerhelland's detailed instructions! The only "weird" thing I've noticed is that my build is 239KB, while @tannerhelland's older build was 405KB. It could be that the PCRE2 team has optimized things, but I'm worried that I've done something wrong like leave an important project property unselected.

Since it seems to pass all the current tests (which are admittedly not exhaustive), I think I'll just publish the new build anyway, and we can always revert to the old one if I've mucked something up.

[jpbro]

Published latest PCRE2-16.dll binary build of 10.31 in commit f2d63cc

[tannerhelland]

Hi @jpbro. Unfortunately, I wasn't joking in my post above when I called it a "quick and dirty" guide. Those steps were enough to get a DLL built, but building something you can safely distribute and use on other PCs requires quite a few more steps. So I wouldn't close this issue just yet! (And in fact, that extra DLL size will make sense momentarily.)

Here are some things that still need to be done. Almost all of this can be handled from the same Project > Properties page we used before.

• Currently, your DLL is unlikely to work on an OS prior to the one on which it was built. Assuming you're on Windows 10, this is rarely desirable. For maximum compatibility, you will need to change the Configuration Properties > General > Platform Toolset value to Visual Studio 2017 - Windows XP (v141_xp). If you ever encounter a project that doesn't build properly in VS 2017, you may find success using an older toolset (e.g. Visual Studio 2012 - Windows XP). Toolsets often require additional installation, so if you don't see an XP option, you may need to find a manual download, or set a different set of install options for VS 2017.

• By default, modern VS projects are dynamically linked to the C runtime on the developers PC (msvcrXX.dll). When built this way, you will need to deploy that same C runtime alongside the DLL using the usual deployment rules. Because this is a huge headache in most cases (especially when covering XP support), it is easier to just statically link the runtime, which is how C projects work on other platforms. Some projects will expose this as a property in CMake (e.g. "use static runtime") or you can set it manually by going to Configuration Properties > C/C++ > Code Generation > Run-time library = /MT (not /MD). Depending on version, this could add another ~200 kb to the DLL on its own, but the DLL will be deployable as a standalone DLL on any Windows version supported by the build toolkit.

[tannerhelland]

• Next, you'll probably want to visit the C/C++ > Optimization page. There are a number of options here, and if you want "raw speed" (which is probably what I built my PCRE2 dll with), you will need to manually enable various optimizations. Each of these has a "size / performance" trade-off, and some developers will build separate "minimal size" and "maximum speed" versions of their DLL, which users can select between at their leisure. What you choose is up to you, but this may also increase DLL size (at some boost to performance). Note that if you select "Whole Program Optimization", you will also need to modify a Linker option to match (as described right there in the VS window).

• If you decide to build using the XP toolkit, VS should automatically set the proper subsystem to match, but I generally set this manually as some projects have wrong numbers built-in. (This is especially common with libraries designed against Linux.) Go to Linker > System settings and set the subsystem version to the matching XP number (5.01 - details here: https://docs.microsoft.com/en-us/dotnet/csharp/language-reference/compiler-options/subsystemversion-compiler-option )

That is kind of the bare minimum to construct a "safely deployable" DLL. You will likely want an XP PC or VM around to test, or if you don't have one, let me know and I'll run the test project on an XP build just to be safe. (Note that in many cases, code built with VS 2017 - even with the XP toolkit - may still requires SSE2. Very old PCs running on netbooks may lack this, but I've hit a point where it's not worth the effort to work around it. Just an FYI in case you ever get a user that hits inexplicable errors - check their PC's SSE2 support before debugging the C code.)

If all those settings result in a build that works on XP, great! You're most of the way there! The last thing I would recommend would be playing with some different PCRE2 CMake options (like their JIT option(s)) to find a happy medium between performance and size/complexity, or to manually remove options that aren't relevant to VB. I believe I did include the JIT in my original build, but I did not do any perf testing to see if it made a huge difference. That may also explain why my build was larger.

Anyway, I think that's everything. Thanks again for taking this over! Hopefully you can add many more open-source C libraries to your VB repertoire!

[jpbro]

I had a feeling I'd be re-opening this one :)

That's for the additional information Tanner - I'm reviewing it now, and I will issue a new build soon.

[jpbro]

Thanks a lot @tannerhelland! I've rebuilt the pcre2-16.dll based on your recommendations in commit 25b68dd .

My build is still coming out smaller that your build even though I optimized for speed over size, and statically linked the VC++ runtime, but it is larger than my previous build suggesting that I've at least done something right :)

I also tried running the test EXE on an XP VM with the new pcre2-16.dll build, and everything worked, so I'm hoping this is now done, but I'll leave the issue open for a while just in case.

Many thanks for your help on this! I probably couldn't, and certainly wouldn't have done it without your assistance 🥇

[jpbro]

For anyone stumbling along here in the future - I didn't have the XP support files installed with VisualStudio 2017 by default. As @tannerhelland alluded to, make sure you install it by selecting the following option in the VS installer:

[tannerhelland]

Excellent work, @jpbro. Congrats on getting everything built!

[jpbro]

Thank you @tannerhelland! There was nothing "tricky" about it at all, but without your advice it would have taken me ages to figure out on my own where all the various options are hidden, and which ones are required use/change. The fact that I didn't have XP support installed by default would have been a particular pain point, because the XP option wouldn't have even been in my drop-down list! Once you pointed out where everything was (or should be) it was all pretty straight forward stuff...and you are right about CMake - it really makes life a lot easier!. Again, thank you very much for your significant contributions to this project :)