Develop blueCFD-Core 2017-1 and port OpenFOAM 5.x to Windows

[wyldckat]

Development steps:

Note: Don't forget to update issue #53 as development goes on.

1. Resync changes which we have internally at the intersection of OpenFOAM-4.x and OpenFOAM-dev, with the fixes which were done in blueCFD-Core-4.x. Note: for both the OpenFOAM-dev and ThirdParty-dev repositories.

2. Merge our internal branch blueCFD-Core-dev at this intersection of 4.x+dev to the intersection of 5.x+dev. Note: for both the OpenFOAM-dev and ThirdParty-dev repositories.

3. Create the new branch blueCFD-Core-5.x and merge up to the latest commit of OpenFOAM-5.x. Note: for both the OpenFOAM-dev and ThirdParty-dev repositories.

4. Create a new MSys2 installation on Windows for blueCFD-Core 2017 and adapt the infrastructure from blueCFD-Core 2016.

5. Git clone OpenFOAM-dev and ThirdParty-dev repositories on Windows, by checking out the blueCFD-Core-5.x branch, adjust the infrastructure and compile OpenFOAM 5.x on Windows. Also fix the remaining loose details from steps 1 to 3, namely changes that couldn't be applied directly.

6. Adapt to the latest MS-MPI version.

7. Run Alltest - see if Implement the ability to run the 'tutorials/Allrun' script for just X seconds per solver #57 can be done here.

8. If there is still enough time, see issues Bring back stack tracing #7, Check status on MSwindows/signals #4 and Using exit(1) instead of ::abort on Windows #5.

9. Test issue CompressibleInterDyMFoam tutorial case fail to pass the test. #22 to see if it still occurs.

10. Adapt the installer scripts from blueCFD-Core 2016-2 to 2017-1, test the installer(s) and wrap up the release - do it in issue Create the installer for blueCFD-Core 2017-1 #59.

11. Release blueCFD-Core 2017-1 to the public,

[wyldckat]

Steps 1, 2 and 3

Development notes:

• On the main repository OpenFOAM-dev, had to create a temporary branch, then used our internal script for adapting the source code to the Windows file-system compatible structure. Then had to manually cherry-pick the changes from the blueCFD-Core-4.x branch.
• Then checked out the commit b11b28db06c427ff7e2e1095a67c59f06eba661d, which is the last commit on OpenFOAM-dev before it was forked to OpenFOAM 5.x. Here created a temporary branch and adapted the source code used our internal script for adapting the source code to the Windows file-system compatible structure.
  • applications/utilities/miscellaneous/foamList was removed and we had a hack for this that is no longer needed since this was removed.
  • applications/utilities/postProcessing/graphics/PV3Readers was removed from OpenFOAM too, so we may need to manually clean this up.
  • bin/foamRunTutorials had to be manually adapted, given that now there were some overlapping features.
  • WARNING src/OpenFOAM/db/IOobject/IOobject.C needs careful attention, because filePath() was replaced by two alternative methods, as well as the objectStream method was removed.
  • WARNING src/OpenFOAM/include/OSspecific.H changed considerably, therefore there are several changes needed on the implementation currently done for Windows.
  • Folder src/thermophysicalModels/thermophysicalFunctions was deleted.
• Have merged with OpenFOAM 5.x with success, but still need to sort out all of the details listed in the previous bullet point.
• Regarding ThirdParty 5.x, there were no critical issues. However, it does look like the README.org file needs some trimming down to the MSys2 environment.

Side note - branches for OpenFOAM 5.x and updated branches for OpenFOAM-dev have also been pushed to the public repositories:

• https://github.com/blueCFD/OpenFOAM-dev/tree/OpenFOAM-5.x
• https://github.com/blueCFD/OpenFOAM-dev/tree/OpenFOAM-dev
• https://github.com/blueCFD/ThirdParty-dev/tree/OpenFOAM-5.x
• https://github.com/blueCFD/ThirdParty-dev/tree/OpenFOAM-dev

[wyldckat]

Step 4

References:

• MSYS2 Wiki - Installation instructions: https://github.com/msys2/msys2/wiki/MSYS2-installation
• Download the MSys2 installer from here: http://msys2.github.io/

Once installed, make sure to start the MSYS2 Shell (Start -> Programs -> MSYS2 -> MSYS2 Shell) and follow the update steps from here: https://github.com/msys2/msys2/wiki/MSYS2-installation#iii-updating-packages

Start the MINGW64 Shell and install the following packages:

1. For essentials:

   pacman -S mingw-w64-x86_64-toolchain flex bison mingw64/mingw-w64-x86_64-zlib mingw64/mingw-w64-x86_64-readline mingw64/mingw-w64-x86_64-ncurses mingw64/mingw-w64-x86_64-boost mingw64/mingw-w64-x86_64-gmp mingw64/mingw-w64-x86_64-mpfr mingw64/mingw-w64-x86_64-cgal tar wget patch make unzip diffutils git
   

   It will ask if all of the members for the group mingw-w64-x86_64-toolchain are to be installed, to which you can simply press the Enter key to continue.

2. For 3rd party software:

   pacman -S mingw64/mingw-w64-x86_64-cmake mingw64/mingw-w64-x86_64-extra-cmake-modules mingw64/mingw-w64-x86_64-meld3 svn p7zip tree doxygen man  mingw64/mingw-w64-x86_64-gnuplot
   

3. gnuplot, Qt and VTK are not installed for now, because gnuplot is installed separately as an add-on and because Qt+VTK would only be useful for either PyFoam or building ParaView from source code...

Development notes

1. The following batch files:

   msys64/msys2_shell.bat
   msys64/mingw32_shell.bat
   msys64/mingw64_shell.bat
   

   were merged into a single batch file msys2_shell.cmd, by using the arguments -mingw32, -mingw64 and -msys to differentiate between each option.

2. The file msys64/etc/profile changed a bit and our hacks for the features in blueCFD-Core needs to be adapted accordingly.

3. The batch file used for launching the MSys2 terminal has been revised accordingly.

[wyldckat]

Step 5

1. Clone the repositories, by checking out the correct branch... e.g.:

   git clone https://github.com/blueCFD/OpenFOAM-dev -b blueCFD-Core-5.x OpenFOAM-5.x
   git clone https://github.com/blueCFD/ThirdParty-dev -b  blueCFD-Core-5.x ThirdParty-5.x
   

2. Adjust the convenience script /home/ofuser/.blueCFDCore/of5x.sh for initializing the shell environment from the menu.

   • Also copy the custom scripts for loading additional details, by running:

   cp -r OpenFOAM-4.x/etc/config.d OpenFOAM-5.x/etc/
   

3. Start looking into the patches that weren't applied properly.

   • osRandomBit() has been dropped from our hack-list.

   • It seems like it might be easiest to rely on Boost threads for adding the new thread management system that OpenFOAM is now using.

4. Look into the new methods and changes made to the OSspecific.H structure.

   1. Opened Add support for following links in 'MSwindows' #60 for extending what is now in Foam::type and related file/directory functions.

   2. Added Function 'domainName()' not implemented on 'MSwindows.C' #61 and Function 'isAdministrator()' not implemented on 'MSwindows.C' #62 for FIXME entries that didn't have tasks associated to them.

   3. Added Symbolic linking function 'ln' in 'MSwindows.C' not yet supported #63 for ln and Function 'ping' in 'MSwindows.C' is not implemented #64 for ping.

   4. Need to revise how outputting messages is being done, with additional support needed for using FUNCTION_NAME instead of using manually written names.

   5. Trying to implement highResLastModified with nanosecond resolution, but sys/stat.h doesn't provide this out-of-the-box on MinGW-w64. Had to create Add 'st_atim.tv_nsec' support to 'fileStat::fileStat' #65 and at least have a workaround for compiling without an updated value.

5. Compile the 3rd party software.

   • Note: MS-MPI 7.1 was already installed on my system, therefore the only solution I could easily use was to simply copy the local installation from 4.x to 5.x.

6. Compile OpenFOAM 5.x and gradually fix issues...

   1. POSTPONED Foam::fileOperations::masterUncollatedFileOperation::splitProcessorPath is going to need some reworking in file src/OpenFOAM/global/fileOperations/masterUncollatedFileOperation/masterUncollatedFileOperation.C, because it's hard-coded on POSIX folder paths.

      • Postponed this to 'Foam::fileOperations::masterUncollatedFileOperation::splitProcessorPath' needs to be system-agnostic #66.

   2. Need to revise how the file names were adapted.

      • The problem was that some files changed in OpenFOAM that were originally template classes and then became normal classes, therefore these file re-renames were not accounted for and therefore the solution was to first undo file renaming and then redo file renaming, in order to repair the broken file structure.

   3. masterUncollatedFileOperation: several problems with mismatched template types.

      global/fileOperations/collatedFileOperation/OFstreamCollator.C:289:45: error: ambiguous overload for 'operator<<' (operand types are 'Foam::Ostream' and 'off_t {aka long int}')
      
      global/fileOperations/masterUncollatedFileOperation/masterUncollatedFileOperationTemplates.C:58:12: error: no match for 'operator>>' (operand types are 'Foam::UIPstream' and 'short unsigned int')
      

      • The first has happened in the past. Had to switch from off_t to the explicit off64_t, to avoid 32-bit sizes, which nowadays are barely usable for high-performance computing.

        • Best to define the macro _FILE_OFFSET_BITS=64 in the compiler rules, instead of hacking into the files!

      • The second one seems to occur due to mode_t, which is short unsigned int on mingw-w64?

        • OK, so it seems that on Linux the definition is unsigned int, but on mingw-w64 it's short unsigned int... must be a very old inheritance.

        • Added the file src/OpenFOAM/include/hacks_on_mingw.h, which is now used by the two critical files that need mode_t: OSspecific.H and fileOperation.H.

   4. In the library twoPhaseMixture, this error happened:

      In file included from alphaContactAngle/alphaContactAngle/alphaContactAngleFvPatchScalarField.C:26:0:
      alphaContactAngle/alphaContactAngle/alphaContactAngleFvPatchScalarField.H:157:58: error: 'blocking' is not a member of 'Foam::Pstream'
                   const Pstream::commsTypes commsType=Pstream::blocking
      

      • This is because we had to get the file from the sibling library, to avoid cyclic link problems.

      • Seems to have been solved, by recovering the majority of the original structure of these libraries.

   5. In the library chemistryModel, this error happened:

      In file included from X:/OpenFOAM-5.x/src/OpenFOAM/db/error/error.H:51:0,
                       from X:/OpenFOAM-5.x/src/OpenFOAM/lnInclude/error.H:1,
                       from X:/OpenFOAM-5.x/src/OpenFOAM/db/typeInfo/typeInfo.H:56,
                       from X:/OpenFOAM-5.x/src/OpenFOAM/lnInclude/typeInfo.H:1,
                       from X:/OpenFOAM-5.x/src/OpenFOAM/db/IOstreams/token/token.T.H:46,
                       from X:/OpenFOAM-5.x/src/OpenFOAM/lnInclude/token.T.H:1,
                       from X:/OpenFOAM-5.x/src/OpenFOAM/containers/LinkedLists/accessTypes/UILList/UILListIO.T.C:28,
                       from X:/OpenFOAM-5.x/src/OpenFOAM/containers/LinkedLists/accessTypes/UILList/UILList.T.C:91,
                       from X:/OpenFOAM-5.x/src/OpenFOAM/containers/LinkedLists/accessTypes/UILList/UILList.T.H:383,
                       from X:/OpenFOAM-5.x/src/OpenFOAM/lnInclude/UILList.T.H:1,
                       from X:/OpenFOAM-5.x/src/OpenFOAM/containers/LinkedLists/accessTypes/ILList/ILList.T.H:39,
                       from X:/OpenFOAM-5.x/src/OpenFOAM/lnInclude/ILList.T.H:1,
                       from X:/OpenFOAM-5.x/src/OpenFOAM/containers/LinkedLists/user/IDLList.H:35,
                       from X:/OpenFOAM-5.x/src/OpenFOAM/lnInclude/IDLList.H:1,
                       from X:/OpenFOAM-5.x/src/OpenFOAM/db/dictionary/entry/entry.H:45,
                       from X:/OpenFOAM-5.x/src/OpenFOAM/lnInclude/entry.H:1,
                       from X:/OpenFOAM-5.x/src/OpenFOAM/db/dictionary/dictionary.H:53,
                       from X:/OpenFOAM-5.x/src/OpenFOAM/lnInclude/dictionary.H:1,
                       from X:/OpenFOAM-5.x/src/OpenFOAM/db/IOobjects/IOdictionary/baseIOdictionary.H:43,
                       from X:/OpenFOAM-5.x/src/OpenFOAM/db/IOobjects/IOdictionary/IOdictionary.H:42,
                       from X:/OpenFOAM-5.x/src/OpenFOAM/lnInclude/IOdictionary.H:1,
                       from ./chemistryModel/TDACChemistryModel/reduction/chemistryReductionMethod/chemistryReductionMethod.H:38,
                       from chemistryModel/TDACChemistryModel/reduction/makeChemistryReductionMethods.H:29,
                       from chemistryModel/TDACChemistryModel/reduction/makeChemistryReductionMethods.C:26:
      X:/OpenFOAM-5.x/src/OpenFOAM/db/error/messageStream.H:304:5: error: expected unqualified-id before 'if'
           if (debug) Info
           ^
      

      • OK, DebugInfo strikes again... here's the place where it broke (compiled file with -E):

        # 7662 "X:/msys64/mingw64/x86_64-w64-mingw32/include/winnt.h"
                                         if (debug) Info
        # 7662 "X:/msys64/mingw64/x86_64-w64-mingw32/include/winnt.h" 3
                                                  ;
        

      • Well, this was rather silly... timeval is defined via time.h.

Step complete. Last commit on branch blueCFD-Core-5.x: 4eb2a93149a5fbc509166be897df4d1532ca6bcb

[wyldckat]

Step 6

Done. Have added MS-MPI 8.1 with success.

Also revised the multiBuilder script, so that we can support both MS-MPI 7.1 and 8.1.

[wyldckat]

Step 7

1. Currently working on Implement the ability to run the 'tutorials/Allrun' script for just X seconds per solver #57.

   • Only finished the implementation that allows the execution Alltest in parallel-only or serial-only cases, with 1 or more parallel jobs.

2. Scripting code to be used for running the tests:

DATETIMESTAMP=$(date +%Y-%m-%d_%H-%M)

TARGETFOLDER=$FOAM_RUN/tutorials-$DATETIMESTAMP-serial
time $FOAM_TUTORIALS/Alltest -jobs 4 -nonParallel -target $TARGETFOLDER 2>&1 | tee $TARGETFOLDER.log

echo "Cases that possibly failed:"
grep -v completed $TARGETFOLDER/testLoopReport

TARGETFOLDER=$FOAM_RUN/tutorials-$DATETIMESTAMP-parallel
time $FOAM_TUTORIALS/Alltest -parallel -target $TARGETFOLDER 2>&1 | tee $TARGETFOLDER.log

echo "Cases that possibly failed:"
grep -v completed $TARGETFOLDER/testLoopReport

3. Cases that failed in parallel:

   In IO folder:
   Application icoUncoupledKinematicParcelFoam - case fileHandler: unconfirmed completion
   Application icoUncoupledKinematicParcelFoam.uncollated - case fileHandler: ** FOAM FATAL ERROR **
   Application icoUncoupledKinematicParcelFoam.collated - case fileHandler: ** FOAM FATAL ERROR **
   Application reconstructPar.collated - case fileHandler: unconfirmed completion
   Application foamFormatConvert - case fileHandler: ** FOAM FATAL ERROR **
   
   In multiphase folder:
   Application decomposePar - case interDyMFoam/RAS/mixerVesselAMI: unconfirmed completion
   Application interDyMFoam - case interDyMFoam/RAS/mixerVesselAMI: ** FOAM FATAL ERROR **
   Application reconstructPar - case interDyMFoam/RAS/mixerVesselAMI: unconfirmed completion
   

   • The problem in interDyMFoam/RAS/mixerVesselAMI is due to the broken snappyHexMesh refinement history files, which has already known from blueCFD-Core 2016-2, namely in commit blueCFD/OpenFOAM-dev@fb74970 - it didn't propagate automatically with the git merge, due to the folder rename that occurred in OpenFOAM, from RAS to ras.

   • The issue with the collated feature has been deferred to Issue Collated IO feature not working with blueCFD-Core 2017-1 #68.

Will considered this as completed, as of commit blueCFD/OpenFOAM-dev@dda40af, because there isn't enough time at the moment to improve this feature further.

---

4. Have run Alltest again after completing Step 8 and resyncing with the upstream. Good to go to Step 9.

5. Have run Alltest once more, after doing another sync with upstream. This time there were a few more problems, although it seems like it was due to a change that was done for using the script wmake/scripts/warnAboutCV2PDBFailing

   • Renamed script to infoCV2PDBworkaround and removed the at symbol from the call in wmake/makefiles/general.

[wyldckat]

Step 8

1. Issue Bring back stack tracing #7 - done.

2. Issue Check status on MSwindows/signals #4 - done.

3. POSTPONED Issue Using exit(1) instead of ::abort on Windows #5, because it was not clear when this was strictly needed.

4. A new issue has risen up and has been assigned to Need to revamp how dependencies are forcefully loaded #70 - done.

5. Also fixed Dependency generation issued by the wmake stack is generating duplicate calls #69.

[wyldckat]

Step 9

The problem is solved in blueCFD-Core 2017-1.

[wyldckat]

Step 10

Take care of Issue #59.

Done!

[wyldckat]

Step 11

1. Write Release Notes page, based on previous major version and issue Release notes for blueCFD-Core 2017-1 #53.

2. Write Downloads section for blueCFD-Core 2017-1.

   • Update SHA1.

3. Write News entry for blueCFD-Core 2017-1.

4. Make release tag at https://github.com/blueCFD/Core/releases for blueCFD-Core 2017-1.

5. Upload installer and patch packages for blueCFD-Core 2017-1 to that tag entry.

6. Revise if download hashes are correct, namely at the Downloads page and the release tag.

7. Revise the product page at http://joomla.bluecape.com.pt/index.php?option=com_content&task=view&id=18&Itemid=30

8. Publish the pages on the Core website.

9. Publish the news page at http://joomla.bluecape.com.pt and affiliate websites.

10. POSTPONED Still need to take care of Need to add FAQ entry regarding stack tracing #76.

11. POSTPONED Add a quick FAQ entry regarding using MS-MPI 8.1.

12. POSTPONED Take care of issue Revise FAQ entries for blueCFD-Core 2017 and 2020 #77.

13. Revise https://github.com/blueCFD/Core/wiki/Released-patch-files

14. POSTPONED Revise other wiki pages: https://github.com/blueCFD/Core/wiki/

15. POSTPONED Wrap up tasks associated to this milestone: https://github.com/blueCFD/Core/milestone/3