What is the maximum grid size for pumgen to support .msh format to puml.h5 format?

[smallriver666]

Based on your experience, what is the maximum grid size for pumgen to support .msh format to puml.h5 format.
I use this gmsh text can generate 2.1 billions meshes, but it seems pmgen can't convert .msh format to .puml.h5 format.

cl_1 =1.5e3;
Point(1001) = { 9e5,  1.2e6, -5.5e5, cl_1};
Point(1002) = { 9e5,  1.2e6,  8.6e5, cl_1};
Point(1003) = {3.1e6,  1.2e6,  8.6e5, cl_1};
Point(1004) = {3.1e6,  1.2e6, -5.5e5, cl_1};
Point(1005) = {3.1e6, 5.7e5,  -5.5e5, cl_1};
Point(1006) = {3.1e6, 5.7e5,   8.6e5, cl_1};
Point(1007) = { 9e5, 5.7e5,   8.6e5, cl_1};
Point(1008) = { 9e5, 5.7e5,  -5.5e5, cl_1};

Line(1001) = {1004, 1005};
Line(1002) = {1005, 1008};
Line(1003) = {1008, 1007};
Line(1004) = {1007, 1006};
Line(1005) = {1006, 1005};
Line(1006) = {1006, 1003};
Line(1007) = {1003, 1004};
Line(1008) = {1004, 1001};
Line(1009) = {1001, 1008};
Line(10010) = {1007, 1002};
Line(10011) = {1002, 1001};
Line(10012) = {1002, 1003};

Line Loop(1001) = {10012, -1006, -1004, 10010};
Plane Surface(1001) = {1001};
Line Loop(1002) = {1005, 1002, 1003, 1004};
Plane Surface(1002) = {1002};
Line Loop(1003) = {1003, 10010, 10011, 1009};
Plane Surface(1003) = {1003};
Line Loop(1004) = {1006, 1007, 1001, -1005};
Plane Surface(1004) = {1004};
Line Loop(1005) = {1007, 1008, -10011, 10012};
Plane Surface(1005) = {1005};
Line Loop(1006) = {1008, 1009, -1002, -1001};
Plane Surface(1006) = {1006};
Surface Loop(111) = {1001,1002,1003,1004,1005,1006};
Volume(1) = {111};

Physical Surface(101) = {1001,1002,1003,1004,1005,1006};
Physical Volume(1) = {1};

Mesh.MshFileVersion = 2.2;

[sebwolf-de]

We have used PUMGen for meshes up to ~500 million elements. I don't see any reason why it should not work for 2.1 billion, given you have sufficient RAM available.
Without more information, e.g. the error message, I can't help you any further.

[smallriver666]

I would like to test the limit of the number of meshes that pumgen can fit, I generated 900 million meshes with this script, but pumgen is reporting errors, have you tried that many meshes?

cl_1 =2e3;
Point(1001) = { 9e5,  1.2e6, -5.5e5, cl_1};
Point(1002) = { 9e5,  1.2e6,  8.6e5, cl_1};
Point(1003) = {3.1e6,  1.2e6,  8.6e5, cl_1};
Point(1004) = {3.1e6,  1.2e6, -5.5e5, cl_1};
Point(1005) = {3.1e6, 5.7e5,  -5.5e5, cl_1};
Point(1006) = {3.1e6, 5.7e5,   8.6e5, cl_1};
Point(1007) = { 9e5, 5.7e5,   8.6e5, cl_1};
Point(1008) = { 9e5, 5.7e5,  -5.5e5, cl_1};

Line(1001) = {1004, 1005};
Line(1002) = {1005, 1008};
Line(1003) = {1008, 1007};
Line(1004) = {1007, 1006};
Line(1005) = {1006, 1005};
Line(1006) = {1006, 1003};
Line(1007) = {1003, 1004};
Line(1008) = {1004, 1001};
Line(1009) = {1001, 1008};
Line(10010) = {1007, 1002};
Line(10011) = {1002, 1001};
Line(10012) = {1002, 1003};

Line Loop(1001) = {10012, -1006, -1004, 10010};
Plane Surface(1001) = {1001};
Line Loop(1002) = {1005, 1002, 1003, 1004};
Plane Surface(1002) = {1002};
Line Loop(1003) = {1003, 10010, 10011, 1009};
Plane Surface(1003) = {1003};
Line Loop(1004) = {1006, 1007, 1001, -1005};
Plane Surface(1004) = {1004};
Line Loop(1005) = {1007, 1008, -10011, 10012};
Plane Surface(1005) = {1005};
Line Loop(1006) = {1008, 1009, -1002, -1001};
Plane Surface(1006) = {1006};
Surface Loop(111) = {1001,1002,1003,1004,1005,1006};
Volume(1) = {111};

Physical Surface(101) = {1001,1002,1003,1004,1005,1006};
Physical Volume(1) = {1};

Mesh.MshFileVersion = 2.2;

My computer configuration is as follows:

[davschneller]

You can check out the bypass-apf branch here SeisSol/PUMGen#68 ; it might fix the problem. At these sizes, APF can sometimes encounter in strange errors—but we can avoid it entirely.

[smallriver666]

How should I clone the code of this branch, I directly use git clone --depth=1 --recursive https://github.com/SeisSol/PUMGen.git pumgen to get the code compiled successfully and still did not avoid the APF.

use git clone -- recursive https://github.com/SeisSol/PUMGen/tree/davschneller/bypass-apf.git encountered this problem:
fatal: repository 'https://github.com/SeisSol/PUMGen/tree/davschneller/bypass-apf.git/' not found

[sebwolf-de]

https://git-scm.com/docs/git-checkout

[davschneller]

In addition to git checkout, you can also define the branch on the clone directly; like this:
git clone --depth=1 --branch=davschneller/bypass-apf --recursive https://github.com/SeisSol/PUMGen.git pumgen-bypass-apf

(you can also use a tag instead of a branch there, e.g. --branch=v1.0.1 would give you the respective version, 1.0.1)

[smallriver666]

I compiled the pumgen-bypass-apf version of pumgen, but I encountered this problem with both msh2 and msh4 meshes, and the gmsh format mesh is relatively inefficient for large scale mesh conversion.

I would like to try to use Simmetrix, but I don't have a login and I can't find the registration location, where is the link to register, and by the way, for linux, do you provide a license here.

[davschneller]

How many elements/vertices does your mesh have exactly? Is it more than 2,147,483,648? At that point of time, unfortunately, you may run into trouble with PUMgen, as it's limited by 32 bit signed integers in some points at the moment—the number mentioned is the most elements that are supported right now.

[smallriver666]

In addition, does SeisSol support other grid format inputs besides the puml.h5 format?

[davschneller]

It does support a Netcdf-based format, but it's effectively deprecated and only used for the convergence tests by now (cf. the program in preprocessing/meshing/cube_c). It may be removed at some point in the future, and I don't know if there's much documentation still online about it. It required an a priori partitioning and it fixed the cell distribution onto MPI ranks (thus, it was also fixed to one particular MPI process count).

Ultimately, we may have to extend PUMgen over that mark; maybe I can get a PR done within the next weeks, but no promises on that. SeisSol will also need some smaller updates to support the PUML format with larger IDs, as it should fail at latest during mesh correctness checking at the moment (the IDs there are also restricted to 32-bit signed ints—even though PUML stores them as 64-bit unsigned).

[krenzland]

Are you actually sure that you have the computational resources to run such a mesh? This would require thousands of compute nodes.

[davschneller]

Ok, here's a branch for PUMgen, but by now no guarantee for success (I'll still have to generate such a large mesh first):

• https://github.com/SeisSol/PUMGen/tree/davschneller/mesh64 (currently not well tested, it works for small msh2 files ... And it also includes compression, if your HDF5 implementation supports that)
• Note that you'll need a newer SCOREC/PUMI version than v2.2.7 for that; none of the existing tags seems to work, master seems to have the change in that we may need. (if you compile master, you may need to compile with CC=mpicc CXX=mpicxx)

If it still doesn't work, then maybe it will with multiple MPI ranks.

[smallriver666]

How many elements/vertices does your mesh have exactly? Is it more than 2,147,483,648? At that point of time, unfortunately, you may run into trouble with PUMgen, as it's limited by 32 bit signed integers in some points at the moment—the number mentioned is the most elements that are supported right now.