Mesh Loader Attributes

[fredrikbk]

The mesh loader currently supports loading topological data from tetgen.ele, .edge and .node files. It needs to be extended to load attributes, either from tetgen files, such as .node files or from a separate file we specify so that we can load initial conditions and other fields.

[jhemmelg]

Hello! I am interested in contributing to simit and this looks like a good place to start. I have experience with C and C++, but I am new to the simulation/FEA side of things. I would need some guidance on what attributes would be expected, and what file formats we would want to support.

[fredrikbk]

That's awesome! I think we might want an API similar to what libigl has with similar file format support. Two file formats we have partial support for are tetgen and wavefront obj. Tetgen supports providing attributes (set fields) as part of their element/vertex files, but I think we also need a way to specify them as a separate file (custom?). The reason for this is that you'd often want many different initial configurations of the same mesh for different simulations.

Example APIs (design borrowed from lbigl):

simit::writeOBJ("cube.obj",V,F);

In Simit V would be a vertex set and F an edge set. The function could be templated on the number of endpoints of F.

Here's a list of file formats we want:

• Tetgen .ele, .edge, .node with attributes
• Wavefront .obj
• Geomview's .off
• Medit .mesh
• Custom .fld files containing the values of one ore more set fields
• MFem .mesh

[jhemmelg]

OK. I am starting on the custom format first. I have been reviewing the formats and any individual format should not be too hard to implement. I think the most bang for my effort is to allow for saving and restoring state of a simulation to a custom format. Once I have that established it should be easier to figure out how to incorporate attribute I/O into the other formats.

Would it be OK to do an early pull request for review once I have a start on the interface?

[fredrikbk]

Sure, that would be great!

It would also be nice if the API of our mesh loader could be brought closer to the design of the mesh loader in libigl since I think their design is better than what we currently have.

By the way, are there any established formats for storing non-topological state? @desaic @dilevin maybe you have some information about this? We need to load/store a file containing initial configurations (x, v, etc.). Are there any standard file formats for this? Do people typically roll their own, such as the .fem scene files that's in eval?

[desaic]

I don't know any established standard format for such information. People usually use use their own formats. E.g. Vega uses some format based on Abaqus' file formats, Nastran uses a relational database to store simulation info. The main problem is that there are too many possibilities for what information to store. In addition to velocity, we may want to store forces, indices for contact vertices, etc. Allowing such flexibility usually just leads one to using a generic file format such as our .fem file that allows arbitrary vector field input, or some xml-like format.

[jhemmelg]

Would there be any interest in an interface to HDF5?

[desaic]

I don't know much about HDF5 to comment on that.

[fredrikbk]

I think there would be long term interest in HDF5. We're planning to make a distributed backend, and then it would be very useful. It's also very common in scientific computing, so it's nice to support.

[VikingScientist]

As far as I understand, then HDF5 is not a geometry-supported file in itself, but rather a convenient container to put large data sets in, with binary storage, compression and parallel read/write capabilities. The actual interpretation of the data varies widely from application to application. I.e. you could put the raw data of tetgen or obj files into a hdf5-wrapper and it would need two completely different parsers to get it out, even if they describe the same geometric object.

Example APIs (design borrowed from lbigl):

simit::writeOBJ("cube.obj",V,F);

In Simit V would be a vertex set and F an edge set. The function could be templated on the number of endpoints of F.

I very much like this. I think simit:Set should be the default interface for mesh read/write. Read operations should produce and return all relevant sets, while write operations should take this as input arguments. One issue I see here is that the main way of storing things in set objects is by FieldRef and these are identified by strings. Different file formats store different information, so care must be taken when adding these in. Examples:

simit::set vertex;
simit::set triangle(vertex, vertex, vertex);
simit::readOBJ("cube.obj", vertex, triangle);
simit::readTETGEN("cube", vertex, triangle);

valid simit-program for OBJ files:

element Vertex
  x : vector[3](float);  % position
  vt : vector[2](float); % texture coordinate
end
element Triangle
  n : vector[3](float);  % normal
end

valid simit-program for OBJ files and TetGen files:

element Vertex
  x : vector[3](float);  % position
end
element Triangle
end

Not a valid simit-program for OBJ files nor TetGen files

element Vertex
  pt : vector[3](float);  % position
end
element Triangle
end

Both readOBJ and readTETGEN should be valid syntax, but should return sets with different FieldRef. That is the Obj file format stores additional information, and this could be utilized if the simit program knows that this is the mesh source, but one could also write more general simit-programs with less assumptions and thus wider application. The thing to take away here is that mesh readers are storing variables as FieldRef identified by strings, and thus care have to be taken in the choice of naming convention.

[VikingScientist]

This is the start of an idea I had for a GMSH reader

https://github.com/VikingScientist/simit/blob/meshreaders/src/gmshreader.cpp

In particular I would like to draw attention to line 93-96, where I define the FieldRef-variables used inside the mesh-reader. Just have to be careful when choosing the naming strings.

[jhemmelg]

@VikingScientist I am planning to have meta-information about the names and types of fields stored in the simit custom format, and use that in creating the values on read.

Conventions for these field names and types depends on the application that is built using simit. Likewise for HDF5, it is possible to have multiple data sets and I would put in a descriptive data set with field names and types, and a "data" data set with the actual data. I see HDF5 being a good alternative for either writing results for each iteration of the simulation or for checkpointing a long simulation. It should be easy to associate one "header" data set with multiple "data" data sets.

[fredrikbk]

All of this generality may cause us to need file loaders with a fair amount of choice, but good defaults.

For example, if I have an obj file with positions then simit::writeOBJ("cube.obj",V,F) requires the position field to exist in V before the call. (We could let the loaders add fields automatically, but I think that is too surprising and would lead users to not realize they are using more memory than they thought.) If the user needs to rename fields, maybe the x field in cube.obj needs to be stored in the q field of V then he/she can perhaps provide a renaming: simit::writeOBJ("cube.obj",V,F, {q:'x'}).

Also, I think our loaders should also provide an overloaded interface that takes an ostream instead of a filename.

[fredrikbk]

For the custom fld, here's some suggestions to start the discussion.

• Allow comments. One liners starting with # or the Simit % should suffice.
• For each data field or scalar, store the field name followed by a whitespace separated list of values (any amount of whitespace \t\n. If the data is a field, then the first number following the name should be the size. For example,

% Two scalars
dt 0.001
mu 0.4

% 3x3 matrix
r 9  1 2 3 4 5 6 7 8 9

% Normals
n 3
1.0
2.0
3.0

% Any type of whitespace is fine
m 3 10 20 30

% Is this the right way to store a 3-component field of type FieldRef<double, 3>? It seems it is not
% necessary to type the blocking in the file format?
x 9  1.1 1.2 1.3  2.1 2.2 2.3  3.1 3.2 3.3

The above suggestion does not have typing in the file format. So both r and x can be stored into one of: 3x3 matrix, FieldRef with 9 components, FieldRef<double,3> with 3 components. I don't see a problem with that. Does anyone else?

• The file loader interface may be something like simit::readFields(string filename, FieldRefsOrTensors fieldsOrTensors) where fields is a variadic template argument where each expanded argument is either a simit::FieldRef or a simit::Tensor. In the first version it may be best to just test with a vector of FieldRefs. (If you are not too familiar with variadics I can help you with that once the basics are there.)

[Lugatod]

We could also consider CGNS format as an input file :
http://cgns.github.io/WhatIsCGNS.html
There ares several good reasons :

• it can be compiled above HDF5 and so have very good performance with parallel I/O
• it is a standard originally defined for CFD but not only, you can store the mesh but also physical quantities with units, and boundary conditions
• it accepts all kinds of grids : structured, unstructured, hybrid
• because you can store everything in the format, it can be also used for checkpoint/restart of a computation.

[jhemmelg]

@Lugatod, That looks very interesting. I will look into what we need to do to use CGNS.