readers.pts doesn't deal with unexpected characters in field names

[hobu]

PTS uses readers.text, and users can name the dimensions whatever they want. We need to strip out and normalize characters we can't consume in field names like ='s and .'s.

[florentPoux]

Here are the first 8 lines (second line is number of points) :

//X;Y;Z;Rf;Gf;Bf;Intensity;curvature;Illuminance (PCV);Volume density (r=0.01);Roughness(0.01);Nx;Ny;Nz
832641
1.48558950;2.78713155;1.71654594;1;0.976471;0.988235;0.514534;0.000000;0.889831;86421136.000000;0.000164;0.842968;-0.279958;-0.459379
0.66884160;2.91032434;1.71380365;0.996078;0.996078;0.996078;0.419318;0.149188;0.851695;41061980.000000;0.001045;-0.557930;-0.818396;-0.137633
1.60763741;2.74084330;1.70552599;1;0.996078;1;0.307012;0.000000;0.614407;56579588.000000;0.000358;0.410376;-0.815962;-0.407183
0.82228744;2.85927010;1.71652365;0.870588;0.870588;0.878431;0.632944;0.000000;0.983051;51088740.000000;0.000713;0.094079;0.464780;-0.880414
1.69874430;2.57411289;1.70909595;1;1;1;0.524788;0.000000;0.745763;47746488.000000;0.000465;-0.370771;-0.556973;0.743175
1.14563203;2.78293252;1.71632600;1;1;0.980392;0.452033;0.000000;0.889831;42016908.000000;0.001126;-0.219483;-0.975079;0.032383

[abellgithub]

The text reader uses the first non-alpha numeric character found on the first line as a delimiter. All other characters are valid and used to make the field name. In the case of the example in the above note, the '/' character is used as the delimiter, which creates a single dimension with the name X;Y;Z;Rf;Gf;Bf;Intensity;curvature;Illuminance (PCV);Volume density (r=0.01);Roughness(0.01);Nx;Ny;Nz If you remove the '/' characters from the beginning of the first line, things should be fine.

I'll try to clarify the documentation and add a command-line option to allow you to explicitly set the delimiter.

[hobu]

He has no way to update because this is a .pts file and he doesn't control the output.

[abellgithub]

What is a pts file? Perhaps we need a pts reader if we want to support this. And we need to understand the format if we're going to do this.

[hobu]

yet another ascii point cloud format. We have a reader, https://www.pdal.io/stages/readers.pts.html and I simply used the readers.text internals. It sould be updated to handle this situation.

[abellgithub]

Awaiting details on the format of a PTS file.

[abellgithub]

@Poux-Florent : What software were you using that generated the file that you provided above?

[florentPoux]

@abellgithub : I was using at this time CloudCompare (batch export can only generate ASCII files readable by PDAL)

[hobu]

CloudCompare (batch export can only generate ASCII files readable by PDAL)

Can CloudCompare write LAS/LAZ? Those would be readable by PDAL (and much smaller!)

[florentPoux]

@hobu : Yes it can, but unfortunately, not as a batch export (I often have thousands of point clouds, exporting them without one by one manual operation can only be done in ASCII format)

[hobu]

Which format are they in that only CloudCompare can read them and batch?

[florentPoux]

I am not sure I understood correctly, but in CloudCompare it is a binary format .bin that necessitate libraries from CloudCompare to be readable