Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

lascopcindex64 not properly referencing subdirs in windows distribution? #172

Closed
aritchie-usgs opened this issue Feb 8, 2024 · 7 comments

Comments

@aritchie-usgs
Copy link

Hi LAStools folks! I'm excited that you're adding a copc writer! I tried to use it though (in windows) and had issues. Let me know if I can help figure them out. For now I'm just using QGIS to convert although I've used PDAL too.

I have lastools installed in Windows in the c:\utils\LAStools\ dir with c:\utils\LAStools\bin added to my system environment variables. This has worked fine for the last ~10 years. when I run lascopcindex64 though I get some errors:

first, it looks like it doesn't allow specifying source coordinate systems/datums/etc? It would be nice if it supported those flags in the standard convention. For example:

lascopcindex64 -i *.laz -epsg 6340

ERROR: cannot understand argument '-epsg'

second, it looks like there's an issue discovering relative paths, and it doesn't support the standard multicore parallelization that Martin typically implemented for multiple files. This is what I see when I try to run it. Didn't let it finish because of the errors:

lascopcindex64 -i *.laz -cores 8 -odir copc -odix copc
WARNING: not compiled with multi-core batching. ignoring '-cores' ...
WARNING: building a COPC file without gpstime.
GeogGeodeticDatumGeoKey: look-up for 1116 not implemented
ERROR: cannot open 'pcs.csv' file. maybe your LAStools distribution
has no .\LAStools\bin\serf\geo\pcs.csv file. download the
latest version at http://lastools.org/download/LAStools.zip
set_ProjectedCSTypeGeoKey: look-up for 6340 not implemented
ERROR: cannot open 'vertcs.csv' file. maybe your LAStools distribution
has no .\LAStools\bin\serf\geo\vertcs.csv file. download the
latest version at http://lastools.org/download/LAStools.zip
WARNING: cannot convert CRS from GeoTIFF to OGC WKT.
^C

@Jean-Romain
Copy link
Contributor

Jean-Romain commented Feb 9, 2024

I did the implementation of lascopcindex:

First, it looks like it doesn't allow specifying source coordinate systems/datums/etc.

No, it does not. It would be a nice addition but in my mandate I designed a fast an efficient copc file builder and did not include nice convenient features.

Second, it looks like there's an issue discovering relative paths.

Well, I don't know about this one. The paths are parsed by LASlib as usual as far as I can tell, but this case could be checked.

WARNING: Not compiled with multi-core batching. Ignoring '-cores'...

lascopcindex is not multicore for two reasons. Multicore features are part of the closed-source version of the tools. I did not have access to the closed-source code in my mandate, and only implemented a single-core version. The second reason is that the current implementation is not parallelizable. However, indeed, two independent files could run simultaneously, which falls in case one.

WARNING: Building a COPC file without gpstime.

COPC is a LAZ formats 6, 7, 8 that include gpstime. Moreover, the gpstime is required to sort the points in a way that minimizes the suboptimal compression that results from the COPC standard. Because the points are not in the natural acquisition order, the LAZ compression is less effective on a COPC file. However, if the points are properly sorted by gpstime, it is not that bad and a COPC file can be only 20% bigger than its counterpart pure LAZ with ordered according to the acquisition. This is why you have a warning when building a COPC from a file format without gpstime. Without gpstime the points are literally in a random order and the compression is likely to be terrible.

GeogGeodeticDatumGeoKey: Look-up for 1116 not implemented.
ERROR: Cannot open 'pcs.csv' file. Maybe your LAStools distribution
does not have a .\LAStools\bin\serf\geo\pcs.csv file. Download the
latest version at http://lastools.org/download/LAStools.zip
set_ProjectedCSTypeGeoKey: Look-up for 6340 not implemented.
ERROR: Cannot open 'vertcs.csv' file. Maybe your LAStools distribution
does not have a .\LAStools\bin\serf\geo\vertcs.csv file. Download the
latest version at http://lastools.org/download/LAStools.zip
WARNING: Cannot convert CRS from GeoTIFF to OGC WKT.

I'm not sure of that one. It seems that the CRS recorded in your file is not recognized. The CRS of your file is likely in a GeoTIFF format and must be converted to a WKT string in order to build valid COPC files that follow the specifications. I'd say that your CRS is not recognized and can't be converted. This one needs investigation and should at least print a more informative error.

@aritchie-usgs
Copy link
Author

Hi Jean-Romain,

I did the implementation of lascopcindex:

Thank you! I love that it's coming to lastools!

The version of lastools I'm using seems to be 240125 for reference - lascopcindex64 and lasinfo both report that version. I downloaded from rapidlasso a few days ago.

ERROR: Cannot open 'pcs.csv' file. Maybe your LAStools distribution
does not have a .\LAStools\bin\serf\geo\pcs.csv file. Download the
latest version at http://lastools.org/download/LAStools.zip
set_ProjectedCSTypeGeoKey: Look-up for 6340 not implemented.
ERROR: Cannot open 'vertcs.csv' file. Maybe your LAStools distribution
does not have a .\LAStools\bin\serf\geo\vertcs.csv file. Download the
latest version at http://lastools.org/download/LAStools.zip
WARNING: Cannot convert CRS from GeoTIFF to OGC WKT.

...I'd say that your CRS is not recognized and can't be converted. This one needs investigation and should at least print a more informative error.

My CRS (EPSG:6340, exported from Metashape) is recognized by lasgrid and other lastools and QGIS without any error messages. QGIS automatically makes COPC files without any errors displayed (using all cores) if I simply drag them into a new project, and properly shows them in the correct CRS.

I think that error might be due to some way that lascopcindex64 was compiled? - in other words build parameters that make it so lascopcindex can't find the pcs.csv and vertcs.csv files in the same way that for example lasgrid or lascanopy does rather than any issue with the pointcloud. Is it possible that @rapidlasso moved things around and broke them or moved then compiled without changing paths when it was packaged in the release?

I was wondering if that's the reason for the error mostly because of this google groups post from Martin, where he said, "...you are running a duplicate version of laszip that was copied onto the desktop. This version being outside the LAStools\bin folder where it expects to be located is not able to find the 'psc.csv' file..."

In the lastools version I downloaded a few days ago, lascopcindex64 is in C:\utils\LAStools\bin, path to LASlib folder is C:\utils\LAStools\LASlib, and pcs.csv and vertcs.csv are in C:\utils\LAStools\bin\serf\geo. I haven't moved anything from any default locations in the distro.

First, it looks like it doesn't allow specifying source coordinate systems/datums/etc.

No, it does not. It would be a nice addition but in my mandate I designed a fast an efficient copc file builder and did not include nice convenient features.

maybe @rapidlasso could add the CRS and output crs command line components and other standard ones that may not yet be integrated? I noticed from the orphan file after I control-c'd that -odir and -odix seem to be supported (but not documented).

WARNING: Not compiled with multi-core batching. Ignoring '-cores'...

lascopcindex is not multicore for two reasons. Multicore features are part of the closed-source version of the tools. I did not have access to the closed-source code in my mandate, and only implemented a single-core version. The second reason is that the current implementation is not parallelizable. However, indeed, two independent files could run simultaneously, which falls in case one.

Martin implemented a "dumb" or "simple" parallelization for many tools when wildcards were used for multiple files - eg las2las - this massively speeds up single-core operations on dirs of dozens to thousands of files - that's what I was referring to. Again, maybe since you don't have access to all of the code, that would be something for @rapidlasso to add?

WARNING: Building a COPC file without gpstime.

COPC is a LAZ formats 6, 7, 8 that include gpstime. Moreover, the gpstime is required to sort the points in a way that minimizes the suboptimal compression that results from the COPC standard. Because the points are not in the natural acquisition order, the LAZ compression is less effective on a COPC file. However, if the points are properly sorted by gpstime, it is not that bad and a COPC file can be only 20% bigger than its counterpart pure LAZ with ordered according to the acquisition. This is why you have a warning when building a COPC from a file format without gpstime. Without gpstime the points are literally in a random order and the compression is likely to be terrible.

I'm not sure how Metashape structures sfm points when exporting to laz file, but there is no gpstime in SfM data. The QGIS implementation of the COPC standard made them about 25% smaller so that was pretty cool. I guess you could look at the QGIS code and see how they do it? I can barely read C or C++ so I'm pretty useless there but to you it might be interesting.

@Jean-Romain
Copy link
Contributor

QGIS uses PDAL and PDAL uses a relatively similar approach to re-organize the points. When I'm saying terrible, I actually did not measure it. That a hypothetical through based on how points are re-organized, but maybe it is not that bad. But sure it would be better with gpstime and this is very likely true for PDAL as well.

@aritchie-usgs
Copy link
Author

Just downloaded the new build. The error with lascopcindex64 not properly referencing the system path to pcs.csv and vertcs.csv still exists. No other lastools throw this error, but lascopcindex64 does:

ERROR: cannot open 'pcs.csv' file. maybe your LAStools distribution
has no .\LAStools\bin\serf\geo\pcs.csv file. download the
latest version at http://lastools.org/download/LAStools.zip
set_ProjectedCSTypeGeoKey: look-up for 6340 not implemented
ERROR: cannot open 'vertcs.csv' file. maybe your LAStools distribution
has no .\LAStools\bin\serf\geo\vertcs.csv file. download the
latest version at http://lastools.org/download/LAStools.zip

@BB-Heidelberg
Copy link
Collaborator

Which command did you run?
Does your installation contain the pcs.csv file at the required directory?
Does this question belongs here?
If it is not a question regarding the source code you rather post it on the support page.
The error message you posted does not match the code situation.
Please clarify.

@aritchie-usgs
Copy link
Author

I'm not sure what you mean by "The error message you posted does not match the code situation." - The error I posted is exactly what the code spits out. I've been using lastools with no problems with the pcs.csv or vertcs.csv for like a dozen years. Lascopcindex64 does not find the files like las2las or any other lastool does. I'm sorry if I didn't explain that clearly enough above.

I'ave included output from commands I just ran on a little test file, and I attached the file for you to play with too:

lowest_dense_clip.zip

Also I've run some comparisons converting with lascopcindex64, generating copc with metashape, and converting with QGIS/untwide/pdal - you can read those here

Here are some commands I issued:

lascopcindex64 -version
LAStools (by info@rapidlasso.de) version 240319

lascopcindex64 -i *.laz
WARNING: building a COPC file without gpstime.
ERROR: GeogGeodeticDatumGeoKey: look-up for 1116 not implemented
ERROR: cannot open 'pcs.csv' file. check for \LAStools\bin\serf\geo\pcs.csv file. download latest version.
ERROR: set_ProjectedCSTypeGeoKey: look-up for 6340 not implemented
ERROR: cannot open 'vertcs.csv' file. maybe your LAStools distribution has no .\LAStools\bin\serf\geo\vertcs.csv file. download the latest version at http://lastools.org/download/LAStools.zip
WARNING: cannot convert CRS from GeoTIFF to OGC WKT.

las2las -i lowest_dense_clip.laz -epsg 6340 -target_epsg 6339 -olaz -o tst.laz -

verbose
reading 176942 and writing all surviving points ...
total time: 0.62 sec. written 176942 surviving points to 'tst.laz'.

where lascopcindex64
C:\utils\LAStools\bin\lascopcindex64.exe

dir C:\utils\LAStools\bin\serf\geo
Volume in drive C has no label.
Volume Serial Number is 760A-71B9

Directory of C:\utils\LAStools\bin\serf\geo

10/07/2023 10:59 PM

.
10/07/2023 10:59 PM ..
10/07/2023 10:59 PM 93,901 gcs.csv
10/07/2023 10:59 PM 1,805 my_epsg.csv
10/07/2023 10:59 PM 421 my_seven.csv
01/22/2024 09:42 AM 882,385 pcs.csv
10/07/2023 10:59 PM 17,299 vertcs.csv

@BB-Heidelberg
Copy link
Collaborator

Thanks for your persistence and the sample. We could fix this issue with latest commit.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants