Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Invalid reading of lax file on windows #50

Closed
Jean-Romain opened this issue Feb 12, 2021 · 15 comments
Closed

Invalid reading of lax file on windows #50

Jean-Romain opened this issue Feb 12, 2021 · 15 comments
Assignees
@Jean-Romain
Labels
Bug A bug in the package Windows Microsoft Windows specific issue

Comments

@Jean-Romain
Copy link
Collaborator

Jean-Romain commented Feb 12, 2021 •
edited
Loading

Based on r-lidar/lidR#408 and using the las + lax files reported there. This issue is only reproducible on Windows

Expected behaviour (tested on R 4.0.3, Linux, rlas 1.3.9)

las <- rlas::read.las('issue 408/157668.5_243478.3.las', filter = "-inside 157980 243480 158270 243770")
dim(las)
#> [1] 8143683      18

Observed behaviour (tested on R 4.0.3, Windows, rlas 1.3.9)

las <- rlas::read.las('issue 408/157668.5_243478.3.las', filter = "-inside 157980 243480 158270 243770")
dim(las)
#> [1] 30595    18

Oberservation:

  • 90% of the region of interest is missing. Look like only one quadrant was loaded
  • when deleting the lax files it works. This confirms it is an issue with the spatial index
  • creating a new index with rlas::writelax() does not help
  • creating a new index with lasindex does not help

One last thing to test on windows with LAStools (tested on Linux)

las2las -i 157668.5_243478.3.las -inside 157980 243480 158270 243770 -o clip.las
lasinfo clip.las
> extended number of point records: 8143683
@Jean-Romain Jean-Romain added Bug A bug in the package Windows Microsoft Windows specific issue labels Feb 12, 2021
@Jean-Romain Jean-Romain self-assigned this Feb 12, 2021
@Jean-Romain Jean-Romain mentioned this issue Feb 12, 2021
Closed
@adrienmichez
Copy link

adrienmichez commented Feb 12, 2021 •
edited by Jean-Romain
Loading

Here is the result of lasinfo:

lasinfo -i "C:\TMP\lidR_bug\transfer_85074_files_52cf1058\clip.las"
>  extended number of point records: 8143683

@Jean-Romain
Copy link
Collaborator Author

Jean-Romain commented Feb 12, 2021 •
edited
Loading

I tried with rlas 1.3.7, 1.3.9 and 1.4.0 either compiling in C++11 and C++14. They all failed to perform the query on Windows. So the bug actually exist for a long time. I can also reproduce with first returns only:

las2las -first_only -i 157668.5_243478.3.las -o test.las
lasindex -i test.las

Linux

las <- rlas::read.las('~/issue 408/test.las', filter = "-inside 157980 243480 158270 243770")
dim(las)
#> [1] 5292612      18

Windows

las <- rlas::read.las('~/issue 408/test.las', filter = "-inside 157980 243480 158270 243770")
dim(las)
#> [1] 686183     18

Rplot

@adrienmichez
Copy link

adrienmichez commented Feb 13, 2021 •
edited
Loading

Does that mean that we should not use spatial indexing in windows?? This has quite broad implications!

@Jean-Romain
Copy link
Collaborator Author

Have you ever encountered other issues on windows ? You are the first one to report an problem and so far we were only able to reproduce on this file. It is definitively a complex problem that needs much more investigation.

@adrienmichez
Copy link

Sure... But I don't want to manually check if something went wrong after using spatial indexing. I hope you will soon find how to fix this :-)

@Jean-Romain
Copy link
Collaborator Author

Bad new for today. I've updated rlas with the latest source code of LASlib and LASzip and it is still failing 😞

@Jean-Romain
Copy link
Collaborator Author

I made very little progress today. It seems that the coordinates of some points are badly read for unkown reasons. This starts at interval [72200247, 72517435]

Linux

> las <- rlas::read.las('~/Téléchargements/issue 50/157668.5_243478.3.las', filter = "-inside 157980 243480 158270 243770")
16 cells of 156900/242600 158500/244200 intersect rect 157980/243480 158270/243770
LASindex: used 16 cells of total 182
Interval start = 2411200, end = 2678575, full = 15060790, total = 16622934, cells = 16 
   Point (157933.73, 243496.61) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
   Point (157980.84, 243478.93) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
Interval start = 5084464, end = 5285031, full = 15060790, total = 16622934, cells = 16 
   Point (157929.04, 243443.84) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
   Point (157958.77, 243464.21) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
Interval start = 7596484, end = 7819087, full = 15060790, total = 16622934, cells = 16 
   Point (157915.35, 243425.31) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
   Point (157971.41, 243410.94) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
Interval start = 9948801, end = 9968545, full = 15060790, total = 16622934, cells = 16 
Interval start = 9975888, end = 10007926, full = 15060790, total = 16622934, cells = 16 
   Point (157902.24, 243378.48) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
Interval start = 10014607, end = 10045098, full = 15060790, total = 16622934, cells = 16 
Interval start = 10051854, end = 10081465, full = 15060790, total = 16622934, cells = 16 
Interval start = 10089090, end = 10118304, full = 15060790, total = 16622934, cells = 16 
   Point (157970.58, 243399.31) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
Interval start = 10126859, end = 10151486, full = 15060790, total = 16622934, cells = 16 
Interval start = 10161245, end = 10191604, full = 15060790, total = 16622934, cells = 16 
Interval start = 72200247, end = 72517435, full = 15060790, total = 16622934, cells = 16 
   Point (157915.09, 243775.11) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
   Point (157950.81, 243804.03) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
   Point (157977.76, 243773.96) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
Interval start = 74745515, end = 75102684, full = 15060790, total = 16622934, cells = 16 
   Point (157890.74, 243766.12) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
   Point (157946.23, 243765.06) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
   Point (157963.39, 243742.14) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
Interval start = 77246009, end = 77609879, full = 15060790, total = 16622934, cells = 16 
   Point (157910.84, 243733.35) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
   Point (157926.59, 243732.23) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
   Point (157963.89, 243737.67) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
Interval start = 80070354, end = 80392642, full = 15060790, total = 16622934, cells = 16 
   Point (157900.10, 243693.55) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
   Point (157944.15, 243688.27) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
   Point (157961.20, 243692.20) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
Interval start = 83094087, end = 83456284, full = 15060790, total = 16622934, cells = 16 
   Point (157902.93, 243668.63) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
   Point (157931.57, 243666.55) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
   Point (157969.85, 243664.62) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
Interval start = 86199040, end = 86575969, full = 15060790, total = 16622934, cells = 16 
   Point (157903.58, 243616.46) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
   Point (157925.78, 243621.48) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
   Point (157968.21, 243635.94) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
Interval start = 89298031, end = 89650155, full = 15060790, total = 16622934, cells = 16 
   Point (157907.23, 243580.75) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
   Point (157932.89, 243609.17) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
   Point (157970.16, 243581.41) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
Interval start = 92151662, end = 92501280, full = 15060790, total = 16622934, cells = 16 
   Point (157893.44, 243554.00) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
   Point (157939.32, 243559.21) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
   Point (157974.29, 243567.49) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
Interval start = 95015614, end = 95316926, full = 15060790, total = 16622934, cells = 16 
   Point (157912.87, 243520.20) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
   Point (157963.63, 243530.99) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
Interval start = 96375793, end = 97411041, full = 15060790, total = 16622934, cells = 16 
   Point (157944.13, 243512.19) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
   Point (158043.70, 243473.14) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
   Point (158137.91, 243469.89) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
   Point (158233.12, 243478.33) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
   Point (158283.49, 243481.59) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
Interval start = 97475634, end = 98317548, full = 15060790, total = 16622934, cells = 16 
   Point (158020.57, 243445.61) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
   Point (158057.78, 243446.28) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
   Point (158074.59, 243457.57) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
   Point (158147.17, 243442.03) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
   Point (158179.26, 243457.68) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
   Point (158203.35, 243442.32) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
   Point (158215.34, 243460.07) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
   Point (158254.61, 243464.20) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
   Point (158292.17, 243452.65) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
Interval start = 98391434, end = 99222479, full = 15060790, total = 16622934, cells = 16 
   Point (158058.71, 243429.41) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
   Point (158069.05, 243410.91) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
   Point (158093.19, 243415.99) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
   Point (158124.65, 243420.24) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
   Point (158158.73, 243438.15) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
   Point (158234.87, 243427.04) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
   Point (158265.06, 243427.81) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
   Point (158283.69, 243433.78) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
Interval start = 99279624, end = 99298414, full = 15060790, total = 16622934, cells = 16 
Interval start = 99306363, end = 99378500, full = 15060790, total = 16622934, cells = 16 
   Point (158001.77, 243384.69) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
Interval start = 99387501, end = 99426476, full = 15060790, total = 16622934, cells = 16 
Interval start = 99432657, end = 99474449, full = 15060790, total = 16622934, cells = 16 
   Point (158085.84, 243400.46) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
Interval start = 99482509, end = 99560218, full = 15060790, total = 16622934, cells = 16 
Interval start = 99566800, end = 99629829, full = 15060790, total = 16622934, cells = 16 
   Point (158116.74, 243399.11) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
Interval start = 99637222, end = 99693219, full = 15060790, total = 16622934, cells = 16 
   Point (158171.97, 243396.99) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
Interval start = 99699866, end = 99756676, full = 15060790, total = 16622934, cells = 16 
Interval start = 99764528, end = 99836639, full = 15060790, total = 16622934, cells = 16 
   Point (158207.29, 243397.96) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
Interval start = 99845578, end = 99922488, full = 15060790, total = 16622934, cells = 16 
   Point (158263.93, 243385.47) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
Interval start = 99931411, end = 100003373, full = 15060790, total = 16622934, cells = 16 
Interval start = 100012971, end = 100055900, full = 15060790, total = 16622934, cells = 16 
   Point (158283.96, 243383.53) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
Interval start = 118993732, end = 119007854, full = 15060790, total = 16622934, cells = 16 
Interval start = 119017700, end = 119063855, full = 15060790, total = 16622934, cells = 16 
Interval start = 119070113, end = 119071244, full = 15060790, total = 16622934, cells = 16 
Interval start = 119082891, end = 119138922, full = 15060790, total = 16622934, cells = 16 
   Point (158052.16, 243807.77) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
Interval start = 119145653, end = 119193799, full = 15060790, total = 16622934, cells = 16 
Interval start = 119202638, end = 119204102, full = 15060790, total = 16622934, cells = 16 
Interval start = 119219563, end = 119277431, full = 15060790, total = 16622934, cells = 16 
   Point (158100.80, 243806.44) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
Interval start = 119285670, end = 119287674, full = 15060790, total = 16622934, cells = 16 
Interval start = 119295790, end = 119295847, full = 15060790, total = 16622934, cells = 16 
Interval start = 119301997, end = 119363882, full = 15060790, total = 16622934, cells = 16 
   Point (158120.94, 243804.61) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
Interval start = 119372969, end = 119374288, full = 15060790, total = 16622934, cells = 16 
Interval start = 119388109, end = 119447150, full = 15060790, total = 16622934, cells = 16 
Interval start = 119455764, end = 119457245, full = 15060790, total = 16622934, cells = 16 
Interval start = 119465232, end = 119465318, full = 15060790, total = 16622934, cells = 16 
Interval start = 119471656, end = 119526017, full = 15060790, total = 16622934, cells = 16 
   Point (158183.29, 243807.22) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
Interval start = 119533834, end = 119535365, full = 15060790, total = 16622934, cells = 16 
Interval start = 119549555, end = 119597124, full = 15060790, total = 16622934, cells = 16 
Interval start = 119605842, end = 119607453, full = 15060790, total = 16622934, cells = 16 
Interval start = 119623613, end = 119689314, full = 15060790, total = 16622934, cells = 16 
   Point (158266.54, 243807.02) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
Interval start = 119698684, end = 119738796, full = 15060790, total = 16622934, cells = 16 
Interval start = 119811561, end = 120617070, full = 15060790, total = 16622934, cells = 16 
   Point (158009.29, 243785.15) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
   Point (158048.24, 243787.49) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
   Point (158114.88, 243795.65) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
   Point (158129.80, 243790.60) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
   Point (158153.09, 243774.73) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
   Point (158223.63, 243782.31) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
   Point (158264.93, 243786.68) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
   Point (158270.94, 243794.14) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
Interval start = 120699724, end = 121585718, full = 15060790, total = 16622934, cells = 16 
Interval start = 121659673, end = 122588799, full = 15060790, total = 16622934, cells = 16 
   Point (158284.51, 243732.69) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
Interval start = 122671044, end = 123647049, full = 15060790, total = 16622934, cells = 16 
   Point (158293.52, 243676.78) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
Interval start = 123741675, end = 124603093, full = 15060790, total = 16622934, cells = 16 
   Point (158271.68, 243675.86) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
Interval start = 124682798, end = 125518909, full = 15060790, total = 16622934, cells = 16 
   Point (158281.31, 243617.25) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
Interval start = 125594171, end = 126463883, full = 15060790, total = 16622934, cells = 16 
   Point (158300.39, 243591.28) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
Interval start = 126582491, end = 127436464, full = 15060790, total = 16622934, cells = 16 
   Point (158282.86, 243557.06) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
Interval start = 127553895, end = 128442590, full = 15060790, total = 16622934, cells = 16 
   Point (158276.12, 243527.84) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
Interval start = 128551794, end = 129319764, full = 15060790, total = 16622934, cells = 16 
   Point (158271.99, 243516.16) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
Read 16622934 points
Read 8143683 points in bbox

Windows

las <- rlas::read.las('issue 50/157668.5_243478.3.las', filter = "-inside 157980 243480 158270 243770")
16 cells of 156900/242600 158500/244200 intersect rect 157980/243480 158270/243770
LASindex: used 16 cells of total 182
Interval start = 2411200, end = 2678575, full = 15060790, total = 16622934, cells = 16 
   Point (157933.73, 243496.61) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
   Point (157980.84, 243478.93) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
Interval start = 5084464, end = 5285031, full = 15060790, total = 16622934, cells = 16 
   Point (157929.04, 243443.84) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
   Point (157958.77, 243464.21) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
Interval start = 7596484, end = 7819087, full = 15060790, total = 16622934, cells = 16 
   Point (157915.35, 243425.31) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
   Point (157971.41, 243410.94) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
Interval start = 9948801, end = 9968545, full = 15060790, total = 16622934, cells = 16 
Interval start = 9975888, end = 10007926, full = 15060790, total = 16622934, cells = 16 
   Point (157902.24, 243378.48) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
Interval start = 10014607, end = 10045098, full = 15060790, total = 16622934, cells = 16 
Interval start = 10051854, end = 10081465, full = 15060790, total = 16622934, cells = 16 
Interval start = 10089090, end = 10118304, full = 15060790, total = 16622934, cells = 16 
   Point (157970.58, 243399.31) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
Interval start = 10126859, end = 10151486, full = 15060790, total = 16622934, cells = 16 
Interval start = 10161245, end = 10191604, full = 15060790, total = 16622934, cells = 16 
Interval start = 72200247, end = 72517435, full = 15060790, total = 16622934, cells = 16 
   Point (-2380267.51, 6309209.15) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
   Point (-1201272.31, 7399820.30) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
   Point (-2325872.62, -5888351.17) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
Interval start = 74745515, end = 75102684, full = 15060790, total = 16622934, cells = 16 
   Point (168468.82, 9866426.47) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
   Point (655.53, 3829931.99) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
   Point (655.53, 21646.23) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
   Point (655.70, 9866426.47) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
Interval start = 77246009, end = 77609879, full = 15060790, total = 16622934, cells = 16 
   Point (4227240.09, 14629603.68) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
   Point (6127128.73, 13868075.36) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
   Point (8919617.69, 15140129.12) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
Interval start = 80070354, end = 80392642, full = 15060790, total = 16622934, cells = 16 
   Point (518389.77, -17634368.44) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
   Point (1372979.21, -5286788.79) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
   Point (-1310717.43, 4624370.70) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
   Point (492833.29, -540613.56) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
Interval start = 83094087, end = 83456284, full = 15060790, total = 16622934, cells = 16 
   Point (168468.81, 21647.50) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
   Point (655.86, 9866426.47) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
   Point (655.87, 3830604.08) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
Interval start = 86199040, end = 86575969, full = 15060790, total = 16622934, cells = 16 
   Point (15029373.28, -17507283.33) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
   Point (15360985.44, -19331805.57) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
   Point (14382532.96, -19356053.89) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
   Point (13392284.00, -17684885.89) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
Interval start = 89298031, end = 89650155, full = 15060790, total = 16622934, cells = 16 
   Point (-35876.71, 15251540.32) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
   Point (3604648.09, 13423741.29) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
   Point (-15603953.51, 11889543.51) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
   Point (-11607568.22, 12917147.99) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
Interval start = 92151662, end = 92501280, full = 15060790, total = 16622934, cells = 16 
   Point (33586.30, 643563.53) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
   Point (11549.12, -1310719.99) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
   Point (13433464.84, 593758.73) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
Interval start = 95015614, end = 95316926, full = 15060790, total = 16622934, cells = 16 
   Point (11016089.07, 157262.77) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
   Point (11016980.28, 157340.85) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
   Point (11016069.01, 157375.08) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
Interval start = 96375793, end = 97411041, full = 15060790, total = 16622934, cells = 16 
   Point (-2560491.51, -6112484.29) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
   Point (1160645.13, -19362610.87) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
   Point (-2599813.11, -16791575.49) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
   Point (1587281.93, -6605314.99) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
   Point (1207173.13, -17684889.27) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
   Point (-1420162.55, -2887365.62) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
   Point (1492910.09, 13322871.89) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
   Point (587205.13, -20036262.84) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
   Point (1150812.17, 10632560.97) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
   Point (1502087.69, -18508618.67) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
Interval start = 97475634, end = 98317548, full = 15060790, total = 16622934, cells = 16 
   Point (2695956.07, 1056442.89) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
   Point (13443874.04, 1429342.73) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
   Point (11568.80, -1310719.99) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
   Point (22063.44, -1528954.87) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
   Point (11595.84, 1355939.85) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
   Point (11734.12, -2654207.99) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
   Point (22029.36, -1528954.87) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
   Point (33821.57, 1350696.97) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
   Point (22447.99, -2641100.79) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
Interval start = 98391434, end = 99222479, full = 15060790, total = 16622934, cells = 16 
   Point (20934819.84, 655.69) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
   Point (-4086824.96, 168632.49) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
   Point (-14041743.36, 655.53) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
   Point (6913392.64, 655.53) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
   Point (13746831.36, 655.70) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
   Point (-16145448.96, 655.53) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
   Point (-14488698.88, 860.33) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
   Point (20101201.92, 168468.82) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
Interval start = 99279624, end = 99298414, full = 15060790, total = 16622934, cells = 16 
Interval start = 99306363, end = 99378500, full = 15060790, total = 16622934, cells = 16 
   Point (758909.45, -12392143.80) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
Interval start = 99387501, end = 99426476, full = 15060790, total = 16622934, cells = 16 
Interval start = 99432657, end = 99474449, full = 15060790, total = 16622934, cells = 16 
   Point (11712.88, 1396572.18) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
Interval start = 99482509, end = 99560218, full = 15060790, total = 16622934, cells = 16 
   Point (9199946.08, -14001107.33) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
Interval start = 99566800, end = 99629829, full = 15060790, total = 16622934, cells = 16 
Interval start = 99637222, end = 99693219, full = 15060790, total = 16622934, cells = 16 
   Point (-21318205.44, 655.70) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
Interval start = 99699866, end = 99756676, full = 15060790, total = 16622934, cells = 16 
Interval start = 99764528, end = 99836639, full = 15060790, total = 16622934, cells = 16 
   Point (-16852578.69, 18006671.36) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
Interval start = 99845578, end = 99922488, full = 15060790, total = 16622934, cells = 16 
   Point (2695881.76, -3122132.47) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
Interval start = 99931411, end = 100003373, full = 15060790, total = 16622934, cells = 16 
   Point (3830599.85, 14088782.95) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
Interval start = 100012971, end = 100055900, full = 15060790, total = 16622934, cells = 16 
Interval start = 118993732, end = 119007854, full = 15060790, total = 16622934, cells = 16 
Interval start = 119017700, end = 119063855, full = 15060790, total = 16622934, cells = 16 
   Point (-15929584.93, 11016089.07) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
Interval start = 119070113, end = 119071244, full = 15060790, total = 16622934, cells = 16 
Interval start = 119082891, end = 119138922, full = 15060790, total = 16622934, cells = 16 
Interval start = 119145653, end = 119193799, full = 15060790, total = 16622934, cells = 16 
   Point (9866418.13, -4865150.47) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
Interval start = 119202638, end = 119204102, full = 15060790, total = 16622934, cells = 16 
Interval start = 119219563, end = 119277431, full = 15060790, total = 16622934, cells = 16 
Interval start = 119285670, end = 119287674, full = 15060790, total = 16622934, cells = 16 
Interval start = 119295790, end = 119295847, full = 15060790, total = 16622934, cells = 16 
Interval start = 119301997, end = 119363882, full = 15060790, total = 16622934, cells = 16 
   Point (-10842768.67, 11016089.07) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
Interval start = 119372969, end = 119374288, full = 15060790, total = 16622934, cells = 16 
Interval start = 119388109, end = 119447150, full = 15060790, total = 16622934, cells = 16 
   Point (-11466665.82, 8976468.31) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
Interval start = 119455764, end = 119457245, full = 15060790, total = 16622934, cells = 16 
Interval start = 119465232, end = 119465318, full = 15060790, total = 16622934, cells = 16 
Interval start = 119471656, end = 119526017, full = 15060790, total = 16622934, cells = 16 
Interval start = 119533834, end = 119535365, full = 15060790, total = 16622934, cells = 16 
Interval start = 119549555, end = 119597124, full = 15060790, total = 16622934, cells = 16 
   Point (-15781065.09, -12244746.24) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
Interval start = 119605842, end = 119607453, full = 15060790, total = 16622934, cells = 16 
Interval start = 119623613, end = 119689314, full = 15060790, total = 16622934, cells = 16 
Interval start = 119698684, end = 119738796, full = 15060790, total = 16622934, cells = 16 
   Point (-14463791.49, -1670512.64) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
Interval start = 119811561, end = 120617070, full = 15060790, total = 16622934, cells = 16 
   Point (11016068.95, 157250.03) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
   Point (11016089.07, 157278.84) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
   Point (11016069.01, 157337.89) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
   Point (11016089.07, 157369.76) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
   Point (11016069.01, 157420.53) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
   Point (11016980.28, 157485.45) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
   Point (11016068.93, 157519.46) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
   Point (11016980.28, 157605.27) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
Interval start = 120699724, end = 121585718, full = 15060790, total = 16622934, cells = 16 
   Point (481.65, 13444219.58) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
   Point (455.67, 2706262.53) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
   Point (439.80, 22655.48) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
   Point (441.52, 11579.36) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
   Point (424.62, 32258.75) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
   Point (414.32, 32429.89) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
   Point (407.12, 11685.62) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
   Point (419.09, 22497.53) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
   Point (421.03, 2695825.20) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
Interval start = 121659673, end = 122588799, full = 15060790, total = 16622934, cells = 16 
   Point (11016068.95, 157964.74) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
   Point (11016068.95, 157010.69) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
   Point (11016068.93, 157064.76) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
   Point (11016089.14, 157082.58) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
   Point (11016069.01, 157153.12) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
   Point (11016089.07, 157165.66) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
   Point (11016089.14, 157200.58) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
   Point (11016089.14, 157257.25) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
   Point (11016980.28, 157321.51) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
Interval start = 122671044, end = 123647049, full = 15060790, total = 16622934, cells = 16 
   Point (243239.13, 470.50) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
   Point (243245.67, 488.38) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
   Point (243237.20, 484.80) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
   Point (243234.74, 483.78) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
   Point (243241.93, 492.92) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
   Point (243226.58, 493.71) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
   Point (243241.43, 490.14) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
   Point (243246.52, 485.97) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
   Point (243221.22, 480.05) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
   Point (243224.10, 478.03) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
Interval start = 123741675, end = 124603093, full = 15060790, total = 16622934, cells = 16 
   Point (5864819.04, -12460355.97) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
   Point (5727193.44, -12974158.21) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
   Point (6107302.24, -14101377.41) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
   Point (5580392.80, -14575858.05) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
   Point (5702945.12, -13981446.53) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
   Point (5134092.64, -14419227.01) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
   Point (6948129.12, -15864951.17) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
   Point (5211425.13, -19396030.85) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
Interval start = 124682798, end = 125518909, full = 15060790, total = 16622934, cells = 16 
   Point (386.09, 2717328.97) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
   Point (388.53, 22779.85) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
   Point (412.70, 11617.89) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
   Point (425.16, 22381.07) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
   Point (433.45, 2706840.56) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
   Point (440.23, 11344.54) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
   Point (447.99, 2706217.43) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
   Point (450.03, 22372.94) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
   Point (464.59, 13433529.37) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
Interval start = 125594171, end = 126463883, full = 15060790, total = 16622934, cells = 16 
   Point (4832627.04, -11932135.81) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
   Point (4831316.32, -10930090.37) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
   Point (3058567.52, -12240155.01) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
   Point (3835169.12, -11726352.77) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
   Point (3500935.52, -11433406.85) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
   Point (4495116.64, -11085410.69) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
   Point (4559997.28, -11906576.77) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
   Point (4137290.08, -12259815.81) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
Interval start = 126582491, end = 127436464, full = 15060790, total = 16622934, cells = 16 
   Point (2706243.31, -2867197.43) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
   Point (2718010.95, -2896033.27) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
   Point (2696038.14, 465963.53) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
   Point (2695997.84, 477104.65) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
   Point (2707101.99, 549849.61) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
   Point (11673.97, 486932.49) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
   Point (11617.03, 549191.69) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
   Point (11556.08, -1966079.99) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
   Point (13433407.52, 1048578.57) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
Interval start = 127553895, end = 128442590, full = 15060790, total = 16622934, cells = 16 
   Point (860.33, 3829880.53) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
   Point (168632.49, 3830590.45) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
   Point (168632.66, 3829942.80) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
   Point (168468.97, 21632.98) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
   Point (168468.99, 9866413.13) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
   Point (168632.49, 3829943.03) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
   Point (655.69, 21633.28) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
   Point (168468.98, 21633.08) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
   Point (655.53, 9866411.47) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
Interval start = 128551794, end = 129319764, full = 15060790, total = 16622934, cells = 16 
   Point (1606945.12, -13190427.01) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
   Point (997460.32, -12993819.01) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
   Point (1522403.68, -13455192.45) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
   Point (2670594.40, -13392933.25) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
   Point (2305558.88, -12499677.57) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
   Point (2425489.76, -11671302.53) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
   Point (2627340.64, -11566444.93) not in rect [157980.00, 243480.00, 158270.00, 243770.00]
Read 16622934 points
Read 30595 points in bbox

@adrienmichez
Copy link

Hi,

I would like to use spatial indexing again but I'm afraid to experience this bug again and not see it.

Is there a way to test if the bug occurred in my process?

@Jean-Romain
Copy link
Collaborator Author

Jean-Romain commented Mar 2, 2021 •
edited
Loading

I'm closer and closer to the bug. I reached very deep lines of code but I'm not yet sure where does it come from. I cannot say anything about this bug expect that I'm sure the lax files is properly read. What happens is that after a certain number of file cursor moves it seems that there is a lag of few bits between where the cursor should be and where it actually is. It is windows specific and never reproduced with another dataset. Your dataset is very big and this does not help to figure out the issue.

It is not reproducible in all OS, not reproducible in other dataset, not reproducible with LAStools, not alway reproducible with another spatial query. My best guess so far is that it is specific to the c++ compiler used by R on windows but cannot explain why only with this dataset.

But I'm still investigating regularly getting closer and closer to the problem.

@Jean-Romain
Copy link
Collaborator Author

Good new I finally identified the issue 👍 . Now let try to fix it.

@adrienmichez
Copy link

Wow... Great news!

@Jean-Romain
Copy link
Collaborator Author

Jean-Romain commented Mar 2, 2021 •
edited
Loading

I fixed it 🎉.

That was a big issue actually and I'm surprised nobody reported it before. I'm tagging @bi0m3trics wondering if you already encountered such problem.

To make it simple the problem came from the fact that your file was more than ~2GB. This is not expected to be a problem to read such big file in the regular case. However when using a lax file, the reader can jump to different parts of the files. If the file is bigger than ~2GB it may try to jump to positions bigger than 2147483647. The problem was that the code badly handled the case when compiled with mingw which is the compiler used by R on windows. It assigned a method to read the file that cannot handle more than 2GB. The points that were missing in our query were those in the file after 2GB.

@Jean-Romain Jean-Romain reopened this Mar 2, 2021
@Jean-Romain Jean-Romain mentioned this issue Mar 2, 2021
Closed
@adrienmichez
Copy link

Great
Thanks for the investigation!

@Jean-Romain
Copy link
Collaborator Author

Thank you for reporting. Please try rlas 1.4.0 on your side and give me some feed back. I fixed the issue in fea7e62 but I will also made a full update of LASlib in rlas 1.4.0. So please stay informed. The current version 1.4.0 will be outdated in few hours/days

@adrienmichez
Copy link

adrienmichez commented Mar 4, 2021 •
edited
Loading

I just tested... It's now working using lidR and rlas from github 🎉

Jean-Romain added a commit that referenced this issue Mar 17, 2021
@Jean-Romain
#50
e24343c
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@Jean-Romain Jean-Romain

Labels
Bug A bug in the package Windows Microsoft Windows specific issue
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@Jean-Romain @adrienmichez