Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

CARS supports PleiadesNeo stereo imagery? #13

Closed
lionlai1989 opened this issue Oct 13, 2022 · 4 comments
Closed

CARS supports PleiadesNeo stereo imagery? #13

lionlai1989 opened this issue Oct 13, 2022 · 4 comments

Comments

@lionlai1989
Copy link

lionlai1989 commented Oct 13, 2022 •
edited

Hi team of CARS:
I've been testing CARS with PNeo (PleiadesNeo) data lately and found that there exists few issues.

  1. Program breaks with the error IndexError: index -1 is out of bounds for axis 0 with size 0:
22-10-13 10:33:39 :: WARNING :: Could not import pygraphviz backend. Will try graphviz backend next
WARNING ossimRpcModel::lineSampleHeightToWorld: 
NaN detected computing RPC polynomials. Results are invalid.
WARNING ossimRpcModel::lineSampleHeightToWorld: 
NaN detected computing RPC polynomials. Results are invalid.
WARNING ossimRpcModel::lineSampleHeightToWorld: 
NaN detected computing RPC polynomials. Results are invalid.
WARNING ossimRpcModel::lineSampleHeightToWorld: 
NaN detected computing RPC polynomials. Results are invalid.
Computing epipolar grids ...: 100% [**************************************************]
/cars/venv/lib/python3.8/site-packages/cars/core/projection.py:218: RuntimeWarning: invalid value encountered in cos
  cos_lat0 = np.cos(np.radians(lat0))
/cars/venv/lib/python3.8/site-packages/cars/core/projection.py:219: RuntimeWarning: invalid value encountered in sin
  sin_lat0 = np.sin(np.radians(lat0))
/cars/venv/lib/python3.8/site-packages/cars/core/projection.py:221: RuntimeWarning: invalid value encountered in cos
  cos_long0 = np.cos(np.radians(lon0))
/cars/venv/lib/python3.8/site-packages/cars/core/projection.py:222: RuntimeWarning: invalid value encountered in sin
  sin_long0 = np.sin(np.radians(lon0))
/cars/venv/lib/python3.8/site-packages/cars/core/projection.py:227: RuntimeWarning: invalid value encountered in double_scalars
  x_east = (-(x_ecef - x0_ecef) * sin_long0) + (
/cars/venv/lib/python3.8/site-packages/cars/core/projection.py:228: RuntimeWarning: invalid value encountered in double_scalars
  (y_ecef - y0_ecef) * cos_long0
/cars/venv/lib/python3.8/site-packages/cars/core/projection.py:231: RuntimeWarning: invalid value encountered in double_scalars
  (-cos_long0 * sin_lat0 * (x_ecef - x0_ecef))
/cars/venv/lib/python3.8/site-packages/cars/core/projection.py:232: RuntimeWarning: invalid value encountered in double_scalars
  - (sin_lat0 * sin_long0 * (y_ecef - y0_ecef))
/cars/venv/lib/python3.8/site-packages/cars/core/projection.py:233: RuntimeWarning: invalid value encountered in double_scalars
  + (cos_lat0 * (z_ecef - z0_ecef))
/cars/venv/lib/python3.8/site-packages/cars/core/projection.py:236: RuntimeWarning: invalid value encountered in double_scalars
  (cos_lat0 * cos_long0 * (x_ecef - x0_ecef))
/cars/venv/lib/python3.8/site-packages/cars/core/projection.py:237: RuntimeWarning: invalid value encountered in double_scalars
  + (cos_lat0 * sin_long0 * (y_ecef - y0_ecef))
/cars/venv/lib/python3.8/site-packages/cars/core/projection.py:238: RuntimeWarning: invalid value encountered in double_scalars
  + (sin_lat0 * (z_ecef - z0_ecef))
Processing Tiles : [  ] ...: 0it [00:00, ?it/s]
22-10-13 10:33:40 :: ERROR :: CARS terminated with following error
Traceback (most recent call last):
  File "/cars/venv/lib/python3.8/site-packages/cars/cars.py", line 172, in main_cli
    used_pipeline.run()
  File "/cars/venv/lib/python3.8/site-packages/cars/pipelines/sensor_to_full_resolution_dsm/sensor_to_full_resolution_dsm_pipeline.py", line 296, in run
    ) = self.resampling_application.run(
  File "/cars/venv/lib/python3.8/site-packages/cars/applications/resampling/bicubic_resampling.py", line 363, in run
    format_transformation.grid_margins_2_overlaps(
  File "/cars/venv/lib/python3.8/site-packages/cars/data_structures/format_transformation.py", line 118, in grid_margins_2_overlaps
    row_max = grid[-1, 0, 1]
IndexError: index -1 is out of bounds for axis 0 with size 0

My guts feeling tells me that the warnings and errors here are related to the goemodel file (RPC*.XML). Thus, I compare the PNeo's RPC file with Pleiades RPC file and find that tags of PNeo and Pleiades are not exactly the same. Here is the mapping:

GroundtoImage_Values --- Inverse_Model
ImagetoGround_Values --- Direct_Model
LON_DEN_COEFF_ --- SAMP_DEN_COEFF_
LON_NUM_COEFF_ --- SAMP_NUM_COEFF_
LAT_DEN_COEFF_ --- LINE_DEN_COEFF_
LAT_NUM_COEFF_ --- LINE_NUM_COEFF_

So, I manually modified the RPC file and make sure that tags of PNeo looks exactly the same as Pleiades'. And rerun the pipeline.

  1. CARS is running eternally, and no changes in output folder.
    In 1., I described what modification is made and rerun the pipeline. Here is the output:
22-10-11 13:15:31 :: WARNING :: Could not import pygraphviz backend. Will try graphviz backend next
Computing epipolar grids ...: 100% [**************************************************] (4s)


Processing Tiles : [ epi_matches_left ] ...:   0%|          | 0/1612 [00:00<?, ?it/s]ERROR 1: /data/2022-07-15/1558/2022-07-11/WO_000028295_1_11_SAL22023513-11_ACQ_PNEO4_01714303116360/000028295_1_11_STD_A/IMG_11_PNEO4_PAN/IMG_PNEO4_202207110043138_PAN_SEN_PWOI_000028295_1_11_F_1_P_R1C1.JP2:Not a TIFF or MDI file, bad magic number 0 (0x0)
Processing Tiles : [ epi_matches_left ] ...: 100%|██████████| 1612/1612 [2:05:42<00:00,  4.68s/it] 
ERROR 1: /data/2022-07-15/1558/2022-07-11/WO_000028295_1_11_SAL22023513-11_ACQ_PNEO4_01714303116360/000028295_1_11_STD_A/IMG_11_PNEO4_PAN/IMG_PNEO4_202207110043138_PAN_SEN_PWOI_000028295_1_11_F_1_P_R1C1.JP2:Not a TIFF or MDI file, bad magic number 0 (0x0)
...
WARNING:transitions.extensions.diagrams:Could not import pygraphviz backend. Will try graphviz backend next
WARNING:transitions.extensions.diagrams:Could not import pygraphviz backend. Will try graphviz backend next
WARNING:transitions.extensions.diagrams:Could not import pygraphviz backend. Will try graphviz backend next
WARNING:transitions.extensions.diagrams:Could not import pygraphviz backend. Will try graphviz backend next
  • At this point, a small AOI (observe 1612 tiles in the output above) has been running in a 16 CPUs machine for more than 24 hours, and I used htop to observe that the program was using only 1 CPU (but 8 CPUs are assigned in config.json).
  • Also at the same time, the size of the output folder doesn't change at all.
  • Thus, I decide to terminate the program and here is the output of 22-10-11_13h15m_sensor_to_full_res_dsm.log:
cat 22-10-11_13h15m_sensor_to_full_res_dsm.log
22-10-12 09:42:01 :: ERROR :: CARS terminated with following error
Traceback (most recent call last):
  File "/cars/venv/lib/python3.8/site-packages/cars/cars.py", line 172, in main_cli
    used_pipeline.run()
  File "/cars/venv/lib/python3.8/site-packages/cars/pipelines/sensor_to_full_resolution_dsm/sensor_to_full_resolution_dsm_pipeline.py", line 558, in run
    _ = self.rasterization_application.run(
  File "/cars/venv/lib/python3.8/site-packages/cars/orchestrator/orchestrator.py", line 314, in __exit__
    self.compute_futures()
  File "/cars/venv/lib/python3.8/site-packages/cars/orchestrator/orchestrator.py", line 253, in compute_futures
    for future_obj in self.cluster.future_iterator(future_objects):
  File "/cars/venv/lib/python3.8/site-packages/cars/orchestrator/cluster/multiprocessing_cluster.py", line 686, in __next__
    if item.ready():
KeyboardInterrupt
  1. An unusual thing about content.json. The resolution of PNeo data is 0.32 meter. In PNeo metadata file DIM*.XML,
			<Sampling_Settings>
				<RESAMPLING_SPACING unit="m">0.32</RESAMPLING_SPACING>
				<RESAMPLING_KERNEL>SPLINE</RESAMPLING_KERNEL>
			</Sampling_Settings>

But the CARS's output file content.json shows:

    "cloud_fusion_params": {
      "method": "mapping_to_terrain_tiles",
      "terrain_tile_size": null,
      "resolution": 0.5
    },
    "cloud_fusion_run": {
      "epsg": 32755,
      "margins": {
        "radius": 1.0,
        "resolution": 0.5
      },
    "rasterization_params": {
      "method": "simple_gaussian",
      "dsm_radius": 1.0,
      "sigma": null,
      "grid_points_division_factor": null,
      "resolution": 0.5
    },

It looks like the target resolution should be entered by users. How can I do it?

  1. The information of the data:
  • PNeo directory layout:
│       ├── IMG_10_PNEO4_MS-FS
│       │   ├── DIM_PNEO4_202207110043428_MS-FS_SEN_PWOI_000028295_1_10_F_1.XML
│       │   ├── IMG_PNEO4_202207110043428_MS-FS_SEN_PWOI_000028295_1_10_F_1_NED_R1C1.J2W
│       │   ├── IMG_PNEO4_202207110043428_MS-FS_SEN_PWOI_000028295_1_10_F_1_NED_R1C1.JP2
│       │   ├── IMG_PNEO4_202207110043428_MS-FS_SEN_PWOI_000028295_1_10_F_1_RGB_R1C1.J2W
│       │   ├── IMG_PNEO4_202207110043428_MS-FS_SEN_PWOI_000028295_1_10_F_1_RGB_R1C1.JP2
│       │   ├── INDEX.HTM
│       │   ├── LIBRARY
│       │   ├── LINEAGE
│       │   ├── LUT_PNEO4_202207110043428_MS-FS_SEN_PWOI_000028295_1_10_F_1_NED.XML
│       │   ├── LUT_PNEO4_202207110043428_MS-FS_SEN_PWOI_000028295_1_10_F_1_RGB.XML
│       │   ├── MASKS
│       │   ├── PREVIEW_PNEO4_202207110043428_MS-FS_SEN_PWOI_000028295_1_10_F_1.JPG
│       │   ├── PREVIEW_PNEO4_202207110043428_MS-FS_SEN_PWOI_000028295_1_10_F_1.KMZ
│       │   └── RPC_PNEO4_202207110043428_MS-FS_SEN_PWOI_000028295_1_10_F_1.XML
│       ├── IMG_10_PNEO4_PAN
│       │   ├── DIM_PNEO4_202207110043428_PAN_SEN_PWOI_000028295_1_10_F_1.XML
│       │   ├── IMG_PNEO4_202207110043428_PAN_SEN_PWOI_000028295_1_10_F_1_P_R1C1.J2W
│       │   ├── IMG_PNEO4_202207110043428_PAN_SEN_PWOI_000028295_1_10_F_1_P_R1C1.JP2
│       │   ├── INDEX.HTM
│       │   ├── LIBRARY
│       │   ├── LINEAGE
│       │   ├── LUT_PNEO4_202207110043428_PAN_SEN_PWOI_000028295_1_10_F_1_P.XML
│       │   ├── MASKS
│       │   ├── PREVIEW_PNEO4_202207110043428_PAN_SEN_PWOI_000028295_1_10_F_1.JPG
│       │   ├── PREVIEW_PNEO4_202207110043428_PAN_SEN_PWOI_000028295_1_10_F_1.KMZ
│       │   └── RPC_PNEO4_202207110043428_PAN_SEN_PWOI_000028295_1_10_F_1_P.XML
│       ├── INDEX.HTM
│       ├── LIBRARY
│       │   ├── VOL_LOGO.JPG
│       │   └── VOL_STYLE.XSL
│       └── VOL_PNEO.XML
  • config.json:
{
    "inputs": {
        "sensors": {
            "one": {
                "image": "/data/WO_000028295_1_11_SAL22023513-11_ACQ_PNEO4_01714303116360/000028295_1_11_STD_A/IMG_11_PNEO4_PAN/IMG_PNEO4_202207110043138_PAN_SEN_PWOI_000028295_1_11_F_1_P_R1C1.JP2",
                "geomodel": "/data/WO_000028295_1_11_SAL22023513-11_ACQ_PNEO4_01714303116360/000028295_1_11_STD_A/IMG_11_PNEO4_PAN/RPC_PNEO4_202207110043138_PAN_SEN_PWOI_000028295_1_11_F_1_P.XML",
                "no_data": 0
            },
            "two": {
                "image": "/data/WO_000028295_1_10_SAL22023513-10_ACQ_PNEO4_01714303116361/000028295_1_10_STD_A/IMG_10_PNEO4_PAN/IMG_PNEO4_202207110043428_PAN_SEN_PWOI_000028295_1_10_F_1_P_R1C1.JP2",
                "geomodel": "/data/WO_000028295_1_10_SAL22023513-10_ACQ_PNEO4_01714303116361/000028295_1_10_STD_A/IMG_10_PNEO4_PAN/RPC_PNEO4_202207110043428_PAN_SEN_PWOI_000028295_1_10_F_1_P.XML",
                "no_data": 0
            }
        },
        "pairing": [
            [
                "one",
                "two"
            ]
        ],
        "initial_elevation": "/data/2022-07-15_srtm"
    },
    "output": {
        "out_dir": "/output_dir"
    },
    "orchestrator": {
        "mode": "mp",
        "nb_workers": 8
    }
}

It's a bit long. Thank you for your patience. 😄
@lionlai1989
Copy link
Author

Another note:
This feature request in OTB gitlab page shows that OTB currently doesn't support PleiadesNeo data. Could it be the issue? However, I still think that the difference between Pleiades and PNeo data is the metadata. If the metadata is modified to be exactly the same, CARS and OTB should be able to run it.

@dyoussef
Copy link
Member

dyoussef commented Oct 20, 2022 •
edited

Hello @lionlai1989,

You are right : OTB currently doesn't not support Pleiades Neo Data so CARS too.

In order to work around the problem, you must :

  • Modify these keys in each RPC :
sed -i 's/GroundtoImage_Values/Inverse_Model/g' $RPC
sed -i 's/ImagetoGround_Values/Direct_Model/g' $RPC
sed -i 's/GroundtoImage/Inverse_Model/g' $RPC
sed -i 's/ImagetoGround/Direct_Model/g' $RPC
  • Point to DIMAP to help OTB to make the link between images and RPC :
{
    "inputs": {
        "sensors": {
            "one": {
                "image": "/data/WO_000028295_1_11_SAL22023513-11_ACQ_PNEO4_01714303116360/000028295_1_11_STD_A/IMG_11_PNEO4_PAN/DIM_PNEO4_202207110043138_PAN_SEN_PWOI_000028295_1_11_F_1.XML",
                "geomodel": "/data/WO_000028295_1_11_SAL22023513-11_ACQ_PNEO4_01714303116360/000028295_1_11_STD_A/IMG_11_PNEO4_PAN/RPC_PNEO4_202207110043138_PAN_SEN_PWOI_000028295_1_11_F_1_P.XML",
                "no_data": 0
            },
            "two": {
                "image": "/data/WO_000028295_1_10_SAL22023513-10_ACQ_PNEO4_01714303116361/000028295_1_10_STD_A/IMG_10_PNEO4_PAN/DIM_PNEO4_202207110043428_PAN_SEN_PWOI_000028295_1_10_F_1.XML",
                "geomodel": "/data/WO_000028295_1_10_SAL22023513-10_ACQ_PNEO4_01714303116361/000028295_1_10_STD_A/IMG_10_PNEO4_PAN/RPC_PNEO4_202207110043428_PAN_SEN_PWOI_000028295_1_10_F_1_P.XML",
                "no_data": 0
            }
        },
        "pairing": [
            [
                "one",
                "two"
            ]
        ],
        "initial_elevation": "/data/2022-07-15_srtm"
    },
    "output": {
        "out_dir": "/output_dir"
    },
    "orchestrator": {
        "mode": "mp",
        "nb_workers": 8
    }
}

NB : To check it works once you have modified the RPC, you can try on a small data using otbcli_ExtractROI : https://cars.readthedocs.io/en/stable/faq.html#how-to-create-cars-compatible-roi-input-data-with-otb :

otbcli_ExtractROI -in /data/WO_000028295_1_11_SAL22023513-11_ACQ_PNEO4_01714303116360/000028295_1_11_STD_A/IMG_11_PNEO4_PAN/DIM_PNEO4_202207110043138_PAN_SEN_PWOI_000028295_1_11_F_1.XML -out img1_xt.tif uint16 -startx startx -starty starty -sizex sizex -sizey sizey

otbcli_ExtractROI -in /data/WO_000028295_1_10_SAL22023513-10_ACQ_PNEO4_01714303116361/000028295_1_10_STD_A/IMG_10_PNEO4_PAN/DIM_PNEO4_202207110043428_PAN_SEN_PWOI_000028295_1_10_F_1.XML -out img2_xt.tif uint16 -mode fit -mode.fit.im img1_xt.tif

Regards,
David

@dyoussef
Copy link
Member

dyoussef commented Jan 2, 2023

Hello @lionlai1989, have you tried with the recommendations ? Can we close the issue ?
Best regard,
David

@dyoussef dyoussef closed this as completed Jan 2, 2023
@dyoussef dyoussef reopened this Jan 2, 2023
@dyoussef
Copy link
Member

I close this issue : @lionlai1989 If you require any further information, please feel free to (re) open it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@lionlai1989 @dyoussef