about dataset

[gpqls]

Hi, I just have a question about DDRs.

In paper, you said "Projection parameters are chosen to evenly sample points on the surface of a sphere, specifically a slight horizontal elevation of 70-85 degrees and , for a full 360- degree vertical rotation. ".

It meas, https://drive.google.com/file/d/1msiXso-nVDqdKWwoD6v129lYjOx7IXy5/view?usp=sharing like this picture?

You generate DDR with that degree at every 5 vertical degree?

[abrilcf]

Hi,
The projection parameters described in that sentence refer to the camera matrices used by the model, not for generating the DRRs. For the DRRs, the elevation angle is fixed to 90 degrees. This was to consider slight variations in the detector of the different scans, and so the model is not too sensitive to those changes.

[gpqls]

What is camera matrix's mean? Didn't this mean the location of the camera?

[gpqls]

Hi,
I still wonder about 70-85 projection parameter. DRR data is fixed to 90 degrees, but how to use 70-85 degree when use model?
I can't understand about that.

And, in chest xray images, you use 20*72=1,440 images. About "20", that means 20 people chest x-ray?

[abrilcf]

Hi,
Sorry for the late reply.

1. A camera matrix describes the mapping from 3D to 2D. It does includes the camera location (camera extrinsics), but also the 3D-2D transformation (camera intrinsics). Here's a great explanation of these concepts
2. The 70-85 choice doesn't really affect results. You can fix it to 90. You can observe the effect by running the script
3. Exactly, 20 CT scans each from 20 patients.

[gpqls]

Thank you! I got it.

And,

1. Did you apply nerf to each of 20 people or did you apply to each of 20 people as one common nerf? Where can I find the code for this content?

2. Is it correct that you set train and valid as the same dataset in the code? If not, where can I find the code to check how the dataset was divided?

[abrilcf]

1. We train a single model for each category (chest and knee). So, one model for all of the instances.
2. Since the datasets were too small, we arbitrarily took out for testing: Knee - 1 instance, and for chest 4 instances. We arbitrarily picked the first instances. For the renderings of X-ray projections, we try with instances from this dataset as well for knee. If you substitute with an instance from the training dataset, then it would be like "searching" the closest learned instance given a single projection

[zcc861007]

Hi Abril,

According to your answer above, do you mean that you trained the GAN, simultaneously using X-ray sequences of 20 persons (e.g., for the chest)? If I have a X-ray sequence of only ONE person, can I train the GAN?

Also, in the code shown below, should N be equal to 72 instead of 40?

mednerf/graf-main/graf/config.py

Line 94 in 0d3e075

N = 40

Thanks!

[gpqls]

Aren't the four random datasets you brought already used for model training?

mednerf/graf-main/train.py

Lines 89 to 91 in 0d3e075

	val_dataset = train_dataset
	val_loader = train_loader
	hwfr_val = hwfr

I don't know where "those four instances" came from.

[abrilcf]

Hi C. Zhou,

Yes, you can train a model using a single instance. However, it may overfit to a point in which is hardly capable to render novel views for an unseen X-ray at inference time.
It looks like we missed that N value. One could think that would hurt results when testing the model but it may indicate it doesn't. As we do use N=72 at inference time.

[abrilcf]

Hi gpqls,
I meant that we left the first 4 instances of the chest dataset for validation.

[gpqls]

Hi, abrilcf. Thank you for answer.
Can you tell me the location of the code that left 4 instances?

[zcc861007]

Hi Abril. Thanks for your answers! I feel I got the point. Do you mean that there are 20 chest NeRFs for the 20 subjects during training? In such way, the GAN can be trained by more Xray-DRR pairs?

[abrilcf]

Hi gpqls,
That's manually done in the render_xray_G.py and render_xray_G_Z.py scripts. Now I understand the confusion with the val_dataset = train_dataset line, you see FID and KID metrics are used for evaluation. Both compare the distribution of real images (those used for training) and the learned distribution.

[abrilcf]

Hi zcc861007,
Yes, That would certainly help generalization

[gpqls]

Thanks to this, I found out that real images and learned distribution are compared and scored in FID and KID.