When estimate the light direction, place a highly reflective sphere next to the object.
Then a fixed camera is used to capture a sequence of images of changing the direction of light source (incoming light vector). When finding light vector, search for the point with the maximum intensity in the sphere firstly. Then, localize the point of mirror reflection - hightlight point. Since the geometry of sphere and viewing direction V are known. The light direction L of a certain image could be calculate through
In typical of photometric stereo, we assume directional light source and therefore all pixels in a given image have the same light direction.
Project description has already give the data link which contains some images and light vector.
(1) Build virtual light direction sphere - build a standard icosahedron. Then, perform subdivision on each face 4 times recursively and get the vertices coordinates. Assume the sample object locates at the center of the light direction sphere. In this model, assume the light source move above the sphere surface. When a light source acts along the line between light source and sphere center, find the nearest vetex and use its coordinate (surface normal N) as Lo. Since the sample is treated as fully diffuse object (lambertian), the outcoming light radiance Ie does not depend on viewing direction. It dose depend on direction of illumination, which could be expressed by the angle between light direction and normal direction of sample. Hence, the light radiance (brightness) Ie = k_d * cos(theata) * Ii, where Ii is the incoming light radiance.
Since light vector Li could represent light direction and N is surface normal vector, cos(theata) = N * Li after normalizing N and Li ( ||N|| = ||Li|| = 1). Hence, lambert's law could be expressed as following equation.
(2) Hence, get incoming light vector Li (clustered) of sphere's highlight in each image i in data set from lightvec file, which represent in (x,y,z). Normalize it before use (||Li|| = 1).
(3) Find the vertex, which has the nearest surface normal N (||N|| = 1) with Li. Then, let Lo = N.
(4) Interpolate the image Io at Lo by
Lo * Li' * Ii(x,y)
Io(x,y) = sum_{i | Li's NN == Lo} * -------------------------------------
sum_{i | Li's NN == Lo} Lo * Li'
Ii(x,y) is the intensity at pixel (x,y) of i_th image in the image dataset. Interpolate the image at Lo, where Lo is the resampled (uniform) light directions on a sphere. The above equation means that the resampled image is a weighted sum of the sampled images, where the weights are defined by the dot products in the equation.
(choose an image to cancel out the surface albedo by producing ratio images)
(1) Stack the resampled images into a space-time volume - This process has been done in Section 1.2 (4)
(2) Get pixel intensities of each images in the space-time sequence - It is still in one matrix (x,y,img_num).
(3) Rank pixel intensities - for each pixel location (x,y), sort the corresponding pixel intensities over time (the index of img_num). Then, the intensity rank at (x,y) is gotten. Then, the intensity rank at (x,y) is greater than the median and smaller than the given upper bound, it has a high probability to be free of shadows and highlight. In the paper, the lower bound is 70% and the upper bound is 90% of whole image.
- kiL: total number of pixels in image(i) satisfiy 0.7 * num_img < rank(i) in image(i)
- riL: mean rank among the pixels in image(i) that satisfy rank < 0.7 * num_img.
Denominator img: the image with maximum kL and rL < 0.9 * num_img Denominartor light: the light vector of denominator image.
(1) Get the light vectors exclude light vector of denominator image (Lrest).
(2) Get the image index exclude denominator image (Irest)
(3) Get the local initial normal by Single value decomposition (SVD) - last column of V matrix. Remember to keep all normal in the same direction (reverse normal with negative z)
From initial normal estimation, a set of normal vecotr of each pixel in one object is acquired. But this set of normal vector is massive, and it would show value jump in some pixels. To make result from initial normal estimation smoother, implement 5 times recursive subdivision of each face of an icosahedron. Get the vertice information in the sphere. |L|=5057 --> filter out light vectors with z>0, L is a discrete set of labels corresponding to different normal orientations. Map the norms from initial normal estimation to L, which only hve 5057 labels. Use graph cut to realize smooth surface process.
According to the given information in project description, use GCO 3.0 to construct graph. Transform 2D coordinate graph to 1D coordinate graph. Then, connect 4 neighbour points of one point when preparation.
Use GCO_expansion to calculate the min energy. Then use GCO_Getlabeling to get the label of segmentation result.
With refine normals of all pixels, calculate gradient or slant & tilt angle of each pixel. Then, reconstruct surface of all pixels to get reconstruction surface. Make use of function shapeletsurf(). The reconstruction surface of each example is shown in the top table.