Repository navigation
TestF3G Results
| CompareOBJ RMS: | The root mean square of the distance from each bigmap pixel/line location to the nearest facet of the truth OBJ. |
|---|---|
| PTG: | The formal camera pointing uncertainty. |
| SCOBJ: | The S/C position vector from the center of the asteroid. |
| VSO: | The formal SCOBJ uncertainty. |
The Detailed Survey PolyCam F3G data set had large S/C position and pointing uncertainties, unsuitable for Detailed Survey Baseball Diamond trajectories:
- VSO = 1km;
- PTG = 1mrad. An additional suite of tests was therefore run with a duplicate Detailed Survey PolyCam dataset with S/C position and pointing uncertainties set to one-sigma:
- VSO = 6.4m;
- PTG = 0.217mrad.
Results for both data sets are presented herein.
In the process of generating the 35cm Preliminary Shape Model, a shift in the body center with respect to the inertial center was applied. The magnitude of the shift was approximately 2m. Since the final S/C position across F3G subtests lay in a region 2m to 8m from the true S/C position, the effect of the body center shift came into question. To investigate, an alternative 35cm Preliminary Shape Model with no body-center shift was generated. One subtest was re-run using the alternative start model.
CompareOBJ RMSs (with and without translation/rotation) show no trend with the magnitude of S/C position and camera pointing perturbation within the perturbation ranges tested. The correlation score also does not vary with the magnitude of S/C position and camera pointing perturbation and is approx. 0.76 giving a very good indication of correlation. The accuracy of the evaluation model is therefore invariant to the S/C position or camera pointing perturbation up to three standard deviations.
The results show no significant difference in the final S/C position (SCOBJ) with respect to the magnitude of S/C position and camera pointing perturbation within the ranges tested, indicating that the SPC-driven S/C position modeling is immune to S/C position and pointing perturbation up to three standard deviations.
With the large S/C position and camera pointing uncertainties, the final SPC-derived S/C position is within 8m of the true S/C position, and in two cases is within 2m of the true S/C position. With the reduced (1 sigma) S/C position and camera pointing uncertainties, all the final SPC-derived S/C positions are within 2m of the true S/C position. The distance between the final S/C position and the true S/C position is not dependent on the initial perturbed S/C position - indeed, in the 0.25 x sigma case, the SPC-derived S/C position in most cases moves further away from the true S/C position (final distance from truth: 1.6m to 6.9m) than its initial position (initial distance from truth: 1.6m).
The final SPC-derived S/C positions appear to be clustered around an incorrect solution 2m distant from the true S/C position (in the VSO=1sigma tests). This distance appears to correspond with a shift in the center-of-body which was applied during the generation of the 35cm Preliminary Survey Start Model. However, this finding persisted when a subtest was re-run with a 35cm Preliminary Survey Start Model which had not had a shift applied to the center-of-body. Measures of both model accuracy and S/C position did not vary with center-of-body shift.
It should be noted that S/C position perturbation was divided equally between the SCOBJ components, resulting in a distance from the truth position which was a multiple of the standard deviation of 6.4m. Therefore:
- maximum lateral perturbation was a multiple of 3.7m (6.4m/sqrt(3));
- maximum normal perturbation (wrt body center) was a multiple of 3.7m (6.4m/sqrt(3)). It is assumed that the worst case scenario is a 3 x sigma (19.2m) lateral perturbation. The maximum possible lateral perturbation tested was 3 x 3.7m = 11.1m.
Three CompareOBJ RMS values for the final 5cm resolution 20m x 20m evaluation bigmap are presented for each subtest and each S/C position and camera pointing uncertainty:
- The largest CompareOBJ RMS (approx. 65cm across subtests) is obtained by running CompareOBJ on the untranslated and unrotated evaluation model.
- The second smallest CompareOBJ RMS (approx. 15cm across subtests) is obtained by running CompareOBJ with its optimal translation and rotation option.
- The smallest CompareOBJ RMS (approx. 9cm across subtests) is obtained by manually translating the evaluation model and searching for a local CompareOBJ RMS minimum.
The CompareOBJ optimal translation routine is not optimized for the evaluation model scale (5cm pix/line resolution). Manual translations of the bigmap were therefore conducted in an attempt to find a minimum CompareOBJ RMS. The manually translated evaluation models gave the smallest CompareOBJ RMSs.
CompareOBJ RMSs do not show a trend with the magnitude of S/C position and pointing perturbation within the ranges tested.
CompareOBJ RMSs differ slightly with S/C position and camera pointing uncertainties, with smaller RMS values corresponding with lower uncertainties:
- approx. 5cm difference for CompareOBJ RMS without translation/rotation;
- approx. 2cm difference for CompareOBJ RMS with optimal translation and rotation;
- approx. 0.4cm difference for CompareOBJ RMS with manual translation.
CompareOBJ Optimal Translations:
| Sub-Test | S/C Position Uncertainty | Perturbation Magnitude | RMS (cm) | Translation (cm) | Translated Distance (cm) | ||
|---|---|---|---|---|---|---|---|
| F3G7a | 1km | 0.25 x sigma | 15.7849 | 85.0698 | 62.3596 | -14.3765 | 106.4532 |
| F3G6a | 1km | 0.50 x sigma | 15.1147 | 84.5538 | 61.6624 | -15.3434 | 105.7687 |
| F3G5a | 1km | 0.75 x sigma | 14.3150 | 95.8438 | 59.9313 | -21.6901 | 115.1011 |
| F3G3a | 1km | 1.00 x sigma | 13.6399 | 106.4870 | 58.2162 | -27.3527 | 124.4057 |
| F3G4a | 1km | 1.50 x sigma | 15.8949 | 79.1224 | 63.2865 | -19.6432 | 103.2056 |
| F3G2a | 1km | 2.00 x sigma | 13.6938 | 110.1339 | 58.9454 | -23.2403 | 127.0596 |
| F3G1a | 1km | 3.00 x sigma | 14.6271 | 93.5937 | 61.6997 | -26.8422 | 115.2698 |
| Sub-Test | S/C Position Uncertainty | Perturbation Magnitude | RMS (cm) | Translation (cm) | Translated Distance (cm) | ||
|---|---|---|---|---|---|---|---|
| F3G7b | 6.4m | 0.25 x sigma | 15.9614 | 95.5555 | 66.8688 | -17.6793 | 117.9613 |
| F3G6b | 6.4m | 0.50 x sigma | 16.0260 | 93.7254 | 66.9687 | -17.2331 | 116.4743 |
| F3G5b | 6.4m | 0.75 x sigma | 16.4387 | 88.3626 | 69.0785 | -12.1628 | 112.8172 |
| F3G3b | 6.4m | 1.00 x sigma | 9.5449 | 167.5406 | 57.3951 | -27.0780 | 179.1571 |
| F3G4b | 6.4m | 1.50 x sigma | 16.0525 | 93.4737 | 70.0956 | -15.0998 | 117.808=1 |
| F3G2b | 6.4m | 2.00 x sigma | 16.1591 | 94.1139 | 69.1165 | -19.95051 | 118.4514 |
| F3G1b | 6.4m | 3.00 x sigma | 15.2938 | 101.8134 | 70.3467 | -20.3179 | 125.4091 |
CompareOBJ Manual Translations:
| Sub-Test | S/C Position Uncertainty | Perturbation Magnitude | RMS (cm) | Translation (cm) | Translated Distance (cm) | ||
|---|---|---|---|---|---|---|---|
| F3G7a | 1km | 0.25 x sigma | 8.0140 | 187.5 | 47.5 | -30 | 195.74 |
| F3G6a | 1km | 0.50 x sigma | 7.9343 | 187.5 | 47.5 | -30 | 195.74 |
| F3G5a | 1km | 0.75 x sigma | 7.9579 | 187.5 | 47.5 | -30 | 195.74 |
| F3G3a | 1km | 1.00 x sigma | 7.6749 | 187.5 | 47.5 | -30 | 195.74 |
| F3G4a | 1km | 1.50 x sigma | 7.9964 | 187.5 | 47.5 | -40 | 197.52 |
| F3G2a | 1km | 2.00 x sigma | 7.8591 | 187.5 | 47.5 | -40 | 197.52 |
| F3G1a | 1km | 3.00 x sigma | 7.6739 | 187.5 | 47.5 | -50 | 209.19 |
| Sub-Test | S/C Position Uncertainty | Perturbation Magnitude | RMS (cm) | Translation (cm) | Translated Distance (cm) | ||
|---|---|---|---|---|---|---|---|
| F3G7b | 6.4m | 0.25 x sigma | 8.4097 | 207.5 | 47.5 | -30 | 214.9709 |
| F3G6b | 6.4m | 0.50 x sigma | 8.3974 | 207.5 | 47.5 | -30 | 214.9709 |
| F3G5b | 6.4m | 0.75 x sigma | 8.5008 | 207.5 | 47.5 | -30 | 214.9709 |
| F3G3b | 6.4m | 1.00 x sigma | 8.2400 | 207.5 | 47.5 | -40 | 216.5929 |
| F3G4b | 6.4m | 1.50 x sigma | 8.5820 | 197.5 | 57.5 | -20 | 206.6700 |
| F3G2b | 6.4m | 2.00 x sigma | 8.4134 | 197.5 | 57.5 | -30 | 207.8762 |
| F3G1b | 6.4m | 3.00 x sigma | 7.9259 | 197.5 | 57.5 | -30 | 207.8762 |
The evaluation maps were compared with a truth map via a cross-correlation routine which derives a correlation score. As a guide the following scores show perfect and excellent correlations:
- A map cross-correlated with itself will give a correlation score of approx. 1.0;
- Different sized maps sampled from the same truth (for example a 1,100 x 1,100 5cm sample map and a 1,000 x 1,000 5cm sample map) give a correlation score of approx. 0.8.
The correlation scores show no trend with S/C position and camera pointing perturbation in the perturbation-range tested. The correlation scores also do not differ with S/C position and camera pointing uncertainties. The correlation score across subtests is approx. 0.76 giving a very good indication of correlation.
Correlation Scores:
| Sub-Test | S/C Position Uncertainty | Perturbation Magnitude | Correlation Score |
|---|---|---|---|
| F3G7a | 1km | 0.25 x sigma | 0.7619 |
| F3G6a | 1km | 0.50 x sigma | 0.7625 |
| F3G5a | 1km | 0.75 x sigma | 0.7617 |
| F3G3a | 1km | 1.00 x sigma | 0.7641 |
| F3G4a | 1km | 1.50 x sigma | 0.7684 |
| F3G2a | 1km | 2.00 x sigma | 0.7537 |
| F3G1a | 1km | 3.00 x sigma | 0.7679 |
| Sub-Test | S/C Position Uncertainty | Perturbation Magnitude | Correlation Score |
|---|---|---|---|
| F3G7b | 6.4m | 0.25 x sigma | 0.7628 |
| F3G6b | 6.4m | 0.50 x sigma | 0.7633 |
| F3G5b | 6.4m | 0.75 x sigma | 0.7637 |
| F3G3b | 6.4m | 1.00 x sigma | 0.7672 |
| F3G4b | 6.4m | 1.50 x sigma | 0.7606 |
| F3G2b | 6.4m | 2.00 x sigma | 0.7510 |
| F3G1b | 6.4m | 3.00 x sigma | 0.7683 |
The first graph shows footprints for all Detailed Survey PolyCam pictures which were included in the model. The second graph shows the four pictures down-selected for S/C position evaluation purposes due to their coverage of the 20m x 20m evaluation region, and their almost complete containment within the iterated 100m x 100m region.
The distance of the final SPC-derived S/C position from the true S/C position is plotted for the evaluation Detailed Survey PolyCam image set for each magnitude of perturbation. The distance of the final SCOBJ from the true S/C position is invariant with the perturbation to the initial SCOBJ within the perturbation-range tested. The range of distances across pictures does vary with S/C position uncertainty however, with:
- distances ranging from approx. 1m to 8m with VSO=1km;
- distances ranging from approx. 1m to 2m with VSO=6.4m (1 sigma).
3D graphs of final SPC-derived SCOBJ and true SCOBJ are plotted for each evaluation picture.
The pattern of final SPC-derived SCOBJ is broadly consistent across magnitudes of initial S/C position perturbation. The position correction is mostly a normal correction with lateral movement, bringing the modeled S/C position within approx. 2m to 8m of the true S/C position. Note that the SCOBJs form a cluster which is offset from the truth indicating that the model is converging to a solution different to the truth.
Example nominal SCOBJs:
** Final solution SCOBJs:**
3D graphs of final SPC-derived SCOBJ and true SCOBJ are re-plotted for each evaluation picture, now including the single result from the additional subtest conducted from a 35cm Preliminary Survey Start Model with no center-of-body shift. The graphs show that the final SCOBJ is not affected by center-of-body shift up to 2m. This result also dispels the notion that the distance of the final SCOBJs from truth is connected to the center-of-body shift.
