ASTAR: Path validity failing on multiple tries

[neoblizz]

Summary:
Path validity failing on multiple tries.

Reproducibility:
Sometimes

To Reproduce:
Run the following command multiple times (~10) to see the error:

cd examples/astar
make
/path/to/bin/astar market ../../dataset/small/map/cities.mtx --mapfile=../../dataset/small/map/cities.map

also the same case with --undirected which uses a different Reference. (may actually only show with undirected)

Expected Behavior:

 Path Validity: 
 CORRECT
 First 40 preds of the GPU result.
 P60->SNY->UMN->ANB->
 
 	Memory Usage(B)	 #keys0,0	 #keys0,1
 GPU_0	 12582912	 1424	 52255
 	 queue_sizing =	 1.201688 	 44.097046

Actual Results:

 Path Validity: 
 INCORRECT: [1]: 973 != 725
 result[...1184, 973, 709, 12, 0, ...]
 reference[...1184, 725, 662, 0, 44923568, ...]
 5 errors occurred.

full output:

mosama@mario:~/forks/neo/gunrock/examples/astar$ ./bin/test_astar_10.0_x86_64 market ../../dataset/small/map/cities.mtx --mapfile=../../dataset/small/map/cities.map --undirected --iterations=10
Loading Matrix-market coordinate-formatted graph ...
  Reading directly from stored binary CSR arrays from ../../dataset/small/map/.cities.mtx.ud.1.bin ...
Done reading (0s).

Degree Histogram (1185 vertices, 262486 edges):
    Degree   0: 0 (0.00%)
    Degree 2^0: 0 (0.00%)
    Degree 2^1: 1 (0.08%)
    Degree 2^2: 3 (0.25%)
    Degree 2^3: 21 (1.77%)
    Degree 2^4: 31 (2.62%)
    Degree 2^5: 59 (4.98%)
    Degree 2^6: 35 (2.95%)
    Degree 2^7: 599 (50.55%)
    Degree 2^8: 436 (36.79%)

Using 1 GPU: [ 0 ].
delta value:1000000.000000
Using traversal mode LB
__________________________
0        1       0       queue3          oversize :      1424 ->         98410
0        2       0       queue3          oversize :      98410 ->        157348
--------------------------
iteration 0 elapsed: 3.220797 ms, src = 0, #iteration = 14
Computing reference value ...

Path Validity:
INCORRECT: [1]: 712 != 725
result[...1184, 712, 662, 0, ...]
reference[...1184, 725, 662, 0, ...]
2 errors occurred.

First 40 preds of the GPU result.
P60->6V1->UMN->ANB->

        Memory Usage(B)  #keys0,0        #keys0,1
GPU_0    18874368        1424    157348
         queue_sizing =  1.201688        132.783122

 [ASTAR] finished.
 avg. elapsed: 3.2208 ms
 iterations: 14
 min. elapsed: 3.2208 ms
 max. elapsed: 3.2208 ms
 rate: 81.3945 MiEdges/s
 src: 0
 nodes_visited: 1184
 edges_visited: 262155
 nodes queued: 2271
 edges queued: 483778
 nodes redundance: 91.81%
 edges redundance: 84.54%
 load time: -1551326883953.2300 ms
 preprocess time: 8.4529 ms
 postprocess time: 8.4512 ms
 total time: 2285.6112 ms

Originally posted by @neoblizz in #499 (comment)

System Configuration:
Reproduce-able on mario.