[BUG] Converting from sparse array to dense array right after computation on the coordinates

[SebastianHambura]

I think there is a synchronization or timing issue when trying to convert a sparse array (COO) into a dense array.

Description

While working on a function that takes a list of coordinates and creates a 3d volumes (and then is supposed to do a bunch of FFT and other things), arrayfire behaved weirdly.
Because the sparse array creation function can only create a 2D array, I created my 3d volumes by extending the array along dimension 1. I can then convert it into a dense array, and reshape it into the expected 3D array. To do this, I need to change the column coordinates appropriately, just before calling the af::sparse function.

In essence :

array af_cols = array(N_POINTS, dim[2], cols_raw, afHost);
array af_rows = array(N_POINTS, dim[2], rows_raw, afHost);
array af_vals = array(N_POINTS, dim[2], vals_raw, afHost);

for (int ii = 0; ii < dim[2]; ii++) {
  af_cols(span, ii) = ii * dim[1] + af_cols(span, ii);
}
array sparse = af::sparse(dim[0], dim[2] * dim[1],
   flat(af_vals), flat(af_rows), flat(af_cols), AF_STORAGE_COO);
array dense = af::dense(sparse);

// reshape dense into 3d
// or retrieve a slice with something like dense(af::span, seq(index * dim[1], (index + 1) * dim[1] - 1, 1))

If running on CPU backend, it works fine.
If running on CUDA backend, it might or might not run fine, depending on the dimension of the array and the number of points.
( However, if I convert the sparse storage representation then it works fine again : sparse = af::sparseConvertTo(sparse, af::storage::AF_STORAGE_CSR);. ) When it is not running fine, I only get values in the first slices, and the later slices are completly 0. My guess is that the computation for the new columns coordinate is not completely done, and so it uses the old values, where they all point to the first slice.

Reproducible Code and/or Steps

Reproducible Code

void af_bug() {
	using namespace af;
	setBackend(af::Backend::AF_BACKEND_CUDA);
#define N_POINTS 50

	// Size of the array
	const int dim[] = { 250, 250, 60 };

	int cols[N_POINTS];
	int rows[N_POINTS];
	float vals[N_POINTS];
	for (int n = 0; n < N_POINTS; n++) {
		cols[n] = 100 + n;
		rows[n] = 100;
		vals[n] = 10;
	}

	int total_n_points = N_POINTS * dim[2];
	int* cols_raw = new int[total_n_points];
	int* rows_raw = new int[total_n_points];
	float* vals_raw = new float[total_n_points];

	for (int n = 0; n < dim[2]; n++) {
		for (int i = 0; i < N_POINTS; i++) {
			int index = n * N_POINTS + i;
			cols_raw[index] = cols[i];
			rows_raw[index] = rows[i];
			vals_raw[index] = vals[i];
		}
	}

	array af_cols = array(N_POINTS, dim[2], cols_raw, afHost);
	array af_rows = array(N_POINTS, dim[2], rows_raw, afHost);
	array af_vals = array(N_POINTS, dim[2], vals_raw, afHost);
	//af_vals = complex(af_vals);


	for (int ii = 0; ii < dim[2]; ii++) {
		af_cols(span, ii) = ii * dim[1] + af_cols(span, ii);
	}

	array sparse = af::sparse(dim[0], dim[2] * dim[1],
		flat(af_vals), flat(af_rows), flat(af_cols), AF_STORAGE_COO);
	//af_print(sparse);

	//sparse = af::sparseConvertTo(sparse, af::storage::AF_STORAGE_CSR);
	array dense = af::dense(sparse);
	//af_print(dense);

	// Keep the window open until the user closes it
	int index = 0;
	af::Window window("Flattened Curve");
	do {
		af::array slicedArray = abs(dense(af::span, seq(index * dim[1], (index + 1) * dim[1] - 1, 1)));
		window.image(slicedArray); 
		char c[80];
		sprintf(c, "Slice # %d", index);
		window.setTitle(c);
		window.image(slicedArray);  
		Sleep(10);

		index++;
		index = index % dim[2];
	} while (!window.close());

	delete[] cols_raw;
	delete[] rows_raw;
	delete[] vals_raw;
	delete[] coords;
#undef N_POINTS
}

With the previous code, you should see a white line on every frame. However, with the GPU backend you only get a white line for the first couple of slices. With the CPU Backend, you indeed get a white line for every slice.

With GPU Backend

With CPU Backend