speed up minimum spanning tree step

[zktuong]

Is your feature request related to a problem?

currently it doesn't scale well when there's many nodes in a single clone. the main issue is the nx.from_pandas_adjacency step.

dandelion/dandelion/tools/_network.py

Lines 558 to 578 in a68de45

	def mst(mat: dict) -> Tree:
	"""
	Construct minimum spanning tree based on supplied matrix in dictionary.
	Parameters
	----------
	mat : dict
	Dictionary containing numpy ndarrays.
	Returns
	-------
	Tree
	Dandelion `Tree` object holding DataFrames of constructed minimum spanning trees.
	"""
	mst_tree = Tree()
	for c in mat:
	tmp = mat[c] + 1
	tmp[np.isnan(tmp)] = 0
	G = nx.from_pandas_adjacency(tmp)
	mst_tree[c] = nx.minimum_spanning_tree(G)
	return mst_tree

Describe the solution you'd like

can we create the graph in batches?

Describe alternatives you've considered

No response

Additional context

No response

[zktuong]

this part is also super slow.

dandelion/dandelion/tools/_network.py

Lines 451 to 454 in a68de45

	for i, row in edge_list_final.iterrows():
	edge_list_final.at[i, "weight"] = tmp_totaldist.loc[
	row["source"], row["target"]
	]