Network creation splits arcs at each node

[anitagraser]

After wondering why the number of rows in the streets GeoDataFrame is different from the number of rows in the spaghetti network arcs, I noticed that the network arcs are split at each node of the input line geometries. Is this on purpose? I find it rather unexpected and it can significantly increase the number of arcs in the network.

>>> shp_path = libpysal.examples.get_path("streets.shp")
>>> gdf = geopandas.read_file(shp_path)
>>> len(gdf)
293
>>> ntw = spaghetti.Network(in_data=shp_path)
>>> en(spaghetti.element_as_gdf(ntw1, arcs=True))
303

Visualization with point markers at start and end nodes:

[jGaboardi]

Is this on purpose? I find it rather unexpected and it can significantly increase the number of arcs in the network.

Yes, this is on purpose and expected, and yes, it will nearly always increase the number of arcs in the network. In Network._extractnetwork() each line feature is iterated over and each (euclidean) vertex within each line feature is iterated over in order to build the network data structure. As I mentioned above, this is expected behavior, but I have also realized it is only clarified in the code/comments itself and not in the docstrings/API docs, which should be remedied.

Moreover, like #530, this procedure is a legacy of the implementation in the code design of pysal.network (see also the migration explanation here), the predecessor of pysal/spaghetti. PySAL itself came into being prior to the advent of the anaconda/conda ecosystem and until several years ago was nearly 100% non-reliant on outside spatial packages (shapely, geopandas, etc.) due to the complexity, and frequent breakages, of installing dependencies. When I took over as the lead dev for spaghetti I wanted to incorporate functionality from other spatial packages as optional, but also retain the character of the original code base (i.e. utilizing native PySAL geometries and operations where possible).

Over the last several years I have gone back and forth on whether to (1) rely only on PySAL where possible; (2) completely restructure the code base to rely on outside packages; or (3) implement a dual workflow whereby users can either use only native PySAL geometries and operations or take advantage of outside packages. Each choice does, of course, come with its own set of pros & cons. Many of these thoughts & ideas can be found in the following issues: #403, #373 , #195.

[anitagraser]

Thank you for the detailed explanations! The joy of legacy systems 😄

So the legacy behavior is to split the original linestrings at each vertex and - if I interpret _extractnetwork correctly - it is not designed to store a linestring / libpysal.cg.Chain with the arcs:

spaghetti/spaghetti/network.py

Lines 531 to 535 in 48210dc

	arc_vertices = sorted([vid, nvid])
	arc = tuple(arc_vertices)
	# record the euclidean arc within the network
	self.arcs.append(arc)

However, later, functions such as _snap_to_link do create chains:

spaghetti/spaghetti/network.py

Lines 1203 to 1211 in 48210dc

	for arc in self.arcs:
	# record the start and end of the arc
	head = self.vertex_coords[arc[0]]
	tail = self.vertex_coords[arc[1]]
	# create a pysal.cg.Chain object of the arc
	# and add it to the arcs enumerator
	arcs_.append(util._chain_constr(None, [head, tail]))

I understand the reluctance to restructure everything and rely on outside packages. Luckily in this case, it seems like that's not necessary because PySAL may already include what we need.

[jGaboardi]

Yeah, before the v2.0 release (whenever that will be) I would like to work out all the oddities and rough edges like these. For now I am going to close this issue and we can reopen if needed once I have the time to focus back on it. In the meantime, I hope to give #526 some attention this weekend.