-
Notifications
You must be signed in to change notification settings - Fork 84
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Oneway by Series #94
Comments
If I recall correctly all edges are one way by default and are directed from node a to node b. You can add whichever new edges you want which are directed from b to a. I'm not sure an API of booleans indicating which edges to flip adds much to the API. In other words, I think Pandana supports what you need, you just need to write code outside of Pandana to read from a shapefile and add additional edges as required. At the least you could start there and we could review the code to see if it makes sense to integrate. Thoughts? |
Hi @fscottfoti , In networkx, the edges are one-way by default. In pandana, your docs say twoway=True is the default. Is this correct? Another approach is I could select out the pandana edges that are oneway, and have their two-way option flagged as false. Then I would merge them with the other pandana edges. However, I am not seeing a way to merge pandana graphs. Is that correct? Generally, I think there is a good reason to add a convenience function for this purpose. I think it would follow the convention of other network analysis tools, and also respect typical typical municipal center line structures (divided one-way on access controlled roads, two-way on others). |
Yup, two way by default. It just seems to me like a handful of lines of Python/Pandas. You have an edges dataframe of A and B nodes and a mask of which edges are 2-way edges. You filter on that mask, reverse the edges and concat to the original dataframe. I'm fine with a convenience function, but it would just take the edges dataframe and mask and return the new edges datagrame - it wouldn't touch any of the rest of the code really. |
That is what I was thinking too in pandas. Do the edge flip then append myself and set twoway=False. I think this could be pretty isolated convenience function for pandana too. I will give it a shot and report back. Hopefully it is simple add so pandana can take a series alongside a simple bool. Maybe we just keep it outside of the network class (which looks like it would be more complicated to include). |
@fscottfoti I felt I should report back after the help I got here and from @sablanchard. Thanks again. Long story short, I gave up on using NetworkX, and found a way to construct a pandana graph from scratch just using geopandas. The notebook documenting this will be shared here in the future, but I felt should at least share the graph construction methods for future notice. If you think parts of this might be nice to include in the library, let me know. def get_nodes_and_edges(shp_file,rounding=5):
"""Use geopandas to read line shapefile and compile all paths and nodes in a line file based on a rounding tolerance.
shp_file:path to polyline file with end to end connectivity
rounding: tolerance parameter for coordinate precision"""
edges = gpd.read_file(shp_file)
edges["from_x"]=edges["geometry"].apply(lambda x:round(x.coords[0][0],rounding))
edges["from_y"]=edges["geometry"].apply(lambda x:round(x.coords[0][1],rounding))
edges["to_x"]=edges["geometry"].apply(lambda x:round(x.coords[-1][0],rounding))
edges["to_y"]=edges["geometry"].apply(lambda x:round(x.coords[-1][1],rounding))
nodes_from = edges[["from_x","from_y"]].rename(index=str,columns={"from_x":"x","from_y":"y"})
nodes_to = edges[["to_x","to_y"]].rename(index=str,columns={"to_x":"x","to_y":"y"})
nodes = pd.concat([nodes_from,nodes_to],axis=0)
nodes["xy"] = list(zip(nodes["x"], nodes["y"]))
nodes = pd.DataFrame(nodes["xy"].unique(),columns=["xy"])
nodes["x"] = nodes["xy"].apply(lambda x: x[0])
nodes["y"] = nodes["xy"].apply(lambda x: x[1])
nodes = nodes[["x","y"]].copy()
return nodes , edges
def generate_pandana_store_from_shp(hdf5_path,shp_file,weights=["weight"],oneway=None,overwrite_existing=True,rounding=6):
"""Generate a pandana ready HDF5 store using geopandas (gdal required) and pandas. Python 3.5.
hdf5_path(str): output path of HDF5 store holding two dataframes ["nodes","edges"]
shp_file(str): input file that geopandas reads to make a graph based on end to end connectivity
weights(list): weights columns transfered to the store edges. Name is maintained.
oneway(str): series where oneway streets (edges) are denoted with a 1, 0 denotes twoway. None, assumes
twoway edge.
overwrite_existing(bool): if true, the existing store is overwritten.
rounding(int): the number of digits to round line coordinates to get unique nodes (precision)
returns hdf5_path(str)"""
if os.path.exists(hdf5_path):
if overwrite_existing:
print("Overwriting existing store...")
os.remove(hdf5_path)
else:
print("Existing store at path: {0}".format(hdf5_path))
return hdf5_path
all_edges_twoway = True
oneway_field_list = []
if oneway is not None:
all_edges_twoway = False
oneway_field_list.append(oneway)
print("Reading shapefile with geopandas: {0}...".format(shp_file))
nodes, edges =get_nodes_and_edges(shp_file,rounding)
h5store = pd.HDFStore(hdf5_path)
print("Establishing node store...")
df_nodes = nodes
h5store['nodes'] = df_nodes
df_nodes['nodeid'] = df_nodes.index.values
edge_cnt = len(edges)
print("Establishing edge store for {0} edges...".format(edge_cnt))
df_edges= edges[['from_x','from_y','to_x','to_y'] + weights + oneway_field_list].copy()
print("Transferring nodeids to edges...")
df_edges=pd.merge(df_edges, df_nodes, how='left', left_on=['from_x','from_y'], right_on=['x','y'])
df_edges=pd.merge(df_edges, df_nodes, how='left', left_on=['to_x','to_y'], right_on=['x','y'], suffixes=('_from', '_to'))
#nodeids are duplicated on from the joined nodes, joined first to from, suffix to on next set
df_edges.rename(columns= {'nodeid_from': 'from', 'nodeid_to': 'to'}, inplace=True)
df_edges=df_edges[['from','to'] + weights + oneway_field_list]
if all_edges_twoway:
pass
else:
print("Setting up twoway edges...")
twoway_edges = df_edges[df_edges[oneway]==0].copy()
twoway_to = twoway_edges["to"].copy()
twoway_edges["to"] = twoway_edges["from"]
twoway_edges["from"] = twoway_to
df_edges = pd.concat([df_edges,twoway_edges])
h5store['edges']=df_edges
h5store.close()
print("Graph store construction complete...")
return hdf5_path Oh, and I hate when people post snippits without declaring a license. I use MIT (Copyright (c) 2017 David Wasserman) for snippits. You can reference here. Please use as you like if you find this on the forums. |
One of the suggestions in @Holisticnature's first comment was to change the default in the I'd always assumed that networks were loaded as directed graphs, because the edge parameters are named And I know it sucks to break API's, but should we also rename the parameter from
Here's the relevant code: |
Works for me. |
@smmaurer I also think this makes sense, but also I have usually seen the convention be that you have to declare one-way segments (directed edges) rather twoway segments. This makes sense in urban networks where most segments are undirected (twoway), and only higher order/downtown streets are directed. I think this convention results from the fact it is easier to flag one-way streets in a database rather than flag everything. |
@Holisticnature Thanks, this is helpful. I don't have a good sense of what the conventions are. Definitely true that most urban street segments are undirected, and that 'one-way' and 'two-way' make sense as labels if we're talking about streets. Removing my earlier post about penciling this in for the next release -- let's see if other people have comments. |
Agreed. To be honest, I am not sure anyone is really qualified to speak to "convention", but I will qualify it by saying 'based on my understanding of the convention based on working with a few municipal centerline networks/OSM data (where oneway is something that requires a tag etc)'. That is where I draw my basis for a broad generalization. |
This code helped a lot. It was not clear to me how network should be created from shapefiles, or even by scratch. @d-wasserman When I try to do use the second method (store to hdf5) i get this error message: "['weight'] not in index" Am I supposed to create a new column with the weights value? Any help would be appreciated. |
Hi @fillipefeitosa, I recently updated this code with a few minor changes that resolves some bugs. To keep the discussion related to Pandana, I have moved this code to this gist. The code change I made did not relate to weights, but the "weights" field should be a column or column of "impedance" or "distance" you want to use with the network. Citation information is in the gist. Thank you! |
This is likely a feature request or a suggestion of where to start for a potential PR. One of the options for network declaration is the ability to set a boolean for twoway edges. If this is true, the edges of the graph are flipped and appended back to the edges.
I have used a previously posted solution (by SEMCOG) using networkx to construct a graph from a shapefile. NetworkX's import does not allow fine control of of how the graph is constructed. I was hoping the twoway option for the network API could be altered to allow a series (of 0s & 1s). To determine which edges are flipped? Also, are there any objections to changing "twoway" to "oneway" defaulting to false as part of that change? I am willing to work on a PR for this, just not sure where to start.
Environment
Linux
3.6
3.0
The text was updated successfully, but these errors were encountered: