Skip to content

Commit

Permalink
For latest release (#470)
Browse files Browse the repository at this point in the history
* fixes errors on qgis (#417)

* runs examples (#422)

Co-authored-by: pveigadecamargo <pveigadecamargo@anl.gov>

* Update LICENSE.TXT (#427)

* fix matrix indices for memory-only matrices (#425)

* fix matrix indices for memory-only matrices

* run black

---------

Co-authored-by: Pelle Koster <pelle.koster@nginfra.nl>

* Pedro/cores skimming (#426)

* Moves the setting of number of cores to the right place

* Moves documentation to the right place

---------

Co-authored-by: pveigadecamargo <pveigadecamargo@anl.gov>

* Updates assignment logging (#423)

* assignment logging

* Updates logging

* Adds __config log

* Adds test and docs

* Update aequilibrae/paths/traffic_assignment.py

* Fixes tests

* fixes coverage testing

* merges testing changes

* style

* style

* style

* style

* narrows coverage test

* Update test_traffic_assignment.py

* Apply suggestions from code review

* Fixes test_traffic_assignment

---------

Co-authored-by: pveigadecamargo <pveigadecamargo@anl.gov>
Co-authored-by: Pedro Camargo <c@margo.co>

* narrows coverage test (#429)

Co-authored-by: pveigadecamargo <pveigadecamargo@anl.gov>

* moves openmatrix to a primary dependency (#431)

* moves openmatrix to a primary dependency

* QGIS moved to Numpy 1.24 before moving to Python 3.10

---------

Co-authored-by: pveigadecamargo <pveigadecamargo@anl.gov>

* Updates software version for release

* Bumps up version

* Bumps up documentation version

---------

Co-authored-by: pveigadecamargo <pveigadecamargo@anl.gov>

* minor change to test CI

* Fix multiple classes not being presented in the returned d (#438)

* Prevent invalid attribute names on AequilibraE data fields

This previously would have resulted in syntax error when accessing but
its best we don't accept them anyway.

* Fix multiple classes not being presented in the returned df

Bug report: https://groups.google.com/g/aequilibrae/c/y_q9nLNs6-Y/m/yWXNpey9AAAJ

* Style

* Skip if network fails

* Rounding

* fixes test

* Revert "Rounding"

Vatican City really did move huh

This reverts commit d9d0a5d.

---------

Co-authored-by: pveigadecamargo <pveigadecamargo@anl.gov>

* Update README.md (#439)

* Add implicit noexcept from Cython<3.0.0 (#440)

With the release of Cython 3.0.0 there are a few changes of note to use. Particularly the performance impact of the
removal of the implicit noexcept.

Now all cdefs allow exceptions by default meaning every cdef must require the gil at the end of the function regardless
of whether it was nogil or not. Adding the noexcept clause reverts to the old behaviour.

There is a compiler directive to result this old behaviour but better be be explicit when the solution is one regex away:
`^(cp?def(?:.|\n).*?)(nogil|):$`

https://cython.readthedocs.io/en/latest/src/userguide/migrating_to_cy30.html#exception-values-and-noexcept

* Disable select link (#443)

* Fix empty matrices not being saved (#444)

Introduced in 7c2beb6

* Matrix exports and deprecation warning (#435)

* Bumps up version

* Bumps up documentation version

* fixes matrix export

* fixes deprecations

* addresses SciPy versions

* addresses SciPy versions

* addresses SciPy versions

* addresses SciPy versions

* addresses SciPy versions

* fixes test

* fixes test

---------

Co-authored-by: pveigadecamargo <pveigadecamargo@anl.gov>

* Select link correctness fix (#447)

* Revert "Disable select link (#443)"

This reverts commit 0cd3d48.

* Add Kai Tang's test and data

* Potential select link fix

* Test formatting

* Fix tests imports

* Add select link test

This test asserts that the results of the select link on the links 7, and 13 are the same as the results of the
assignment. These links were chosen for this particular network to cover all paths used.

* Prevent data races in select link results

Memory for the multi-threaded runs are now allocated in MuliThreadedAoN along side the rest of the multi-threaded memory.

* Installs package to run documentation pipeline

* installing

* Install pandoc

---------

Co-authored-by: Pedro Camargo <c@margo.co>

* bumps up the version (#451)

Co-authored-by: pveigadecamargo <pveigadecamargo@anl.gov>

* new version

* Fix documentation bugs on latest (#456)

* Update requirements-docs.txt

modifies pydata theme version

* Update requirements-docs.txt

* PathResults.compute_path fix stale variables (#457)

Previously if a path that exists was computed, and then a path that does not exist was attempted, the old path variables
were not cleared.

* Replaces triggers with check statements (#455)

* replaces triggers with check statements

* Guarantees that links can be created without adding information on nodes

---------

Co-authored-by: pveigadecamargo <pveigadecamargo@anl.gov>

* Adds zones and connectors (#461)

* Better import from OSM and description of bicycle & bus data in network (#464)

Parameters file is a bit hidden/tricky to use for qaequilibrae users, this should help them for a better default OSM import which keep data for bus and bicycle
Have keep the same structure as OSM but this result in 3 fields for each one (3 for bicycle and 3 for bus).

Maybe we can think of a way to achieve only one field for bicycle and one for bus ? Might not so easy because of the number of combinations available in OSM, example : https://wiki.openstreetmap.org/wiki/Key:cycleway

Co-authored-by: Arthur Evrard <83211842+Art-Ev@users.noreply.github.com>

* Early exit Dijsktra's with path reuse (#466)

* Early exit Dijsktra's with path reuse

* Remove early exit arg from update_trace, add private state

Add a private variable which tracks the status of the tree from the
previous call to `compute_path`. Setting the `PathResults.early_exit`
be used to modify the behaviour of the `compute_path` call in
`update_trace`.

* Adds support for Python 3.12 (#453)

* Adds support for Python 3.12

* Update build_mac.yml

* Move from depreciated `license_file` as its now a build error

* `assertEquals` is a deprecated alias for `assertEqual`

---------

Co-authored-by: pveigadecamargo <pveigadecamargo@anl.gov>
Co-authored-by: Jake-Moss <jake.moss@uqconnect.edu.au>

* Don't mark the path computation as `early_exit` when unreachable (#469)

* Initial A* implementation (#467)

* Initial A* implementation

This implementation uses a haversine distance heuristic, in theory
this a consistent heuristic [1] that would give us "an optimal path
without processing any node more than once" when the graph cost is set
to distance, however, in testing I've found that the distance
calculations based and lon/lat done by myself and QGIS do not align
with those in the project. Unfortunately this makes the heuristic not
admissible [2] meaning A* will not always find the least-cost path,
and may not do the minimum amount of work.

From some basic instrumenting I found A* is consistently an order of
magnitude better than the existing Dijkstra's implementation when it
comes to heap operation counts. Time wise I haven't performed real
benchmarks.

Currently there is only one heuristic implemented for distance, if the
graph cost field is set to `free_flow_time` the heuristic, while not
useless is theory, it is practically as the magnitudes of the distances
dominates any graph cost meaning A* will explore the whole network,
just like Dijkstra's.

[1] https://en.wikipedia.org/wiki/Consistent_heuristic
[2] https://en.wikipedia.org/wiki/Admissible_heuristic

* Indexing fixes found on Arkansas network

* Updates graph types

* A* improvements and corrections, equirectangular heuristic added

* Update "without a model" example

* fixup! A* improvements and corrections, equirectangular heuristic added

* Set lon/lat index manually instead of passing to prepare_graph

* fixup! Set lon/lat index manually instead of passing to prepare_graph

* Split path computation and skimming example, use Coquimbo over Sioux

* Add note about skimming with A*

* General clean up

* Extend existing test to include A* via subtests

* Style

* Add runtime heuristic switching

* Document heuristic switching

* Heuristic swithcing tests

* Python 3.7 compatibility

* Missed changes

* Add note for A* non-distance metrics

* Bumps up version for release

---------

Co-authored-by: pveigadecamargo <pveigadecamargo@anl.gov>

---------

Co-authored-by: Renata Imai <53949163+r-akemii@users.noreply.github.com>
Co-authored-by: pveigadecamargo <pveigadecamargo@anl.gov>
Co-authored-by: PelleK <elfjes@users.noreply.github.com>
Co-authored-by: Pelle Koster <pelle.koster@nginfra.nl>
Co-authored-by: Jamie Cook <jimi.cook@gmail.com>
Co-authored-by: Jake Moss <jake.moss@uqconnect.edu.au>
Co-authored-by: Jamie Cook <jamie.cook@veitchlister.com.au>
Co-authored-by: Arthur Evrard <83211842+Art-Ev@users.noreply.github.com>
  • Loading branch information
9 people committed Nov 25, 2023
1 parent c5ee6d6 commit de847a0
Show file tree
Hide file tree
Showing 40 changed files with 878 additions and 432 deletions.
2 changes: 1 addition & 1 deletion .flake8
Original file line number Diff line number Diff line change
Expand Up @@ -3,4 +3,4 @@ max-line-length = 120
ignore = E203, E266, E501, W503, F403, F401, C901, W605
max-complexity = 20
select = B,C,E,F,W,T4,B9
exclude = .idea,.git,__pycache__,sphinx,.venv,venv,docs/*,benchmarks/*
exclude = .idea,.git,__pycache__,sphinx,.venv*,.venv,venv,docs/*,benchmarks/*
4 changes: 2 additions & 2 deletions .github/workflows/build_linux.yml
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ jobs:
- name: Build manylinux Python wheels
uses: RalfG/python-wheels-manylinux-build@v0.7.1
with:
python-versions: 'cp37-cp37m cp38-cp38 cp39-cp39 cp310-cp310 cp311-cp311'
python-versions: 'cp37-cp37m cp38-cp38 cp39-cp39 cp310-cp310 cp311-cp311 cp312-cp312'
pip-wheel-args: '--no-deps'

- name: Moves wheels
Expand All @@ -38,7 +38,7 @@ jobs:
- name: Build manylinux Python wheels
uses: RalfG/python-wheels-manylinux-build@v0.7.1
with:
python-versions: 'cp38-cp38 cp39-cp39 cp310-cp310'
python-versions: 'cp39-cp39'
pip-wheel-args: '--no-deps'

- name: Moves wheels
Expand Down
7 changes: 3 additions & 4 deletions .github/workflows/build_mac.yml
Original file line number Diff line number Diff line change
Expand Up @@ -14,8 +14,7 @@ jobs:
continue-on-error: true
strategy:
matrix:
python-version: [ '3.7', '3.8', '3.9', '3.10', '3.11']

python-version: [ '3.7', '3.8', '3.9', '3.10', '3.11', '3.12']
steps:
- uses: actions/checkout@v3
- name: Set Python environment
Expand Down Expand Up @@ -51,8 +50,8 @@ jobs:
twine upload dist/*.whl
- name: Publish tar files
if: ${{ (github.event_name == 'release') && (env.HAS_SECRETS == 'true') && (matrix.python-version == '3.9') }}
if: ${{ (github.event_name == 'release') && (env.HAS_SECRETS == 'true') && (matrix.python-version == '3.11') }}
env:
TWINE_USERNAME: ${{ secrets.PYPI_USERNAME }}
TWINE_PASSWORD: ${{ secrets.PYPI_PASSWORD }}
run: twine upload dist/*.gz
run: twine upload dist/*.gz
2 changes: 1 addition & 1 deletion .github/workflows/build_windows.yml
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ jobs:
continue-on-error: true
strategy:
matrix:
python-version: [ '3.7', '3.8', '3.9', '3.10', '3.11']
python-version: [ '3.7', '3.8', '3.9', '3.10', '3.11', '3.12']
architecture: ['x64']
steps:
- uses: actions/checkout@v3
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/unit_tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@ jobs:
runs-on: ${{ matrix.os}}
strategy:
matrix:
python-version: [ '3.7', '3.8', '3.9', '3.10', '3.11']
python-version: [ '3.7', '3.8', '3.9', '3.10', '3.11', '3.12']
os: [windows-latest, ubuntu-latest]

max-parallel: 20
Expand Down
2 changes: 1 addition & 1 deletion __version__.py
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
version = 0.9
minor_version = "4"
minor_version = "5"
release_name = "Queluz"

release_version = f"{version}.{minor_version}"
34 changes: 33 additions & 1 deletion aequilibrae/parameters.yml
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,30 @@ network:
description: name
osm_source: name
type: text
- cycleway:
description: cycleway, both way
osm_source: cycleway
type: text
- cycleway_right:
description: cycleway, right
osm_source: cycleway:right
type: text
- cycleway_left:
description: cycleway, left
osm_source: cycleway:left
type: text
- busway:
description: busway
osm_source: busway
type: text
- busway_right:
description: busway, right
osm_source: busway:right
type: text
- busway_left:
description: busway, left
osm_source: busway:left
type: text
two-way:
- lanes:
description: lanes
Expand Down Expand Up @@ -128,6 +152,14 @@ network:
modes:
bicycle:
link_types:
- primary
- primary_link
- secondary
- secondary_link
- tertiary
- tertiary_link
- living_street
- residential
- cycleway
- corridor
- pedestrian
Expand Down Expand Up @@ -313,7 +345,7 @@ network:
required: false
lanes:
description: Number of lanes in the direction of travel
type: inetger
type: integer
required: false
bike_facility:
description: Type of bicycle accommodation
Expand Down
77 changes: 67 additions & 10 deletions aequilibrae/paths/AoN.pyx
Original file line number Diff line number Diff line change
Expand Up @@ -114,6 +114,7 @@ def one_to_all(origin, matrix, graph, result, aux_result, curr_thread):
original_b_nodes_view)

w = path_finding(origin_index,
-1, # destination index to disable early exit
g_view,
b_nodes_view,
graph_fs_view,
Expand Down Expand Up @@ -178,6 +179,7 @@ def path_computation(origin, destination, graph, results):
"""
cdef ITYPE_t nodes, orig, dest, p, b, origin_index, dest_index, connector, zones
cdef long i, j, skims, a, block_flows_through_centroids
cdef bint early_exit_bint = results.early_exit

results.origin = origin
results.destination = destination
Expand Down Expand Up @@ -216,6 +218,18 @@ def path_computation(origin, destination, graph, results):
new_b_nodes = graph.graph.b_node.values.copy()
cdef long long [:] b_nodes_view = new_b_nodes

cdef bint a_star_bint = results.a_star
cdef double [:] lat_view
cdef double [:] lon_view
cdef long long [:] nodes_to_indices_view
cdef Heuristic heuristic
if results.a_star:
lat_view = graph.lonlat_index.lat.values
lon_view = graph.lonlat_index.lon.values
nodes_to_indices_view = graph.nodes_to_indices
heuristic = HEURISTIC_MAP[results._heuristic]


#Now we do all procedures with NO GIL
with nogil:
if block_flows_through_centroids: # Unblocks the centroid if that is the case
Expand All @@ -227,14 +241,31 @@ def path_computation(origin, destination, graph, results):
b_nodes_view,
original_b_nodes_view)

w = path_finding(origin_index,
g_view,
b_nodes_view,
graph_fs_view,
predecessors_view,
ids_graph_view,
conn_view,
reached_first_view)
if a_star_bint:
w = path_finding_a_star(origin_index,
dest_index,
g_view,
b_nodes_view,
graph_fs_view,
nodes_to_indices_view,
lat_view,
lon_view,
predecessors_view,
ids_graph_view,
conn_view,
reached_first_view,
heuristic)
else:
w = path_finding(origin_index,
dest_index if early_exit_bint else -1,
g_view,
b_nodes_view,
graph_fs_view,
predecessors_view,
ids_graph_view,
conn_view,
reached_first_view)


if skims > 0:
skim_single_path(origin_index,
Expand Down Expand Up @@ -280,14 +311,22 @@ def path_computation(origin, destination, graph, results):
del all_nodes
del all_connectors
del mileposts

else:
results.path = None
results.path_nodes = None
results.path_link_directions = None
results.milepost = None

def update_path_trace(results, destination, graph):
# type: (PathResults, int, Graph) -> (None)
"""
If `results.early_exit` is `True`, early exit will be enabled if the path is to be recomputed.
If `results.a_star` is `True`, A* will be used if the path is to be recomputed.
:param graph: AequilibraE graph. Needs to have been set with number of centroids and list of skims (if any)
:param results: AequilibraE Matrix properly set for computation using matrix.computational_view([matrix list])
:param skimming: if we will skim for all nodes or not
:param early_exit: Exit Dijkstra's once the destination has been found if the shortest path tree must be reconstructed.
"""
cdef p, origin_index, dest_index, connector

Expand All @@ -298,9 +337,20 @@ def update_path_trace(results, destination, graph):
else:
dest_index = graph.nodes_to_indices[destination]
origin_index = graph.nodes_to_indices[results.origin]

results.milepost = None
results.path_nodes = None

# If the predecessor is -1 and early exit was enabled we cannot differentiate between
# an unreachable node and one we just didn't see yet. We need to recompute the tree with the new destination
# If `a_star` was enabled then the stored tree has no guarantees and may not be useful due to the heuristic used
# TODO: revisit with heuristic specific reuse logic
if results.predecessors[dest_index] == -1 and results._early_exit or results._a_star:
results.compute_path(results.origin, destination, early_exit=results.early_exit, a_star=results.a_star)

# By the invariant hypothesis presented at https://en.wikipedia.org/wiki/Dijkstra%27s_algorithm#Proof_of_correctness
# Dijkstra's algorithm produces the shortest path tree for all scanned nodes. That is if a node was scanned,
# its shortest path has been found, even if we exited early. As the un-scanned nodes are marked as unreachable this
# invariant holds.
if results.predecessors[dest_index] >= 0:
all_connectors = []
link_directions = []
Expand All @@ -321,6 +371,12 @@ def update_path_trace(results, destination, graph):
results.path_nodes = graph.all_nodes[np.asarray(all_nodes, graph.default_types('int'))][::-1]
mileposts.append(0)
results.milepost = np.cumsum(mileposts[::-1])
else:
results.path = None
results.path_nodes = None
results.path_link_directions = None
results.milepost = None


def skimming_single_origin(origin, graph, result, aux_result, curr_thread):
"""
Expand Down Expand Up @@ -384,6 +440,7 @@ def skimming_single_origin(origin, graph, result, aux_result, curr_thread):
b_nodes_view,
original_b_nodes_view)
w = path_finding(origin_index,
-1, # destination index to disable early exit
g_view,
b_nodes_view,
graph_fs_view,
Expand Down
Loading

0 comments on commit de847a0

Please sign in to comment.