Skip to content

ENH: Add "limit" parameter to dijkstra calculation #2992

Merged
merged 5 commits into from Oct 19, 2013

6 participants

@Eric89GXL

This is pretty straightforward. Tests pass, and I have also tried it with a real example with > 100,000 nodes. Results are identical (to numerical precision) with C code someone else wrote. Using a limit in my use-case speeds up computation by a factor of 20.

@Eric89GXL Eric89GXL referenced this pull request in mne-tools/mne-python Oct 14, 2013
Merged

ENH: Add source-space distances #802

@coveralls

Coverage Status

Coverage remained the same when pulling a115ccc on Eric89GXL:dijkstra-limit into 57ac922 on scipy:master.

@coveralls

Coverage Status

Coverage remained the same when pulling a115ccc on Eric89GXL:dijkstra-limit into 57ac922 on scipy:master.

@argriffing

cool story, coveralls

@rgommers
SciPy member

:) It's useful when you actually click the badge.

@argriffing

It looks like it commented twice on the same commit.

@larsmans larsmans commented on an outdated diff Oct 14, 2013
scipy/sparse/csgraph/_shortest_path.pyx
@@ -404,6 +409,10 @@ def dijkstra(csgraph, directed=True, indices=None,
if np.any(indices < 0) or np.any(indices >= N):
raise ValueError("indices out of range 0...N")
+ if not np.isscalar(limit):
+ raise ValueError('limit must be numeric (float)')
@larsmans
larsmans added a note Oct 14, 2013
I think a `TypeError` would be more appropriate.

On second thought, can't limit be declared a double variable in the function prototype? That way, Cython generates the proper exception and the limit = float(limit) is no longer needed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
@Eric89GXL

Thanks @larsmans, fixed now.

@coveralls

Coverage Status

Coverage remained the same when pulling e2bf1d7 on Eric89GXL:dijkstra-limit into 57ac922 on scipy:master.

@jakevdp jakevdp commented on an outdated diff Oct 15, 2013
scipy/sparse/csgraph/_shortest_path.pyx
@@ -348,6 +348,11 @@ def dijkstra(csgraph, directed=True, indices=None,
If True, then find unweighted distances. That is, rather than finding
the path between each point such that the sum of weights is minimized,
find the path such that the number of edges is minimized.
+ limit : float, optional
+ The maximum distance to calculate. Using a smaller limit will
+ decrease computation time by aborting calculations between pairs
+ that are separated by a distance > limit.
+ .. versionadded:: 0.14.0
@jakevdp
SciPy member
jakevdp added a note Oct 15, 2013

The doc string should specify that any pair with distance > limit will be set to distance = infinity.

Also, in the code we should enforce limit >= 0, otherwise this claim won't be true!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
@jakevdp
SciPy member
jakevdp commented Oct 15, 2013

Really nice work here. A simple change, and it adds some nice flexibility to the algorithm. Examining the code, it looks to be algorithmically correct, and it won't slow down the default case.

One potential addition that comes to mind (which would be much less straightforward to implement): an option to allow building a sparse rather than dense output matrix. Because you don't know from the beginning how many nodes would be included, it would require dynamically expanding the array during the algorithm. I don't think that's worth tackling right now, but I thought I'd mention it!

@Eric89GXL

@jakevdp thanks for the comments, they have been (hopefully) addressed by the latest commit. Let me know if you find any other issues.

I agree that a sparse representation could be useful. It's actually what we end up converting our representation to eventually anyway. Probably good for a Sunday hack PR at some point.

@coveralls

Coverage Status

Coverage remained the same when pulling f792a6f on Eric89GXL:dijkstra-limit into 57ac922 on scipy:master.

@Eric89GXL

@jakevdp anything else I need to do to move this toward merge? Or are we just waiting for another person or two to have a look?

@jakevdp
SciPy member
jakevdp commented Oct 16, 2013

@Eric89GXL - This looks good to me: I'm :+1: for merge.

I'd like another dev to give a second opinion. Perhaps @pv, @larsmans, or @rgommers?

@argriffing argriffing commented on the diff Oct 16, 2013
scipy/sparse/csgraph/tests/test_shortest_path.py
@@ -49,6 +58,21 @@
@dec.skipif(np.version.short_version < '1.6', "Can't test arrays with infs.")
@argriffing
argriffing added a note Oct 16, 2013

These version-conditional tests are going to stop running when numpy version is up to 1.10.x?

@Eric89GXL
Eric89GXL added a note Oct 16, 2013

Yep. Should this become a new PR, or do you want me to tackle this? Shouldn't be hard with LooseVersion from distutils.

@jakevdp
SciPy member
jakevdp added a note Oct 16, 2013

Looks like that existed before this PR. It should be fixed, but probably under a separate issue.

Edit: opened in #2999

@rgommers
SciPy member
rgommers added a note Oct 16, 2013

They will, we have to fix that in multiple places at once for the next release. I'll open an issue.

@rgommers
SciPy member
rgommers added a note Oct 16, 2013
@rgommers
SciPy member
rgommers added a note Oct 16, 2013

Glad we're on the same page:)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
@jakevdp jakevdp referenced this pull request Oct 16, 2013
Closed

Unsafe version checking #2999

@Eric89GXL

@rgommers any other thoughts beyond the version testing?

Sorry to keep pinging, I'm just waiting for this to be merged before a PR in a different package can be merged.

@rgommers
SciPy member

Looks good to me. Did some testing with invalid inputs, all gives the expected errors. Merging. Thanks Eric, all

@rgommers rgommers merged commit f82e5b2 into scipy:master Oct 19, 2013

1 check passed

Details default The Travis CI build passed
@Eric89GXL

Thanks @rgommers @jakevdp for the quick reviews + merge.

@Eric89GXL Eric89GXL deleted the Eric89GXL:dijkstra-limit branch Mar 11, 2014
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Something went wrong with that request. Please try again.