Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Handle SPEC 2 #229

Closed
5 tasks done
stefanv opened this issue Jun 26, 2023 · 2 comments · Fixed by #307
Closed
5 tasks done

Handle SPEC 2 #229

stefanv opened this issue Jun 26, 2023 · 2 comments · Fixed by #307

Comments

@stefanv
Copy link
Member

stefanv commented Jun 26, 2023

jarrodmillman added a commit to jarrodmillman/specs that referenced this issue Jul 11, 2023
This was decided in a SPEC conference call.
See scientific-python#229.
jarrodmillman added a commit that referenced this issue Jul 12, 2023
This was decided in a SPEC conference call.
See #229.

This is the fist step and more needs to be done.
@MridulS
Copy link
Member

MridulS commented Mar 5, 2024

I think we can handle this in a new way:

A successful prototype implementation of this SPEC (at least in spirit) is already
available inside NetworkX.

NetworkX has developed a dispatching layer which can plug into multiple backends.
Currently we have backends which can dispatch to CuGraph, GraphBLAS and
a joblib backed parallel implementation of algorithms in NetworkX.

One of the goals here is to provide drop in replacement for old code written with NetworkX. Users
should be able to set a config option/env variable and dispatch their code to different backends.
These backends could be hardware specific ones, reimplementations in other languages (hello rust!) or
using a totally new data structure (GraphBLAS in the case of NetworkX).

On the NetworkX side we are still ironing out more details, and we are still missing out on a detailed
spec (or SPEC). I think other projects in the scientific python ecosystem could also benefit from some
kind of dispatching along these lines. There is some progress towards creating an engine API in
scikit-learn
which follows similar rational.

Getting a SPEC together with all the experiments across the ecosystem could be a fruitful task.
Maybe this SPEC could be more of an informational document about various efforts across the
ecosystem and we can extract the common bits to form some recommendations?

@rgommers
Copy link
Contributor

rgommers commented Mar 5, 2024

That sounds like a great idea @MridulS. It'd be useful to have an overview of which projects are trying what, and some learnings. It's still early days in a way, but hopefully at some point one or a couple of patterns appear that can be adopted by others.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants