Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Vectorize the edges and attributes of the networks #119

Open
LegrandNico opened this issue Oct 11, 2023 · 0 comments
Open

Vectorize the edges and attributes of the networks #119

LegrandNico opened this issue Oct 11, 2023 · 0 comments
Assignees
Labels
networks performances wontfix This will not be worked on

Comments

@LegrandNico
Copy link
Collaborator

LegrandNico commented Oct 11, 2023

The data format we are currently using to represent attributes and edges is suboptimal regarding JAX transformation. Per JAX standard, PyTrees are only accessible at compile time and cannot be indexed using Tracer, that are accessible at run time. For this reason, update functions need to set the node_idx and edges variables as static arguments, which cache a new function for each node separately. This makes us lose the advantages provided by the modularity of the implementation, and large models will definitely benefit from having a uniquely cached update function.

The solution I see would be:

  • [x ] Use a dictionary of arrays to store the edges using a connectivity matrix representation.
  • [x ] Use a dictionary of arrays for each node parameter.

Update: The current status is that it is (very) difficult to write readable update functions that can pass messages with a dynamically valued number of nodes without using something like Dynamic shapes. It is under development in JAX but not yet available. Until such a feature is available it seems unreasonable to try to move the code to this implementation. We have a working example for the two-level binary HGF and the total execution time is longer than the default implementation, so it is unclear if we would really benefit from this, besides compilation time.

@LegrandNico LegrandNico self-assigned this Oct 23, 2023
@LegrandNico LegrandNico removed the good first issue Good for newcomers label Oct 23, 2023
@LegrandNico LegrandNico changed the title Use a dictionary instead of tuples to store the edges of a network Use dictionaries of array to store the edges and attributes of a network Oct 23, 2023
@LegrandNico LegrandNico changed the title Use dictionaries of array to store the edges and attributes of a network Use dictionaries of arrays to store the edges and attributes of a network Oct 23, 2023
@LegrandNico LegrandNico changed the title Use dictionaries of arrays to store the edges and attributes of a network Vectorize the edges and attributes of the networks Oct 24, 2023
@LegrandNico LegrandNico added the wontfix This will not be worked on label Nov 6, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
networks performances wontfix This will not be worked on
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant