Retriving a dumped module via "from_dump" function

[EnricoMariaFerrari]

Describe the bug:

When I want to save an approximate module in my laptop, I need to use the function "dump()" to save the module. However, when I want to retrive such module, I have error: "a bytes-like object is required, not 'list'".

(Minimal) example:
This is an example of the code, where _s is a slicer:

import multipers as mp

mod = mp.module_approximation(_s)
dumped_mod = mod.dump()
new_mod = mp.mma_structures.from_dump_f64(dumped_mod )

which outputs

then the error about the last row of code:

File c:\Users\Enrico\AppData\Local\Programs\Python\Python311\Lib\site-packages\multipers\mma_structures.pyx:1011, in multipers.mma_structures.from_dump_f64()

File c:\Users\Enrico\AppData\Local\Programs\Python\Python311\Lib\site-packages\multipers\mma_structures.pyx:988, in multipers.mma_structures.cmod_from_dump_f64()

File c:\Users\Enrico\AppData\Local\Programs\Python\Python311\Lib\site-packages\multipers\mma_structures.pyx:968, in multipers.mma_structures.summand_from_dump_f64()

File <stringsource>:663, in View.MemoryView.memoryview_cwrapper()

File <stringsource>:353, in View.MemoryView.memoryview.__cinit__()

TypeError: a bytes-like object is required, not 'list'

Maybe my problem is stupid to solve, let me know it. Thank you very much!!

[DavidLapous]

Hey @EnricoMariaFerrari, thanks for your interest in this library !

This is indeed a bug, and should be fixed by 8ff1fdd.
Some filtrations classes were updated some time ago, and I forgot to update this… thanks for noticing it

I'll try to do a release “soon”, but you can compile multipers from the sources / take the GitHub artifacts once it's done compiling (around here : https://github.com/DavidLapous/multipers/actions/runs/14308018261).

[DavidLapous]

BTW, the multipers structures should all be pickle-compatible so you can use something like (once this is updated)

import pickle as pkl

pkl.dump(mod, open("some_file", "wb"))
mod_ = pkl.load(open("some_file", "rb"))
assert mod == mod_

where mod is any structure

[EnricoMariaFerrari]

Thank you very much!
Last question: is it possible to differentiate invariants from a multi-filtration of Cubical Complexes using Multipers?
Thank you in advance.

[DavidLapous]

Last question: is it possible to differentiate invariants from a multi-filtration of Cubical Complexes using Multipers?

Yes definitely ! The interface is a bit convoluted for this kind of filtrations, but something like this should work

image = torch.rand(size=(50,50,2), requires_grad=True, dtype=float) # differentiable tensor
hom_degree=1
cub_cplx = mp.filtrations.Cubical(image.detach())
cub_cplx = cub_cplx.minpres(degree=hom_degree) # preprocessing for this degree
# computes the differentiable grid (cf [Differentiability and Optimization of Multiparameter Persistent Homology])
filtration_grid = mp.grids.compute_grid([image[...,i].ravel() for i in range(image.shape[-1])])
# signed measure differentiable computation requires the grid argument 
mp.signed_measure(cub_cplx, grid=filtration_grid, degree=hom_degree, invariant="hilbert", plot=True)

This is due to the fact that the simplicial structures / presentations cannot “store their differentiability structure” yet, so it has to be given again (the grid argument here) to the invariant to recover a differentiable object in the end.
I have a branch with a better interface for differentiability (https://github.com/DavidLapous/multipers/tree/test_diff_stuff), but a bunch of stuff need to be changed to cover everything.

[EnricoMariaFerrari]

Thank you for the clarification. However, it seems that the grid we provide must be a NumPy object (and not a PyTorch tensor), which prevents us from fully integrating the computation into a neural network where the filtration (i.e., the image in your example) can be learned. The inability to keep the gradient flow intact due to the conversion to NumPy seems to be the limiting factor in building a network that learns the cubical filtration while being fully differentiable. Is there a way to overcome this limitation, or is it simply not possible with the current implementation?

[DavidLapous]

I've just tested this snippet of code on both latest release version and lastest commit version, and both of them work, and have a proper gradient.

Which version of multipers are you using ?

[EnricoMariaFerrari]

I'm using version 2.3.1 and I was wrong when I said the grid is numpy object, sorry.
I didn't explain myself well, the code works for me as well, it's just that I find it difficult to integrate this code into neural networks that want to learn the filtration ('image') as a parameter. I'm not sure if you can update the code you sent me earlier with a very simple vectorization done on the calculated signed measure, so that I can compute a loss and observe the actual gradient calculation of it wrt "image". Thank you again!

[DavidLapous]

The simplest way to vectorize a measure is with a convolution,
for which there is a notebook to achieve this in the documentation : https://davidlapous.github.io/multipers/notebooks/graph_autodiff.html
I agree that this notebook is really too complicated (but is significantly more optimized than the "naive" version), I'll try to do a simpler one when I'll have time.
BTW, if you have time yourself, feel free to contribute to this library if you have a nice example in a notebook.

Here is a snippet of code to achieve this (KDE can be replaced by any pytorch based convolution here)

from multipers.filtrations.density import KDE
res = (100,100)
grid_=mp.grids.compute_grid(sms, strategy="regular", resolution = res) ## little helper to get grids
grid = mp.grids.todense(grid_) ## dense version of the grid
(pts,w), = sms # differentiable w.r.t. the input
img = (
    KDE(bandwidth=.01, kernel="gaussian")
    .fit(pts, sample_weights=w)
    .score_samples(grid)
    .reshape(*res)
)
from multipers.plots import plot_surface
plot_surface(grid_, img.detach().numpy())
img ## diff w.r.t. differentiable inputs 

I also noticed that not all ML pipelines work properly with torch, so I'll have to fix that at some point as well