Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

More metrics ? e.g. adjusted randindex #278

Open
rfayat opened this issue Nov 30, 2022 · 2 comments
Open

More metrics ? e.g. adjusted randindex #278

rfayat opened this issue Nov 30, 2022 · 2 comments

Comments

@rfayat
Copy link

rfayat commented Nov 30, 2022

Hi !

First of all thanks a lot for this great toolbox (and for the great review that comes with it).

While using ruptures I noticed that only a few metrics were available for comparing two segmentations and thought it would maybe be a good idea to implement additional ones.

For instance, it is super straightforward to implement the adjusted randindex by leveraging the efficient implementation of the adjusted rand score in scikit learn (although adding this to ruptures would imply to add scikit as a dependency) :

from sklearn.metrics import adjusted_rand_score


def chpt_to_label(bkps):
    """Return the segment index each sample belongs to.
    
    Example
    -------
    >>> chpt_to_label([4, 10])
    array([0, 0, 0, 0, 1, 1, 1, 1, 1, 1])
    """
    duration = np.diff([0] + bkps)
    return np.repeat(np.arange(len(bkps)), duration)


def randindex_adjusted(bkps1, bkps2):
    "Compute the adjusted rand index for two lists of changepoints."
    label1 = chpt_to_label(bkps1)
    label2 = chpt_to_label(bkps2)
    return adjusted_rand_score(label1, label2)

I guess there are much more similar metrics (Intersection over Union...) that could be added to ruptures in order to make the package even more complete than it is right now.

Of course this is simply a suggestion and I would be happy to give a hand if you think this direction would indeed be interesting to pursue :)

Cheers,

Romain

@deepcharles
Copy link
Owner

deepcharles commented Dec 6, 2022

Hi Romain,

Thanks for you message. This would indeed be a great addition to have more metrics.

For the adjusted Rand Index, I do not mind to have scikit-learn as a dependency so your current implementation is fine by me. Do not hesite to make a pull request.

Same comment for a metric derived from the Intersection over Union.

If you make a PR, just put the code, and we'll work together from here.

Many thanks!

@rfayat
Copy link
Author

rfayat commented Dec 7, 2022

I should have some bandwidth to take care of it and open a PR early next week 😄

Cheers,

Romain

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants