Update:
I can approve that this workaround significantly reduces the performance issue:

$.ui.sortable.prototype._setHandleClassName = function() {
    this._removeClass( this.element.find( ".ui-sortable-handle" ), "ui-sortable-handle" );
    $.each( this.items, function() {
        (this.instance.options.handle
         ? this.item.find( this.instance.options.handle )
         : this.item
        ).addClass('ui-sortable-handle');
    });
};

If set before the initialization of the sortable.

Thanks for the report. Since the issue is already in 1.12, given limited team resources it's not likely to be fixed by the UI team; see the project status at https://blog.jqueryui.com/2021/10/jquery-maintainers-update-and-transition-jquery-ui-as-part-of-overall-modernization-efforts/. I'd review a PR if you'd like to submit one, though.

@mgol I've submitted a PR ;-)

I can confirm for the last 5 years in Primefaces we have used this same patch.

Here is the original report ticket: primefaces/primefaces#3675

// GitHub PrimeFaces #3675 performance
$.widget( "ui.sortable", $.ui.sortable, {
    _setHandleClassName: function() {
        this._removeClass( this.element.find( ".ui-sortable-handle" ), "ui-sortable-handle" );
        $.each( this.items, function() {
                        (this.instance.options.handle 
                        ? this.item.find( this.instance.options.handle ) 
                        : this.item
                        ).addClass('ui-sortable-handle');
        } );
    }
});

@mgol, @fnagel, the cause for the excessive time complexity is an O(n*n*log(n)) algorithm.

Analysis

Consider this call stack:

. _setHandleClassName (loops over all items)
. _addClass
. _toggleClass
. _classes
. processClassString
. bindRemoveEvent
. _on
. add
. jQuery.uniqueSort (loops over all items)

Given that the jQuery.uniqueSort will have a time complexity in O(n*log(n)), _setHandleClassName will have a time complexity in O(n*n*log(n)). This more than quadratic time complexity makes _setHandleClassName inacceptably slow for large inputs (e.g. 5000 items which shall be sortable).