[Bug]: Transforms with multiple inputs may halt due to incorrect BlockingRowSet removal in BaseTransform::inputRowSets

[fskorgen]

Apache Hop version?

2.18

Java version?

21

Operating system

Windows

What happened?

We have a MultiUnion transform supporting multiple inputs with different metadata. This transform has worked fine for many years, but since version 2.18 it can sometimes halt because the wrong BlockingRowSet is removed when processing completes. This appears to be caused by issues in the following methods:

BaseRowSet::compareTo
BaseRowSet::equals
BaseRowSet::hashCode

The problem is that the following fields are not unique:

• remoteHopServerName
• destinationTransformName
• destinationTransformCopy

As a result, RowSet instances may be incorrectly identified as equal, leading to removal of the wrong element from the internal ArrayList.

Including 'originTransformName' in the comparison logic ensures uniqueness and results in the correct BlockingRowSet being removed.

The following change fixed the problem for me (BaseRowSet):

/**
 * Compares using the target transforms and copy, not the source. That way, re-partitioning is
 * always done in the same way.
 */
@Override
public int compareTo(@NonNull IRowSet rowSet) {
  lock.readLock().lock();
  String target;

  try {
    target =
        remoteHopServerName
            + "."
            + destinationTransformName
            + "."
            + destinationTransformCopy.intValue()
            + "."
            + originTransformName; // ADDED

  } finally {
    lock.readLock().unlock();
  }

  String comp =
      rowSet.getRemoteHopServerName()
          + "."
          + rowSet.getDestinationTransformName()
          + "."
          + rowSet.getDestinationTransformCopy()
          + "."
          + rowSet.getOriginTransformName(); // ADDED

  return target.compareTo(comp);
}

@Override
public boolean equals(Object o) {
  if (this == o) {
    return true;
  }
  if (!(o instanceof IRowSet other)) {
    return false;
  }

  return compareTo(other) == 0;
}

@Override
public int hashCode() {
  return Objects.hash(
      remoteHopServerName,
      destinationTransformName,
      destinationTransformCopy.get(),
      originTransformName); // ADDED
}

Issue Priority

Priority: 1

Issue Component

Component: Transforms

[hansva]

Feel free to make a PR

[mattcasters]

Or we could revert @lance's 3848430 currently in HEAD

[leehaut]

I'm fine with reverting my change (3848430) for now. It seems the current approach may have some edge cases or side effects.

.take-issue

[leehaut]

Apologies, this was caused by my custom implementation of hashCode/equals. I've removed it for now and reverted to the default behavior. (BaseRowSet)