[Bug] [RLLIB] SampleBatch.concat_samples has incorrect max_seq_len logic

[mvindiola1]

Search before asking

• I searched the issues and found no similar issues.

Ray Component

RLlib

What happened + What you expected to happen

SampleBatch.concat_samples uses the first sample batch in the list of samples to determine the max_seq_len. This value is not updated after initialization which makes it possible that it does not reflect the actual max_seq_len in the list of samples.

Versions / Dependencies

ray - master branch

Reproduction script

def test_concat_max_seq_len(self):
    """Tests, SampleBatches.concat() and ...concat_samples()."""
    s1 = SampleBatch({
        "a": np.array([1, 2, 3]),
        "b": {
            "c": np.array([4, 5, 6])
        },
        SampleBatch.SEQ_LENS: [1, 2]
    })
    s2 = SampleBatch({
        "a": np.array([2, 3, 4]),
        "b": {
            "c": np.array([5, 6, 7])
        },
        SampleBatch.SEQ_LENS: [3]
    })
    concatd = SampleBatch.concat_samples([s1, s2])
    check(concatd.max_seq_len, 3)

Anything else

No response

Are you willing to submit a PR?

• Yes I am willing to submit a PR!