token_next_match and token_next semantics don't match

[kcrossen]

If token_next_match is given an index (first param) that already matches the next token being sought, the token at the given index is returned.

If token_next is given an index, it always returns the succeeding 'real' token, if any.

In other words, the functionality implied by the analogous name, is in practice different.

This doesn't work as expected:

sql = u"""
SELECT tbl_alias_a.col_a AS alias_a,
       tbl_c.col_c AS alias_c
FROM tbl_a AS tbl_alias_a
join (SELECT col_b AS col_c, tbl_e.col_d AS col_e FROM tbl_b JOIN tbl_d AS tbl_e) AS tbl_c
ON (tbl_alias_a.id = tbl_c.id)
WHERE (tbl_c.col_n = '☠')
""".strip()

statement = sqlparse.parse(sql)[0]
select = statement.token_first()
result_cols = statement.token_next(statement.token_index(select))
select_from = \
    statement.token_next_match(
        statement.token_index(select), Keyword, 'FROM')
select_join_a = \
    statement.token_next_match(
        statement.token_index(select_from), Keyword, 'JOIN')
select_join_b = \
    statement.token_next_match(
        statement.token_index(select_join_a), Keyword, 'JOIN')

This does by virtue of pre-offsetting the index, and returns None:

select_join_b = \
    statement.token_next_match(
        statement.token_next(statement.token_index(select_join_a)), Keyword, 'JOIN')

[vmuriart]

which of these behaviors would be the preferred one? I am thinking token_next's behavior is better.

[kcrossen]

Likely, yes.