Can current token see next token in encoder for code completion ?

[mf1832146]

Hello, I have not found there exists a mask matrix in encoder for code completion, so every node of ast will participate in self attention computation?

[jxzhn]

I found it here (cc/main/src/model.py: line 181)

        w = w * b - INF * (1 - b)

where b is from lower triangular matrix bias defined on lines 77-79

        self.register_buffer(
            "bias", torch.tril(torch.ones(n_ctx, n_ctx)).view(1, 1, n_ctx, n_ctx)
        )

[mf1832146]

@jxzhn Thanks, it looks right!!!

[serjtroshin]

@jxzhn thanks for helping, I am closing this issue