Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

question about neg_out from blip pretrain files #617

Open
jumbokun opened this issue Dec 19, 2023 · 0 comments
Open

question about neg_out from blip pretrain files #617

jumbokun opened this issue Dec 19, 2023 · 0 comments

Comments

@jumbokun
Copy link

Hello Lavis team,

I kinda don't get why this snippet would deliever a negative text for each image

        # select a negative text for each image
        text_ids_neg = []
        text_atts_neg = []
        for b in range(bs):
            neg_idx = torch.multinomial(weights_i2t[b], 1).item()
            text_ids_neg.append(encoder_input_ids[neg_idx])
            text_atts_neg.append(text.attention_mask[neg_idx])

so encoder_input_ids is simply input ids of text..? You picked some token ids from that instead of picking up from queue?.. Could you please explain it a little bit?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant