Skip to content

issues Search Results · repo:lucidrains/CoCa-pytorch language:Python

Filter by

18 results
 (294 ms)

18 results

inlucidrains/CoCa-pytorch (press backspace or delete to remove)

lucidrains/CoCa-pytorch
contrastive_labels - dist batch size isn't correct

https://github.com/lucidrains/CoCa-pytorch/blob/edee92c74e311ccfa4a0024412fd991c98aff5fd/coca_pytorch/coca_pytorch.py#L532 fyi the dist batch size isn t correct torch.arange(batch, device=device) - torch.arange(text_latents.shape[0], ...
  • gillotte
  • Opened 
    on Apr 14, 2024
  • #20

lucidrains/CoCa-pytorch
Reproduce Problem

Thanks for this repo. I m using my own dataset for pre-training via CoCa, and I found that the contrastive loss output from each batch is basically unchanged, is it the contrastive_label that needs to ...
  • Camellia-tx
  • 2
  • Opened 
    on Oct 23, 2023
  • #19

lucidrains/CoCa-pytorch
pad_dim_to() is not implemented

The implementation for pad_dim_to() seems to be missing from coca_pytorch.py. https://github.com/lucidrains/CoCa-pytorch/blob/0d3f2521a5126e7cc7e58a2374754502944562c7/coca_pytorch/coca_pytorch.py#L29C16-L29C26 ...
  • aspartate
  • 3
  • Opened 
    on Oct 10, 2023
  • #18

lucidrains/CoCa-pytorch
register_buffer for masks and position encodings breaks DDP

Hi! Unfortunately, using buffers to cache masks and pos encodings fails when running with DDP. https://github.com/lucidrains/CoCa-pytorch/blob/790415ceaf2af3e937cf2dc16826ccef91ffddfa/coca_pytorch/coca_pytorch.py#L116-L130 ...
  • gshaikov-paige
  • 14
  • Opened 
    on Sep 2, 2023
  • #17

lucidrains/CoCa-pytorch
Residual connection misssing?

[RESOLVED] Residual connection is in the CoCa class
  • gshaikov-paige
  • Opened 
    on Aug 31, 2023
  • #16

lucidrains/CoCa-pytorch
convolution encoder better result then vit

Thank you for this work. The generalizing ability of neural networks based on convolution layers is much greater. class Model(torch.nn.Module): def __init__(self): super(Model, self).__init__() ...
  • Alexandr1111111
  • 13
  • Opened 
    on Jan 24, 2023
  • #14

lucidrains/CoCa-pytorch
LayerNorm after attentional_pooler

Hi, if I understand correctly, there is a single LayerNorm that is applied to all the queries output by the attentional pooler, however in the paper it seems like they use a different one for the one query ...
  • gpucce
  • 1
  • Opened 
    on Jan 21, 2023
  • #13

lucidrains/CoCa-pytorch
Image tokens dimension mismatch with the layernorm bias dimension

File /home/usr/anaconda3/envs/varpt13/lib/python3.8/site-packages/torch/nn/modules/module.py , line 1194, in _call_impl return forward_call(*input, **kwargs) File /home/usr/anaconda3/envs/varpt13/lib/python3.8/site-packages/coca_pytorch/coca_pytorch.py ...
  • szxuhongye
  • 12
  • Opened 
    on Jan 15, 2023
  • #11

lucidrains/CoCa-pytorch
Contrastive loss should be applied to L2-normed embeddings instead of layer normed?

Hi @lucidrains, thank you for the implementation. Just wanted to confirm this with you, based on your code we re normalizing the img embedding and text embedding respectively using a learnable Layer Norm ...
  • fedshyvana
  • 2
  • Opened 
    on Dec 4, 2022
  • #10

lucidrains/CoCa-pytorch
How to train the model using my own dataset?

Can someone tell me how to train the model using my own dataset? is it like below？But I have many images and texts... # train by giving CoCa your text and images with `return_loss = True` loss = coca( ...
  • keepcodeandsmile
  • 1
  • Opened 
    on Aug 23, 2022
  • #9
Issue origami icon

Learn how you can use GitHub Issues to plan and track your work.

Save views for sprints, backlogs, teams, or releases. Rank, sort, and filter issues to suit the occasion. The possibilities are endless.Learn more about GitHub Issues
ProTip! 
Press the
/
key to activate the search input again and adjust your query.
Issue origami icon

Learn how you can use GitHub Issues to plan and track your work.

Save views for sprints, backlogs, teams, or releases. Rank, sort, and filter issues to suit the occasion. The possibilities are endless.Learn more about GitHub Issues
ProTip! 
Press the
/
key to activate the search input again and adjust your query.
Issue search results · GitHub