Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add sampling #7

Open
wants to merge 34 commits into
base: main
Choose a base branch
from
Open

Add sampling #7

wants to merge 34 commits into from

Commits on Jan 13, 2021

  1. add basic sampling code

    sid committed Jan 13, 2021
    Configuration menu
    Copy the full SHA
    99d11e1 View commit details
    Browse the repository at this point in the history

  2. Merge remote-tracking branch 'origin/main' into sample

    sid committed Jan 13, 2021
    Configuration menu
    Copy the full SHA
    8bb3322 View commit details
    Browse the repository at this point in the history

  3. add prediction input / output fns

    sid committed Jan 13, 2021
    Configuration menu
    Copy the full SHA
    b9fd039 View commit details
    Browse the repository at this point in the history

Commits on Jan 14, 2021

  1. get sample_autoregressive working

    @ConnorJL
    ConnorJL committed Jan 14, 2021
    Configuration menu
    Copy the full SHA
    cf76c6c View commit details
    Browse the repository at this point in the history

  2. truncate text tokens properly

    @ConnorJL
    ConnorJL committed Jan 14, 2021
    Configuration menu
    Copy the full SHA
    c7ff6c4 View commit details
    Browse the repository at this point in the history

  3. log model params to tensorboard

    @ConnorJL
    ConnorJL committed Jan 14, 2021
    Configuration menu
    Copy the full SHA
    4d51fd9 View commit details
    Browse the repository at this point in the history

  4. add vae decoding and write to jpeg

    @kingoflolz
    kingoflolz committed Jan 14, 2021
    Configuration menu
    Copy the full SHA
    346871f View commit details
    Browse the repository at this point in the history

  5. unshift image outputs at decode time

    @kingoflolz
    kingoflolz committed Jan 14, 2021
    Configuration menu
    Copy the full SHA
    d13c330 View commit details
    Browse the repository at this point in the history

  6. dirty hack to use vae decoder params when training dalle

    @kingoflolz
    kingoflolz committed Jan 14, 2021
    Configuration menu
    Copy the full SHA
    f8a7449 View commit details
    Browse the repository at this point in the history

Commits on Jan 17, 2021

  1. Move initialize_vae_weights to after lowering

    @leogao2
    leogao2 committed Jan 17, 2021
    Configuration menu
    Copy the full SHA
    ff56d12 View commit details
    Browse the repository at this point in the history

  2. fix vae checkpoint load in training

    @ConnorJL
    ConnorJL committed Jan 17, 2021
    Configuration menu
    Copy the full SHA
    4c4e0e0 View commit details
    Browse the repository at this point in the history

Commits on Jan 18, 2021

  1. fix parameter count logging

    @ConnorJL
    ConnorJL committed Jan 18, 2021
    Configuration menu
    Copy the full SHA
    2c14bde View commit details
    Browse the repository at this point in the history

  2. fix image vocab size

    @ConnorJL
    ConnorJL committed Jan 18, 2021
    Configuration menu
    Copy the full SHA
    130c26e View commit details
    Browse the repository at this point in the history

Commits on Apr 2, 2021

  1. add missing dep

    @lucidrains
    lucidrains committed Apr 2, 2021
    Configuration menu
    Copy the full SHA
    42e7677 View commit details
    Browse the repository at this point in the history

Commits on Apr 3, 2021

  1. ready tests for rewiring

    @lucidrains
    lucidrains committed Apr 3, 2021
    Configuration menu
    Copy the full SHA
    fcab065 View commit details
    Browse the repository at this point in the history

  2. no 3.9 for tests

    @lucidrains
    lucidrains committed Apr 3, 2021
    Configuration menu
    Copy the full SHA
    3858108 View commit details
    Browse the repository at this point in the history

  3. make DALLE-mtf work with text and image logits created with separate … 

    …projections
    @lucidrains
    lucidrains committed Apr 3, 2021
    Configuration menu
    Copy the full SHA
    002343a View commit details
    Browse the repository at this point in the history

  4. add axial positional embedding

    @lucidrains
    lucidrains committed Apr 3, 2021
    Configuration menu
    Copy the full SHA
    037aefb View commit details
    Browse the repository at this point in the history

  5. cleanup

    @lucidrains
    lucidrains committed Apr 3, 2021
    Configuration menu
    Copy the full SHA
    e203de4 View commit details
    Browse the repository at this point in the history

  6. shift by text vocab size (not text seq len)

    @lucidrains
    lucidrains committed Apr 3, 2021
    Configuration menu
    Copy the full SHA
    5ecb496 View commit details
    Browse the repository at this point in the history

  7. make sure sampling can be forced to never start below a certain minim… 

    …um starting position, useful when the text tokens contain padding tokens of 0
    @lucidrains
    lucidrains committed Apr 3, 2021
    Configuration menu
    Copy the full SHA
    ec5c298 View commit details
    Browse the repository at this point in the history

  8. add unique pad token ids, which obviates the need to remove attention… 

    … and cross entropy loss from text padding tokens
    @lucidrains
    lucidrains committed Apr 3, 2021
    Configuration menu
    Copy the full SHA
    3492d80 View commit details
    Browse the repository at this point in the history

Commits on Apr 4, 2021

  1. fix text_vocab_dim error

    @sdtblck
    sdtblck committed Apr 4, 2021
    Configuration menu
    Copy the full SHA
    3b76849 View commit details
    Browse the repository at this point in the history

  2. Update models.py

    @sdtblck
    sdtblck committed Apr 4, 2021
    Configuration menu
    Copy the full SHA
    ef96537 View commit details
    Browse the repository at this point in the history

  3. fix bug

    @lucidrains
    lucidrains committed Apr 4, 2021
    Configuration menu
    Copy the full SHA
    a1634d4 View commit details
    Browse the repository at this point in the history

  4. fix bug with shift

    @lucidrains
    lucidrains committed Apr 4, 2021
    Configuration menu
    Copy the full SHA
    a3d2115 View commit details
    Browse the repository at this point in the history

  5. Add adam weight decay optimizer

    @sdtblck
    sdtblck committed Apr 4, 2021
    Configuration menu
    Copy the full SHA
    5f279ad View commit details
    Browse the repository at this point in the history

  6. fix args.steps_per_checkpoint

    @sdtblck
    sdtblck committed Apr 4, 2021
    Configuration menu
    Copy the full SHA
    8db0d55 View commit details
    Browse the repository at this point in the history

  7. add variable scope

    @lucidrains
    lucidrains committed Apr 4, 2021
    Configuration menu
    Copy the full SHA
    d3112ca View commit details
    Browse the repository at this point in the history

  8. do an early copy of inputs for labels

    @lucidrains
    lucidrains committed Apr 4, 2021
    Configuration menu
    Copy the full SHA
    33de203 View commit details
    Browse the repository at this point in the history

  9. tweak

    @lucidrains
    lucidrains committed Apr 4, 2021
    Configuration menu
    Copy the full SHA
    4dbf727 View commit details
    Browse the repository at this point in the history

Commits on Apr 5, 2021

  1. fix initial positions at text_seq_len

    @lucidrains
    lucidrains committed Apr 5, 2021
    Configuration menu
    Copy the full SHA
    10dc024 View commit details
    Browse the repository at this point in the history

  2. more cleanup

    @lucidrains
    lucidrains committed Apr 5, 2021
    Configuration menu
    Copy the full SHA
    9dacd35 View commit details
    Browse the repository at this point in the history

Commits on Apr 6, 2021

  1. make sure axial positional embedding is correctly shifted by one due … 

    …to <bos>
    @lucidrains
    lucidrains committed Apr 6, 2021
    Configuration menu
    Copy the full SHA
    c073fbc View commit details
    Browse the repository at this point in the history