Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Improved metadata #240

Closed
wants to merge 40 commits into from
Closed

Improved metadata #240

wants to merge 40 commits into from

Commits on Apr 18, 2024

  1. Template for Clay V1 model (#221) 

    - Add starter code for clay v1 model
- Add `model_clay_v1.py` that handles images of different size & spectral bands
- Add `metadata.yaml` that contains meta information of satellite data, like wavelengths & gsd
- Add cls tokens to ClayMAE
    @srmsoumya
    srmsoumya committed Apr 18, 2024
    Configuration menu
    Copy the full SHA
    21338d3 View commit details
    Browse the repository at this point in the history

Commits on Apr 19, 2024

  1. Remove Pixelify from Decoder

    SRM committed Apr 19, 2024
    Configuration menu
    Copy the full SHA
    74c65ad View commit details
    Browse the repository at this point in the history

  2. Merge branch 'main' into dev

    SRM committed Apr 19, 2024
    Configuration menu
    Copy the full SHA
    71645e9 View commit details
    Browse the repository at this point in the history

  3. Merge branch 'dev' into multinode-training

    SRM committed Apr 19, 2024
    Configuration menu
    Copy the full SHA
    d5ed36f View commit details
    Browse the repository at this point in the history

  4. Modify lightning module for ClayMAE

    SRM committed Apr 19, 2024
    Configuration menu
    Copy the full SHA
    b6f31ab View commit details
    Browse the repository at this point in the history

Commits on Apr 20, 2024

  1. Use lightning config.yaml

    SRM committed Apr 20, 2024
    Configuration menu
    Copy the full SHA
    e6a276e View commit details
    Browse the repository at this point in the history

  2. Modify dataset & datamodule to read metadata.yaml & npz files instead… 

    … of tifs
    SRM committed Apr 20, 2024
    Configuration menu
    Copy the full SHA
    7b31a09 View commit details
    Browse the repository at this point in the history

Commits on Apr 21, 2024

  1. Modify the model to handle EODataset

    SRM committed Apr 21, 2024
    Configuration menu
    Copy the full SHA
    261cd14 View commit details
    Browse the repository at this point in the history

  2. Pass metadata path to both ClayDataModule & ClayMAEModule

    SRM committed Apr 21, 2024
    Configuration menu
    Copy the full SHA
    e0e31a6 View commit details
    Browse the repository at this point in the history

  3. Add model variants tiny, small, medium, large

    SRM committed Apr 21, 2024
    Configuration menu
    Copy the full SHA
    3bcf7e8 View commit details
    Browse the repository at this point in the history

  4. Lr 1e-3 to 1e-5

    SRM committed Apr 21, 2024
    Configuration menu
    Copy the full SHA
    801ddea View commit details
    Browse the repository at this point in the history

  5. Add teacher encoder

    SRM committed Apr 21, 2024
    Configuration menu
    Copy the full SHA
    fa9edcf View commit details
    Browse the repository at this point in the history

  6. Pass band order as argument to get rgb for teacher model. Replace tra… 

    …nsforms with v2.transforms
    SRM committed Apr 21, 2024
    Configuration menu
    Copy the full SHA
    d0d9544 View commit details
    Browse the repository at this point in the history

  7. Add 0.75 weight to reconstruction loss

    SRM committed Apr 21, 2024
    Configuration menu
    Copy the full SHA
    9587232 View commit details
    Browse the repository at this point in the history

Commits on Apr 22, 2024

  1. Pass rgb indices in the metadata.yaml file

    SRM committed Apr 22, 2024
    Configuration menu
    Copy the full SHA
    880cbf7 View commit details
    Browse the repository at this point in the history

Commits on Apr 23, 2024

  1. Don't add bias for decoder side of Dynamic EMbedding

    SRM committed Apr 23, 2024
    Configuration menu
    Copy the full SHA
    afe1460 View commit details
    Browse the repository at this point in the history

  2. Freeze the teacher model on start of every train epoch

    SRM committed Apr 23, 2024
    Configuration menu
    Copy the full SHA
    40af48a View commit details
    Browse the repository at this point in the history

  3. pre-commit fix lint errors

    @weiji14
    weiji14 committed Apr 23, 2024
    Configuration menu
    Copy the full SHA
    b68c5f9 View commit details
    Browse the repository at this point in the history

Commits on Apr 24, 2024

  1. ➕ Add timm 

    PyTorch Image Models!
    @weiji14
    weiji14 committed Apr 24, 2024
    Configuration menu
    Copy the full SHA
    a317860 View commit details
    Browse the repository at this point in the history

  2. 🔇 Silence set_float32_matmul_precision tip and a print statement

    @weiji14
    weiji14 committed Apr 24, 2024
    Configuration menu
    Copy the full SHA
    4aa9fc0 View commit details
    Browse the repository at this point in the history

  3. Modify config & datamodule to run on multi-gpu mode

    SRM committed Apr 24, 2024
    Configuration menu
    Copy the full SHA
    1e4ea81 View commit details
    Browse the repository at this point in the history

Commits on Apr 25, 2024

  1. add rec_loss & rep_loss to the logger

    SRM committed Apr 25, 2024
    Configuration menu
    Copy the full SHA
    80a0778 View commit details
    Browse the repository at this point in the history

  2. Add a temporary env file for v1 runs

    SRM committed Apr 25, 2024
    Configuration menu
    Copy the full SHA
    c4c8e8d View commit details
    Browse the repository at this point in the history

  3. Fix lr to 1e-5

    SRM committed Apr 25, 2024
    Configuration menu
    Copy the full SHA
    701f412 View commit details
    Browse the repository at this point in the history

  4. Add multinode sbatch script

    SRM committed Apr 25, 2024
    Configuration menu
    Copy the full SHA
    7f8e633 View commit details
    Browse the repository at this point in the history

Commits on Apr 26, 2024

  1. Sampler to load data as batches of sensor (#233) 

    - First working version of sampler

---------

Co-authored-by: Daniel Wiesmann <daniel@wiesmann.pt>
    @srmsoumya @yellowcap
    srmsoumya and yellowcap committed Apr 26, 2024
    Configuration menu
    Copy the full SHA
    6df5aca View commit details
    Browse the repository at this point in the history

  2. Add a sampler to load multi sensor data

    SRM committed Apr 26, 2024
    Configuration menu
    Copy the full SHA
    cc605c6 View commit details
    Browse the repository at this point in the history

  3. Write collate, Sampler returns one element, use BatchSampler

    SRM committed Apr 26, 2024
    Configuration menu
    Copy the full SHA
    11fe5a8 View commit details
    Browse the repository at this point in the history

  4. Simplify names of landsat

    SRM committed Apr 26, 2024
    Configuration menu
    Copy the full SHA
    819c25e View commit details
    Browse the repository at this point in the history

Commits on Apr 27, 2024

  1. Sampler returns batches of input, use_distributed_sampler: False in l… 

    …ightning
    SRM committed Apr 27, 2024
    Configuration menu
    Copy the full SHA
    757132f View commit details
    Browse the repository at this point in the history

  2. Add support to provide images of different size as input to the model

    SRM committed Apr 27, 2024
    Configuration menu
    Copy the full SHA
    9628f86 View commit details
    Browse the repository at this point in the history

Commits on Apr 30, 2024

  1. Add jsonargparse[signatures]>=4.27.7 to env yaml, required by lightni… 

    …ng CLI
    SRM committed Apr 30, 2024
    Configuration menu
    Copy the full SHA
    f0efcb4 View commit details
    Browse the repository at this point in the history

  2. Improved metadata 

    This needs another sanity check, and the actual band stats are in computation right now. Should update tomorrow morning
    @yellowcap
    yellowcap committed Apr 30, 2024
    Configuration menu
    Copy the full SHA
    8f69719 View commit details
    Browse the repository at this point in the history

  3. [pre-commit.ci] auto fixes from pre-commit.com hooks 

    for more information, see https://pre-commit.ci
    @pre-commit-ci @yellowcap
    pre-commit-ci[bot] authored and yellowcap committed Apr 30, 2024
    Configuration menu
    Copy the full SHA
    0b15e83 View commit details
    Browse the repository at this point in the history

  4. Update configs/metadata.yaml 

    Co-authored-by: Wei Ji <23487320+weiji14@users.noreply.github.com>
    @yellowcap @weiji14
    yellowcap and weiji14 committed Apr 30, 2024
    Configuration menu
    Copy the full SHA
    8f6386e View commit details
    Browse the repository at this point in the history

  5. Update configs/metadata.yaml 

    Co-authored-by: Wei Ji <23487320+weiji14@users.noreply.github.com>
    @yellowcap @weiji14
    yellowcap and weiji14 committed Apr 30, 2024
    Configuration menu
    Copy the full SHA
    698b692 View commit details
    Browse the repository at this point in the history

  6. Update configs/metadata.yaml 

    Co-authored-by: Wei Ji <23487320+weiji14@users.noreply.github.com>
    @yellowcap @weiji14
    yellowcap and weiji14 committed Apr 30, 2024
    Configuration menu
    Copy the full SHA
    e04338b View commit details
    Browse the repository at this point in the history

  7. Update statistics on each band for each platform

    @yellowcap
    yellowcap committed Apr 30, 2024
    Configuration menu
    Copy the full SHA
    4c0c846 View commit details
    Browse the repository at this point in the history

  8. Fixed metadata by excluding nodata

    @yellowcap
    yellowcap committed Apr 30, 2024
    Configuration menu
    Copy the full SHA
    6107496 View commit details
    Browse the repository at this point in the history

  9. Replace legacy np.random.shuffle call with np.random.Generator 

    Fix NPY002 lint error, xref https://docs.astral.sh/ruff/rules/numpy-legacy-random
    @weiji14
    weiji14 committed Apr 30, 2024
    Configuration menu
    Copy the full SHA
    5e9ccfe View commit details
    Browse the repository at this point in the history