Discussions

• 2 You must be logged in to vote

  🤖

  DataModule: Explicit Barrier Needed Between prepare_data and setup for Multi-GPU

  piyushumate asked Mar 7, 2025 in DDP / multi-GPU / multi-node · Unanswered

  

  

  0
• 1 You must be logged in to vote

  🤖

  Train in DDP Multi GPU and launch Validation on Single GPU during training

  thad75 asked Jan 11, 2023 in DDP / multi-GPU / multi-node · Unanswered

  

  

  

  

  2
• 7 You must be logged in to vote

  🤖

  PartialFC: Distributing only part of the model

  w2ex asked Nov 16, 2022 in DDP / multi-GPU / multi-node · Unanswered

  

  

  1
• 1 You must be logged in to vote

  ⚡

  How can I run validation on a different worker or node than training?

  jlu-figma asked Mar 3, 2025 in Lightning Trainer API: Trainer, LightningModule, LightningDataModule · Unanswered

  

  

  0
• 1 You must be logged in to vote

  ⚡

  What is trainer/global_step in wandb logging?

  edmcman asked Oct 30, 2024 in Lightning Trainer API: Trainer, LightningModule, LightningDataModule · Unanswered

  

  

  

  

  5
• 4 You must be logged in to vote

  ⚡

  Gradient checkpointing with DDP in a loop

  shivammehta25 asked Nov 11, 2021 in Lightning Trainer API: Trainer, LightningModule, LightningDataModule · Unanswered

  

  

  

  

  10
• 1 You must be logged in to vote

  ⚡

  Dynamically Setting out_dim Based on LightningDataModule in LightningCLI

  aditya0by0 asked Feb 25, 2025 in Lightning Trainer API: Trainer, LightningModule, LightningDataModule · Answered

  

  

  2
• 9 You must be logged in to vote

  ⚡

  Does pytorch lightning divide the loss by number of gradient accumulation steps?

  offchan42 asked Mar 11, 2023 in Lightning Trainer API: Trainer, LightningModule, LightningDataModule · Answered

  

  

  

  

  4
• 2 You must be logged in to vote

  ⚡

  self.global_step understanding

  ajinkyaambatwar asked Jun 16, 2021 in Lightning Trainer API: Trainer, LightningModule, LightningDataModule · Unanswered

  

  

  

  

  5
• 2 You must be logged in to vote

  🤖

  Training on SLURM with multiple GPUs

  matval asked Jul 2, 2023 in DDP / multi-GPU / multi-node · Unanswered

  

  

  

  

  3
• 1 You must be logged in to vote

  🤖

  DDP freez becasue of torchsummary.summary

  mehran66 asked Feb 20, 2025 in DDP / multi-GPU / multi-node · Unanswered

  

  

  0
• 1 You must be logged in to vote

  ⚡

  Memory Leak Issue Using PyTorch Profiler with PyTorch Lightning

  SinaTavakoli asked Feb 18, 2025 in Lightning Trainer API: Trainer, LightningModule, LightningDataModule · Unanswered

  

  

  0
• 1 You must be logged in to vote

  ⚡

  How to force finishing the training loop and moving to the validation loop from within the trainer?

  motykatomasz asked Feb 13, 2025 in Lightning Trainer API: Trainer, LightningModule, LightningDataModule · Unanswered

  

  

  0
• 1 You must be logged in to vote

  ⚡

  Customizing LightningCLI for multiple Test Stages

  Amerehei asked Feb 9, 2025 in Lightning Trainer API: Trainer, LightningModule, LightningDataModule · Unanswered

  

  

  1
• 1 You must be logged in to vote

  💡

  Add ExecuTorch support

  byjlw started Feb 8, 2025 in Idea pool

  

  

  0
• 2 You must be logged in to vote

  ⚡

  How the model.train() and model.eval() switch work in Pytorch Lightning?

  lndip asked Sep 10, 2024 in Lightning Trainer API: Trainer, LightningModule, LightningDataModule · Unanswered

  

  

  

  

  2
• 3 You must be logged in to vote

  ⚡

  How to use LightningCLI to resume training from checkpoint

  KirillShmilovich asked Feb 22, 2023 in Lightning Trainer API: Trainer, LightningModule, LightningDataModule · Unanswered

  

  

  

  

  15
• 1 You must be logged in to vote

  🤖

  Ensuring _fair_ validation on epoch end in multi-gpu setting

  strentom asked Jan 30, 2025 in DDP / multi-GPU / multi-node · Unanswered

  

  

  0
• 1 You must be logged in to vote

  💬

  How to log the best validation metrics

  FDUguchunhui started Jan 27, 2025 in General

  

  

  0
• 2 You must be logged in to vote

  ⚡

  Is it possible to permanently move a complete dataset to GPU memory before training?

  NiklasKappel asked Jan 20, 2025 in Lightning Trainer API: Trainer, LightningModule, LightningDataModule · Unanswered

  

  

  0
• 4 You must be logged in to vote

  🤖

  Warning: find_unused_parameters=True was specified in DDP constructor, but did not find any unused parameters. This flag results in an extra traversal of the autograd graph every iteration, which can adversely affect performance.

  strategy: ddp DistributedDataParallel

  athenawisdoms asked Mar 31, 2021 in DDP / multi-GPU / multi-node · Answered

  

  

  

  

  19
• 1 You must be logged in to vote

  💬

  How to log on test end

  Boltzmachine started Jan 14, 2025 in General

  

  

  0
• 1 You must be logged in to vote

  🤖

  Clear example for custom DistributedSampler

  Rizhiy asked May 20, 2021 in DDP / multi-GPU / multi-node · Unanswered

  

  

  

  

  10
• 1 You must be logged in to vote

  ⚡

  Default batch transfer behavior

  ziw-liu asked Jan 9, 2025 in Lightning Trainer API: Trainer, LightningModule, LightningDataModule · Unanswered

  

  

  1
• 1 You must be logged in to vote

  ⚡

  About average in progress bar with on_epoch

  logging Related to the `LoggerConnector` and `log()` progress bar: tqdm

  Nokimann asked Aug 17, 2022 in Lightning Trainer API: Trainer, LightningModule, LightningDataModule · Answered

  

  

  

  

  7