Mistake in GLUE code example

[Valahaar]

Hi, I found out there is a mistake in how the total_steps is computed in https://pytorch-lightning.readthedocs.io/en/stable/notebooks/lightning_examples/text-transformers.html

For instance, in the setup() method, we have:

# Calculate total steps
tb_size = self.hparams.train_batch_size * max(1, self.trainer.gpus)
ab_size = self.trainer.accumulate_grad_batches * float(self.trainer.max_epochs)
self.total_steps = (len(train_loader.dataset) // tb_size) // ab_size

If I'm not mistaken, it should be something along the lines of:

# Calculate total steps
tb_size = self.hparams.train_batch_size * max(1, self.trainer.gpus)
ab_size = tb_size * self.trainer.accumulate_grad_batches
self.total_steps = int((len(train_loader.dataset) / ab_size) * float(self.trainer.max_epochs))

In the first version, on MRPC (3668 instances), with 30 epochs, 32 batch size, 1 gpu and 1 batch accumulation total_steps amounts to 3; in the second version, it amounts to 3438.

cc @Borda

[Borda]

@phlippe mind have look? 🐰

[phlippe]

@Borda I think the issue is in a tutorial which is not from the UvA-DL course. But the suggested change looks good to me, the original total step calculation seems to have a bug.

[Borda]

@Valahaar mind sending a PR to fix it? :)

[Valahaar]

Sure! At the time I opened this issue I didn't know this repo existed, and the "Edit on GitHub" link on the page is broken.

Before I proceed, can you confirm that the only file that needs editing is https://github.com/PyTorchLightning/lightning-tutorials/blob/main/lightning_examples/text-transformers/text-transformers.py ? Thanks @Borda :)

And I guess the updated field in the .meta.yml file?

[Borda]

And I guess the updated field in the .meta.yml file?

correct 🐰