Skip to content

Commit

Permalink
Update README.md
Browse files Browse the repository at this point in the history
Add more compact code for Approach 2
  • Loading branch information
Tony-Y committed Jan 24, 2024
1 parent 9abeaa3 commit eb995c6
Showing 1 changed file with 14 additions and 0 deletions.
14 changes: 14 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -104,6 +104,20 @@ for epoch in range(1,num_epochs+1):
lr_scheduler.step()
```

This code can be rewritten more compactly:

```python
for epoch in range(1,num_epochs+1):
for iter, batch in enumerate(dataloader):
optimizer.zero_grad()
loss = ...
loss.backward()
optimizer.step()
with warmup_scheduler.dampening():
if iter + 1 == len(dataloader):
lr_scheduler.step()
```

### Warmup Schedules

#### Manual Warmup
Expand Down

0 comments on commit eb995c6

Please sign in to comment.