Wild Time Benchmarks and Small Memory Hack #363
Conversation
…to keep the buffer smaller
Coverage reportThe coverage rate went from
Diff Coverage details (click to unfold)src/renate/benchmark/experiment_config.py
src/renate/benchmark/datasets/vision_datasets.py
|
| @@ -317,6 +318,8 @@ def _get_normalize_transform(dataset_name): | |||
|
|
|||
| def train_transform(dataset_name: str) -> Optional[Callable]: | |||
| """Returns a transform function to be used in the training.""" | |||
| if dataset_name == "fmow": | |||
| return FMoW.default_transform | |||
There was a problem hiding this comment.
There seems to exist a default_transform(datasetname) in the package. Any reason to go this route?
| momentum: float = 0.0, # TODO: fix problem that occurs when removing this | ||
| ) -> Callable: | ||
| if optimizer == "AdamW": | ||
| return partial(AdamW, lr=learning_rate, weight_decay=weight_decay) |
There was a problem hiding this comment.
can this written as:
partial(getattr(torch.optim, optimizer), lr=learning_rate, weight_decay=weight_decay)? Why the specific handling of AdamW?
There was a problem hiding this comment.
in principle there is no problem if we make it more general. however, how would that work with SGD + momentum? right now, this logic will only be triggered for AdamW and otherwise we will fall back to the standard optimizers (SGD, Adam), we have in the library.
Add files required to run experimentation with Wild Time Benchmarks.
Save PIL images rather than tensors in buffers to keep memory space lower.
Found bug:
optimizer_fndoesn't work as intended. If it exists, it must return an optimizer. We would like that it can exist but it either returns an optimizer orNone. Currently "fixed" by adding all arguments expected for an optimizer to the function.By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.