Add dynamic batch size for VALLE #19
Conversation
models/tts/valle/valle_dataset.py
Outdated
| @@ -82,6 +93,10 @@ def __getitem__(self, index): | |||
|
|
|||
| return single_feature | |||
|
|
|||
| def get_num_frames(self, index): | |||
| utt_info = self.metadata[index] | |||
| return int(utt_info['Duration'] * 75) | |||
There was a problem hiding this comment.
What's the meaning of 75? Is it a fixed parameter or a variable parameter?
There was a problem hiding this comment.
it is a fixed parameter. 75 means 1s have 75 tokens for encodec
There was a problem hiding this comment.
It would be better to move this parameter to the config file because it relies on the codec.
models/tts/valle/valle_trainer.py
Outdated
| if not self.cfg.train.use_dynamic_batchsize: | ||
| return super()._build_dataloader() | ||
| Dataset, Collator = self._build_dataset() | ||
| train_dataset = Dataset(self.cfg, self.cfg.dataset[0], is_valid=False) |
There was a problem hiding this comment.
Does it only work on dataset[0] instead of all elements of the dataset list?
There was a problem hiding this comment.
It only works for dynamic batchsize training for VALLE, if not, it will use super()._build_dataloader()
There was a problem hiding this comment.
cfg.dataset specifics a dataset list to be processed, but your code seems to only process the first dataset (self.cfg.dataset[0]). How about other datasets?
models/tts/base/tts_trainer.py
Outdated
| if (self.model_type == "VALLE") and (not self.cfg.train.use_dynamic_batchsize): | ||
| ( | ||
| self.train_dataloader, | ||
| self.valid_dataloader, | ||
| ) = self.accelerator.prepare( | ||
| self.train_dataloader, | ||
| self.valid_dataloader, | ||
| ) |
There was a problem hiding this comment.
This judgment statement means that if the model_type is VITS or Fastspeech2， the dataloader will not be prepared. It is wrong!
There was a problem hiding this comment.
rewrite accelerator prepare in valle
Add dynamic batch size for VALLE