Fix tokens_per_sec calculation in SpeedMonitor

[konstantinjdobler]

The calculations for tokens_per_sec (and device/tokens_per_sec) in SpeedMonitor are wrong, specifically an overestimation by a factor of window_size (default to 100). To fix, we need to multiply samples_per_second by the mean number of tokens per sample (instead of multiplying by the sum of the number of tokens per sample over the last window_size batches, which is done currently).

For comparison, see the reference implementation:

# Compute token stats if dataloader.dataset has max_seq_len. Assumes no padding.
try:
    max_seq_len = state.dataloader.dataset.max_seq_len  # type: ignore
    # Only applicable to seq data / models
    logger.log_metrics({'throughput/tokens_per_sec': samples_per_sec * max_seq_len})
    logger.log_metrics({'throughput/device/tokens_per_sec': dev_samples_per_sec * max_seq_len})

https://github.com/mosaicml/composer/blob/f2a2dc820cb75023b9eb7c46fdfd25273712abd0/composer/callbacks/speed_monitor.py#L279-L284

[carmocca]

Thank you!